diff options
Diffstat (limited to 'doc')
875 files changed, 16998 insertions, 11122 deletions
diff --git a/doc/.vale/gitlab/HeadingDepth.yml b/doc/.vale/gitlab/HeadingDepth.yml index f29ec0e4ac7..000baf633d7 100644 --- a/doc/.vale/gitlab/HeadingDepth.yml +++ b/doc/.vale/gitlab/HeadingDepth.yml @@ -10,4 +10,4 @@ link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html level: suggestion scope: raw raw: - - '(?<=\n)#{5,}\s.*' + - '(?<=\n)#{6,}\s.*' diff --git a/doc/.vale/gitlab/Uppercase.yml b/doc/.vale/gitlab/Uppercase.yml index 19e0fec6622..1948842d026 100644 --- a/doc/.vale/gitlab/Uppercase.yml +++ b/doc/.vale/gitlab/Uppercase.yml @@ -147,6 +147,7 @@ exceptions: - NPM - NTP - OCI + - OIDC - OKD - OKR - ONLY diff --git a/doc/.vale/gitlab/spelling-exceptions.txt b/doc/.vale/gitlab/spelling-exceptions.txt index 83e187fe1b5..b24608f1a1c 100644 --- a/doc/.vale/gitlab/spelling-exceptions.txt +++ b/doc/.vale/gitlab/spelling-exceptions.txt @@ -327,6 +327,7 @@ ETag ETags Etsy Excon +exfiltrate exfiltration ExifTool expirable @@ -392,6 +393,8 @@ Gitter GLab globals globbing +globstar +globstars Gmail Godep Golang @@ -1055,6 +1058,7 @@ unencoded unencoder unencodes unencrypted +unescaped unfollow unfollowed unfollows @@ -1074,6 +1078,7 @@ unoptimize unoptimized unoptimizes unoptimizing +unparsable unpatched unpause unprioritized diff --git a/doc/administration/audit_event_streaming.md b/doc/administration/audit_event_streaming.md index bbf4c0699ca..afc4926fec0 100644 --- a/doc/administration/audit_event_streaming.md +++ b/doc/administration/audit_event_streaming.md @@ -46,8 +46,8 @@ Users with the Owner role for a group can add streaming destinations for it: 1. Select **Add streaming destination** to show the section for adding destinations. 1. Enter the destination URL to add. 1. Optional. Locate the **Custom HTTP headers** table. -1. Ignore the **Active** checkbox because it isn't functional. To track progress on adding functionality to the **Active** checkbox, see the - [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/361925). +1. Ignore the **Active** checkbox because it isn't functional. To track progress on adding functionality to the + **Active** checkbox, see [issue 367509](https://gitlab.com/gitlab-org/gitlab/-/issues/367509). 1. Select **Add header** to create a new name and value pair. Enter as many name and value pairs as required. You can add up to 20 headers per streaming destination. 1. After all headers have been filled out, select **Add** to add the new streaming destination. @@ -169,8 +169,8 @@ To update a streaming destinations custom HTTP headers: 1. To the right of the item, select **Edit** (**{pencil}**). 1. Locate the **Custom HTTP headers** table. 1. Locate the header that you wish to update. -1. Ignore the **Active** checkbox because it isn't functional. To track progress on adding functionality to the **Active** checkbox, see the - [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/361925). +1. Ignore the **Active** checkbox because it isn't functional. To track progress on adding functionality to the + **Active** checkbox, see [issue 367509](https://gitlab.com/gitlab-org/gitlab/-/issues/367509). 1. Select **Add header** to create a new name and value pair. Enter as many name and value pairs as required. You can add up to 20 headers per streaming destination. 1. Select **Save** to update the streaming destination. diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index 47833224c0a..92bd73d1a75 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -216,6 +216,8 @@ The following actions on groups generate group audit events: [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/377877) in GitLab 15.6. - An environment is protected or unprotected. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216164) in GitLab 15.8. +- Changes to Code Suggestions. + [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/405295) in GitLab 15.11. ### Project events @@ -316,48 +318,62 @@ The following actions on projects generate project audit events: ### GitLab agent for Kubernetes events -The following actions on projects generate agent audit events: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/382133) in GitLab 15.10. -- A cluster agent token is created. - Introduced in GitLab 15.9 -- A cluster agent token is revoked. - Introduced in GitLab 15.9 +GitLab generates audit events when a cluster agent token is created or revoked. ### Instance events **(PREMIUM SELF)** +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16826) in GitLab 13.5, audit events for failed second-factor authentication attempt. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276250) in GitLab 13.6, audit events for when a user is approved using the Admin Area. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276921) in GitLab 13.6, audit events for when a user's personal access token is successfully or unsuccessfully created or revoked. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/298783) in GitLab 13.9, audit events for when a user requests access to an instance or is rejected using the Admin Area. + The following user actions on a GitLab instance generate instance audit events: -- Sign-in events and the authentication type (such as standard, LDAP, or OmniAuth) -- Failed sign-ins -- Added SSH key -- Added or removed email -- Changed password -- Ask for password reset -- Grant OAuth access -- Started or stopped user impersonation -- Changed username. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7797) in GitLab 12.8. -- User was deleted. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/251) in GitLab 12.8. -- User was added. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/251) in GitLab 12.8. -- User requests access to an instance. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/298783) in GitLab 13.9. -- User was approved using the Admin Area. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276250) in GitLab 13.6. -- User was rejected using the Admin Area. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/298783) in GitLab 13.9. -- User was blocked using the Admin Area. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/251) in GitLab 12.8. -- User was blocked using the API. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25872) in GitLab 12.9. -- Failed second-factor authentication attempt. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16826) in - GitLab 13.5. -- A user's personal access token was successfully created or revoked. - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276921) in GitLab 13.6. -- A failed attempt to create or revoke a user's personal access token. - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276921) in GitLab 13.6. +- Sign-in events and the authentication type such as standard, LDAP, or OmniAuth. +- Failed sign-ins. +- Added SSH key. +- Added or removed email. +- Changed password. +- Ask for password reset. +- Grant OAuth access. +- Started or stopped user impersonation. +- Changed username. +- User was added or deleted. +- User requests access to an instance. +- User was approved, rejected, or blocked using the Admin Area. +- User was blocked using the API. +- Failed second-factor authentication attempt. +- A user's personal access token was successfully or unsuccessfully created or revoked. - Administrator added or removed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/323905) in GitLab 14.1. - Removed SSH key. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220127) in GitLab 14.1. - Added or removed GPG key. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220127) in GitLab 14.1. - A user's two-factor authentication was disabled. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/238177) in GitLab 15.1. - Enabled Admin Mode. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362101) in GitLab 15.7. +- All [group events](#group-events) and [project events](#project-events). +- User was unblocked using the Admin Area or API. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115727) in GitLab 15.11. +- User was banned using the Admin Area or API. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116103) in GitLab 15.11. +- User was unbanned using the Admin Area or API. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116221) in GitLab 15.11. Instance events can also be accessed using the [Instance Audit Events API](../api/audit_events.md#instance-audit-events). +### GitLab Runner events + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335509) in GitLab 14.8, audit events for when a runner is registered. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/349540) in GitLab 14.9, audit events for when a runner is unregistered. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/349542) in GitLab 14.9, audit events for when a runner is assigned to or unassigned from a project. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355637) in GitLab 15.0, audit events for when a runner registration token is reset. + +GitLab generates audit events for the following GitLab Runner actions: + +- Instance, group, or project runner is registered. +- Instance, group, or project runner is unregistered. +- Runner is assigned to or unassigned from a project. +- Instance, group, or project runner registration token is reset. + [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/102579) in GitLab 15.6. + ## "Deleted User" events Audit events created after users are deleted are created for "Deleted User". For example, if a deleted user's access to @@ -373,6 +389,7 @@ Some events are not tracked in audit events. The following epics and issues prop - [Group settings and activity](https://gitlab.com/groups/gitlab-org/-/epics/475). - [Instance-level settings and activity](https://gitlab.com/groups/gitlab-org/-/epics/476). - [Deployment Approval activity](https://gitlab.com/gitlab-org/gitlab/-/issues/354782). +- [Approval rules processing by a non GitLab user](https://gitlab.com/gitlab-org/gitlab/-/issues/407384). If you don't see the event you want in any of the epics, you can either: diff --git a/doc/administration/auth/atlassian.md b/doc/administration/auth/atlassian.md index 58412078802..45617b9965c 100644 --- a/doc/administration/auth/atlassian.md +++ b/doc/administration/auth/atlassian.md @@ -41,7 +41,7 @@ To enable the Atlassian OmniAuth provider for passwordless authentication you mu sudo -u git -H editor /home/git/gitlab/config/gitlab.yml ``` -1. Edit the [common configuration file settings](../../integration/omniauth.md#configure-common-settings) +1. Configure the [common settings](../../integration/omniauth.md#configure-common-settings) to add `atlassian_oauth2` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/administration/auth/cognito.md b/doc/administration/auth/cognito.md index a73595c731b..cfac958e297 100644 --- a/doc/administration/auth/cognito.md +++ b/doc/administration/auth/cognito.md @@ -41,7 +41,7 @@ To enable AWS Cognito as an authentication provider, complete the following step ## Configure GitLab -1. Edit the [common configuration file settings](../../integration/omniauth.md#configure-common-settings) +1. Configure the [common settings](../../integration/omniauth.md#configure-common-settings) to add `cognito` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. On your GitLab server, open the configuration file. diff --git a/doc/administration/auth/crowd.md b/doc/administration/auth/crowd.md index 277e59b1a8a..e403eb800d6 100644 --- a/doc/administration/auth/crowd.md +++ b/doc/administration/auth/crowd.md @@ -40,7 +40,7 @@ this provider also allows Crowd authentication for Git-over-https requests. sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](../../integration/omniauth.md#configure-common-settings) +1. Configure the [common settings](../../integration/omniauth.md#configure-common-settings) to add `crowd` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/administration/auth/index.md b/doc/administration/auth/index.md index 2b978dd1f2c..4a8e230a944 100644 --- a/doc/administration/auth/index.md +++ b/doc/administration/auth/index.md @@ -1,5 +1,4 @@ --- -comments: false type: index stage: Manage group: Authentication and Authorization @@ -34,3 +33,7 @@ For more information, see the links shown on this page for each external provide | **User Removal** | SCIM (remove user from top-level group) | LDAP (remove user from groups and block from the instance)<br>SCIM | 1. Using Just-In-Time (JIT) provisioning, user accounts are created when the user first signs in. + +## Test OIDC/OAuth in GitLab + +See [Test OIDC/OAuth in GitLab](test_oidc_oauth.md) to learn how to test OIDC/OAuth authentication in your GitLab instance using your client application. diff --git a/doc/administration/auth/jwt.md b/doc/administration/auth/jwt.md index 8c9800aa792..9994b374038 100644 --- a/doc/administration/auth/jwt.md +++ b/doc/administration/auth/jwt.md @@ -25,7 +25,7 @@ JWT provides you with a secret key for you to use. sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](../../integration/omniauth.md#configure-common-settings) +1. Configure the [common settings](../../integration/omniauth.md#configure-common-settings) to add `jwt` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. Add the provider configuration. diff --git a/doc/administration/auth/ldap/google_secure_ldap.md b/doc/administration/auth/ldap/google_secure_ldap.md index e0612099221..042a65be500 100644 --- a/doc/administration/auth/ldap/google_secure_ldap.md +++ b/doc/administration/auth/ldap/google_secure_ldap.md @@ -18,6 +18,8 @@ The steps below cover: - Configuring the Secure LDAP Client in the Google administrator console. - Required GitLab configuration. +Secure LDAP is only available on specific Google Workspace editions. For more information, see the [Google Secure LDAP service documentation](https://support.google.com/a/answer/9048516). + ## Configuring Google LDAP client 1. Go to <https://admin.google.com/Dashboard> and sign in as a Google Workspace domain administrator. @@ -88,7 +90,7 @@ values obtained during the LDAP client configuration earlier: encryption: 'simple_tls' verify_certificates: true retry_empty_result_with_codes: [80] - + base: "DC=example,DC=com" tls_options: cert: | -----BEGIN CERTIFICATE----- @@ -152,7 +154,7 @@ values obtained during the LDAP client configuration earlier: servers: main: # 'main' is the GitLab 'provider ID' of this LDAP server label: 'Google Secure LDAP' - + base: "DC=example,DC=com" host: 'ldap.google.com' port: 636 uid: 'uid' diff --git a/doc/administration/auth/ldap/ldap_synchronization.md b/doc/administration/auth/ldap/ldap_synchronization.md index 59ed3a4153d..f32a4af9e27 100644 --- a/doc/administration/auth/ldap/ldap_synchronization.md +++ b/doc/administration/auth/ldap/ldap_synchronization.md @@ -7,12 +7,16 @@ info: To determine the technical writer assigned to the Stage/Group associated w # LDAP synchronization **(PREMIUM SELF)** If you have [configured LDAP to work with GitLab](index.md), GitLab can automatically synchronize -users and groups. This process updates user and group information. +users and groups. + +LDAP synchronization updates user and group information for existing GitLab users that have an LDAP identity assigned. It does not create new GitLab users through LDAP. You can change when synchronization occurs. ## User sync +> Preventing LDAP username synchronization [introduced](<https://gitlab.com/gitlab-org/gitlab/-/issues/11336>) in GitLab 15.11. + Once per day, GitLab runs a worker to check and update GitLab users against LDAP. @@ -36,11 +40,109 @@ For more information, see [Bitmask Searches in LDAP](https://ctovswild.com/2009/ The process also updates the following user information: - Name. Because of a [sync issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342598), `name` is not synchronized if - [**Prevent users from changing their profile name**](../../../user/admin_area/settings/account_and_limit_settings.md#disable-user-profile-name-changes) is enabled. + [**Prevent users from changing their profile name**](../../../user/admin_area/settings/account_and_limit_settings.md#disable-user-profile-name-changes) is enabled or `sync_name` is set to `false`. - Email address. - SSH public keys if `sync_ssh_keys` is set. - Kerberos identity if Kerberos is enabled. +### Synchronize LDAP username + +By default, GitLab synchronizes the LDAP username field. + +To prevent this synchronization, you can set `sync_name` to `false`. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['ldap_servers'] = { + 'main' => { + 'sync_name' => false, + } + } + ``` + +1. Save the file and reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +:::TabTitle Helm chart (Kubernetes) + +1. Export the Helm values: + + ```shell + helm get values gitlab > gitlab_values.yaml + ``` + +1. Edit `gitlab_values.yaml`: + + ```yaml + global: + appConfig: + ldap: + servers: + main: + sync_name: false + ``` + +1. Save the file and apply the new values: + + ```shell + helm upgrade -f gitlab_values.yaml gitlab gitlab/gitlab + ``` + +:::TabTitle Docker + +1. Edit `docker-compose.yml`: + + ```yaml + version: "3.6" + services: + gitlab: + environment: + GITLAB_OMNIBUS_CONFIG: | + gitlab_rails['ldap_servers'] = { + 'main' => { + 'sync_name' => false, + } + } + ``` + +1. Save the file and restart GitLab: + + ```shell + docker compose up -d + ``` + +:::TabTitle Self-compiled (source) + +1. Edit `/home/git/gitlab/config/gitlab.yml`: + + ```yaml + production: &base + ldap: + servers: + main: + sync_name: false + ``` + +1. Save the file and restart GitLab: + + ```shell + # For systems running systemd + sudo systemctl restart gitlab.target + + # For systems running SysV init + sudo service gitlab restart + ``` + +::EndTabs + ### Blocked users A user is blocked if either the: diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md index 4edc00b0d62..afc2f74201a 100644 --- a/doc/administration/auth/oidc.md +++ b/doc/administration/auth/oidc.md @@ -29,7 +29,7 @@ The OpenID Connect provides you with a client's details and secret for you to us sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](../../integration/omniauth.md#configure-common-settings) +1. Configure the [common settings](../../integration/omniauth.md#configure-common-settings) to add `openid_connect` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. @@ -40,7 +40,7 @@ The OpenID Connect provides you with a client's details and secret for you to us ```ruby gitlab_rails['omniauth_providers'] = [ { - name: "openid_connect", + name: "openid_connect", # do not change this parameter label: "Provider name", # optional label for login button, defaults to "Openid Connect" icon: "<custom_provider_icon>", args: { @@ -66,7 +66,7 @@ The OpenID Connect provides you with a client's details and secret for you to us For installation from source: ```yaml - - { name: 'openid_connect', + - { name: 'openid_connect', # do not change this parameter label: 'Provider name', # optional label for login button, defaults to "Openid Connect" icon: '<custom_provider_icon>', args: { @@ -165,7 +165,7 @@ for more details: ```ruby gitlab_rails['omniauth_providers'] = [ { - name: "openid_connect", + name: "openid_connect", # do not change this parameter label: "Google OpenID", # optional label for login button, defaults to "Openid Connect" args: { name: "openid_connect", @@ -203,7 +203,7 @@ Example Omnibus configuration block: ```ruby gitlab_rails['omniauth_providers'] = [ { - name: "openid_connect", + name: "openid_connect", # do not change this parameter label: "Azure OIDC", # optional label for login button, defaults to "Openid Connect" args: { name: "openid_connect", @@ -335,7 +335,7 @@ but `LocalAccounts` authenticates against local Active Directory accounts. Befor ```ruby gitlab_rails['omniauth_providers'] = [ { - name: "openid_connect", + name: "openid_connect", # do not change this parameter label: "Azure B2C OIDC", # optional label for login button, defaults to "Openid Connect" args: { name: "openid_connect", @@ -395,7 +395,7 @@ Example Omnibus configuration block: ```ruby gitlab_rails['omniauth_providers'] = [ { - name: "openid_connect", + name: "openid_connect", # do not change this parameter label: "Keycloak", # optional label for login button, defaults to "Openid Connect" args: { name: "openid_connect", @@ -475,7 +475,7 @@ To use symmetric key encryption: ```ruby gitlab_rails['omniauth_providers'] = [ { - name: "openid_connect", + name: "openid_connect", # do not change this parameter label: "Keycloak", # optional label for login button, defaults to "Openid Connect" args: { name: "openid_connect", @@ -519,7 +519,7 @@ Example Omnibus GitLab configuration (file path: `/etc/gitlab/gitlab.rb`): ```ruby gitlab_rails['omniauth_providers'] = [ { - name: "openid_connect", + name: "openid_connect", # do not change this parameter label: "Casdoor", # optional label for login button, defaults to "Openid Connect" args: { name: "openid_connect", @@ -542,7 +542,7 @@ gitlab_rails['omniauth_providers'] = [ Example installations from source configuration (file path: `config/gitlab.yml`): ```yaml - - { name: 'openid_connect', + - { name: 'openid_connect', # do not change this parameter label: 'Casdoor', # optional label for login button, defaults to "Openid Connect" args: { name: 'openid_connect', diff --git a/doc/administration/auth/test_oidc_oauth.md b/doc/administration/auth/test_oidc_oauth.md new file mode 100644 index 00000000000..95cca1ced86 --- /dev/null +++ b/doc/administration/auth/test_oidc_oauth.md @@ -0,0 +1,57 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Test OIDC/OAuth in GitLab **(FREE)** + +To test OIDC/OAuth in GitLab, you must: + +1. [Enable OIDC/OAuth](#enable-oidcoauth-in-gitlab) +1. [Test OIDC/OAuth with your client application](#test-oidcoauth-with-your-client-application) +1. [Verify OIDC/OAuth authentication](#verify-oidcoauth-authentication) + +## Prerequisites + +Before you can test OIDC/OAuth on GitLab, you'll need the following: + +- Publicly accessible GitLab instance +- A client application that you want to use to test OIDC/OAuth +- A user account on the GitLab instance that you can use to log in and test OIDC/OAuth + +## Enable OIDC/OAuth in GitLab + +First, you must create OIDC/OAuth application on your GitLab instance. To do this: + +1. Sign in to GitLab as an administrator. +1. Select **Profile > Preferences > Applications**. +1. Fill in the details for your client application, including the name, redirect URI, and allowed scopes. +1. Make sure the `openid` scope is enabled. +1. Select **Save application** to create the new OAuth application. + +## Test OIDC/OAuth with your client application + +After you've created your OAuth application in GitLab, you can use it to test OIDC/OAuth: + +1. You can use <https://openidconnect.net> as the OIDC/OAuth playground. +1. Sign out of GitLab. +1. Visit your client application and initiate the OIDC/OAuth flow, using the GitLab OAuth application you created in the previous step. +1. Follow the prompts to sign in to GitLab and authorize the client application to access your GitLab account. +1. After you've completed the OIDC/OAuth flow, your client application should have received an access token that it can use to authenticate with GitLab. + +## Verify OIDC/OAuth authentication + +To verify that OIDC/OAuth authentication is working correctly on GitLab, you can perform the following checks: + +1. Check that the access token you received in the previous step is valid and can be used to authenticate with GitLab. You can do this by making a test API request to GitLab, using the access token to authenticate. For example: + + ```shell + curl --header "Authorization: Bearer <access_token>" https://mygitlabinstance.com/api/v4/user + ``` + + Replace `<access_token>` with the actual access token you received in the previous step. If the API request succeeds and returns information about the authenticated user, then OIDC/OAuth authentication is working correctly. + +1. Check that the scopes you specified in your OAuth application are being enforced correctly. You can do this by making API requests that require the specific scopes and checking that they succeed or fail as expected. + +That's it! With these steps, you should be able to test OIDC/OAuth authentication on your GitLab instance using your client application. diff --git a/doc/administration/clusters/kas.md b/doc/administration/clusters/kas.md index 7a289b6a500..6d6e8e5513c 100644 --- a/doc/administration/clusters/kas.md +++ b/doc/administration/clusters/kas.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -43,6 +43,33 @@ To enable the agent server on a single node: For additional configuration options, see the **Enable GitLab KAS** section of the [`gitlab.rb.template`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-config-template/gitlab.rb.template). +##### Configure KAS to listen on a UNIX socket + +If you use GitLab behind a proxy, KAS might not work correctly. You can resolve this issue on a single-node installation, you can configure KAS to listen on a UNIX socket. + +To configure KAS to listen on a UNIX socket: + +1. Create a directory for the KAS sockets: + + ```shell + sudo mkdir -p /var/opt/gitlab/gitlab-kas/sockets/ + ``` + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_kas['internal_api_listen_network'] = 'unix' + gitlab_kas['internal_api_listen_address'] = '/var/opt/gitlab/gitlab-kas/sockets/internal-api.socket' + gitlab_kas['private_api_listen_network'] = 'unix' + gitlab_kas['private_api_listen_address'] = '/var/opt/gitlab/gitlab-kas/sockets/private-api.socket' + gitlab_kas['env'] = { + 'SSL_CERT_DIR' => "/opt/gitlab/embedded/ssl/certs/", + 'OWN_PRIVATE_API_URL' => 'unix:///var/opt/gitlab/gitlab-kas/sockets/private-api.socket' + } + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + #### Enable on multiple nodes To enable the agent server on multiple nodes: diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index 592c110b446..9ae0e9d7790 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -18,25 +18,13 @@ standards or mandates from regulatory bodies. The following features help you define rules and policies to adhere to workflow requirements, separation of duties, and secure supply chain best practices: -- [**Credentials inventory**](../user/admin_area/credentials_inventory.md) (for - instances): With a credentials inventory, GitLab administrators can keep track - of the credentials used by all of the users in their GitLab instance. -- [**Granular user roles and flexible permissions**](../user/permissions.md) - (for instances, groups, and projects): Manage access and permissions with five - different user roles and settings for external users. Set permissions according - to people's role, rather than either read or write access to a repository. Don't - share the source code with people that only need access to the issue tracker. -- [**Merge request approvals**](../user/project/merge_requests/approvals/index.md) - (for instances, groups, and projects): Configure approvals required for - merge requests. -- [**Push rules**](../user/project/repository/push_rules.md) (for instances, groups, and - projects): Control pushes to your repositories. -- Separation of duties using [**protected branches**](../user/project/protected_branches.md#require-code-owner-approval-on-a-protected-branch) - and [**custom CI/CD configuration paths**](../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file) (for projects): Users can leverage the GitLab cross-project YAML configurations - to define deployers of code and developers of code. See how to use this setup - to define these roles in: - - The [Separation of Duties deploy project](https://gitlab.com/guided-explorations/separation-of-duties-deploy/blob/master/README.md). - - The [Separation of Duties project](https://gitlab.com/guided-explorations/separation-of-duties/blob/master/README.md). +| Feature | Instances | Groups | Projects | Description | +|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Credentials inventory](../user/admin_area/credentials_inventory.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Keep track of the credentials used by all of the users in a GitLab instance. | +| [Granular user roles<br/>and flexible permissions](../user/permissions.md) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Manage access and permissions with five different user roles and settings for external users. Set permissions according to people's role, rather than either read or write access to a repository. Don't share the source code with people that only need access to the issue tracker. | +| [Merge request approvals](../user/project/merge_requests/approvals/index.md) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Configure approvals required for merge requests. | +| [Push rules](../user/project/repository/push_rules.md) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Control pushes to your repositories. | +| Separation of duties using<br/>[protected branches](../user/project/protected_branches.md#require-code-owner-approval-on-a-protected-branch) and<br/>[custom CI/CD configuration paths](../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file) | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Leverage the GitLab cross-project YAML configurations to define deployers of code and developers of code. See how to use this setup to define these roles in the [Separation of Duties deploy project](https://gitlab.com/guided-explorations/separation-of-duties-deploy/blob/master/README.md) and the [Separation of Duties project](https://gitlab.com/guided-explorations/separation-of-duties/blob/master/README.md). | ## Compliant workflow automation @@ -48,66 +36,42 @@ settings and automation to ensure that whatever a compliance team has configured stays configured and working correctly. These features can help you automate compliance: -- [**Compliance frameworks**](../user/group/compliance_frameworks.md) (for groups): Create a custom - compliance framework at the group level to describe the type of compliance requirements any child project needs to follow. -- [**Compliance pipelines**](../user/group/compliance_frameworks.md#compliance-pipelines) (for groups): Define a - pipeline configuration to run for any projects with a given compliance framework. +| Feature | Instances | Groups | Projects | Description | +|:------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:-------------------------------------------------------------------------------------------| +| [Compliance frameworks](../user/group/compliance_frameworks.md) | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Describe the type of compliance requirements projects must follow. | +| [Compliance pipelines](../user/group/compliance_frameworks.md#compliance-pipelines) | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Define a pipeline configuration to run for any projects with a given compliance framework. | ## Audit management An important part of any compliance program is being able to go back and understand -what happened, when it happened, and who was responsible. This is useful in audit +what happened, when it happened, and who was responsible. You can use this in audit situations as well as for understanding the root cause of issues when they occur. -It is useful to have both low-level, raw lists of audit data as well as high-level, + +It is helpful to have both low-level, raw lists of audit data as well as high-level, summary lists of audit data. Between these two, compliance teams can quickly identify if problems exist and then drill down into the specifics of those issues. These features can help provide visibility into GitLab and audit what is happening: -- [**Audit events**](audit_events.md) (for instances, groups, and projects): To - maintain the integrity of your code, audit events give administrators the - ability to view any modifications made within the GitLab server in an advanced - audit events system, so you can control, analyze, and track every change. -- [**Audit reports**](audit_reports.md) (for instances, groups, and projects): - Create and access reports based on the audit events that have occurred. Use - pre-built GitLab reports or the API to build your own. -- [**Auditor users**](auditor_users.md) (for instances): Auditor users are users - who are given read-only access to all projects, groups, and other resources on - the GitLab instance. -- [**Compliance report**](../user/compliance/compliance_report/index.md) (for - groups): Quickly get visibility into the compliance posture of your organization. +| Feature | Instances | Groups | Projects | Description | +|:-------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Audit events](audit_events.md) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | To maintain the integrity of your code, audit events give administrators the ability to view any modifications made in the GitLab server in an advanced audit events system, so you can control, analyze, and track every change. | +| [Audit reports](audit_reports.md) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Create and access reports based on the audit events that have occurred. Use pre-built GitLab reports or the API to build your own. | +| [Auditor users](auditor_users.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Auditor users are users who are given read-only access to all projects, groups, and other resources on the GitLab instance. | +| [Compliance report](../user/compliance/compliance_report/index.md) | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Quickly get visibility into the compliance posture of your organization. | ## Other compliance features These features can also help with compliance requirements: -- [**Email all users of a project, group, or entire server**](../user/admin_area/email_from_gitlab.md) - (for instances): An administrator can email groups of users based on project - or group membership, or email everyone using the GitLab instance. These emails - are great for scheduled maintenance or upgrades. -- [**Enforce ToS acceptance**](../user/admin_area/settings/terms.md) (for - instances): Enforce your users accepting new terms of service by blocking GitLab - traffic. -- [**External Status Checks**](../user/project/merge_requests/status_checks.md) - (for projects): Interface with third-party systems you already use during - development to ensure you remain compliant. -- [**Generate reports on permission levels of users**](../user/admin_area/index.md#user-permission-export) - (for instances): Administrators can generate a report listing all users' access - permissions for groups and projects in the instance. -- [**License compliance**](../user/compliance/license_compliance/index.md) (for - projects): Search dependencies for their licenses. This lets you determine if - the licenses of your project's dependencies are compatible with your project's - license. -- [**Lock project membership to group**](../user/group/access_and_permissions.md#prevent-members-from-being-added-to-projects-in-a-group) - (for groups): Group owners can prevent new members from being added to projects - within a group. -- [**LDAP group sync**](auth/ldap/ldap_synchronization.md#group-sync) (for - instances): Gives administrators the ability to automatically sync groups and - manage SSH keys, permissions, and authentication, so you can focus on building - your product, not configuring your tools. -- [**LDAP group sync filters**](auth/ldap/ldap_synchronization.md#group-sync) - (for instances): Gives more flexibility to synchronize with LDAP based on - filters, meaning you can leverage LDAP attributes to map GitLab permissions. -- [**Omnibus GitLab package supports log forwarding**](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-forwarding) - (for instances): Forward your logs to a central system. -- [**Restrict SSH Keys**](../security/ssh_keys_restrictions.md) (for instances): - Control the technology and key length of SSH keys used to access GitLab. +| Feature | Instances | Groups | Projects | Description | +|:------------------------------------------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Email all users of a project,<br/>group, or entire server](../user/admin_area/email_from_gitlab.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Email groups of users based on project or group membership, or email everyone using the GitLab instance. These emails are great for scheduled maintenance or upgrades. | +| [Enforce ToS acceptance](../user/admin_area/settings/terms.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Enforce your users accepting new terms of service by blocking GitLab traffic. | +| [External Status Checks](../user/project/merge_requests/status_checks.md) | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Interface with third-party systems you already use during development to ensure you remain compliant. | +| [Generate reports on permission<br/>levels of users](../user/admin_area/index.md#user-permission-export) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Generate a report listing all users' access permissions for groups and projects in the instance. | +| [License compliance](../user/compliance/license_compliance/index.md) | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Search dependencies for their licenses. This lets you determine if the licenses of your project's dependencies are compatible with your project's license. | +| [Lock project membership to group](../user/group/access_and_permissions.md#prevent-members-from-being-added-to-projects-in-a-group) | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Group owners can prevent new members from being added to projects in a group. | +| [LDAP group sync](auth/ldap/ldap_synchronization.md#group-sync) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Automatically synchronize groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools. | +| [LDAP group sync filters](auth/ldap/ldap_synchronization.md#group-sync) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions. | +| [Omnibus GitLab package supports<br/>log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-forwarding) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Forward your logs to a central system. | +| [Restrict SSH Keys](../security/ssh_keys_restrictions.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Control the technology and key length of SSH keys used to access GitLab. | diff --git a/doc/administration/configure.md b/doc/administration/configure.md index bf618345a94..d68e81ebf69 100644 --- a/doc/administration/configure.md +++ b/doc/administration/configure.md @@ -7,44 +7,39 @@ type: reference # Configure your GitLab installation **(FREE SELF)** -Customize and configure your self-managed GitLab installation. Here are some quick links to get you started: +Customize and configure your self-managed GitLab installation. - [Authentication](auth/index.md) +- [CI/CD](../administration/cicd.md) - [Configuration](../user/admin_area/index.md) -- [Repository storage](repository_storage_paths.md) -- [Geo](geo/index.md) -- [Packages](packages/index.md) - -The following tables are intended to guide you to choose the right combination of capabilities based on your requirements. It is common to want the most -available, quickly recoverable, highly performant, and fully data resilient solution. However, there are tradeoffs. - -The tables lists features on the left and provides their capabilities to the right along with known trade-offs. - -## Gitaly Capabilities - -| | Availability | Recoverability | Data Resiliency | Performance | Risks/Trade-offs| -|-|--------------|----------------|-----------------|-------------|-----------------| -|Gitaly Cluster | Very high - tolerant of node failures | RTO for a single node of 10 s with no manual intervention | Data is stored on multiple nodes | Good - While writes may take slightly longer due to voting, read distribution improves read speeds | **Trade-off** - Slight decrease in write speed for redundant, strongly-consistent storage solution. **Risks** - [Does not support snapshot backups](gitaly/index.md#snapshot-backup-and-recovery-limitations), GitLab backup task can be slow for large data sets | -|Gitaly Shards | Single storage location is a single point of failure | Would need to restore only shards which failed | Single point of failure | Good - can allocate repositories to shards to spread load | **Trade-off** - Need to manually configure repositories into different shards to balance loads / storage space **Risks** - Single point of failure relies on recovery process when single-node failure occurs | -|Gitaly + NFS | Single storage location is a single point of failure | Single node failure requires restoration from backup | Single point of failure | Average - NFS is not ideally suited to large quantities of small reads / writes which can have a detrimental impact on performance | **Trade-off** - Familiar administration though NFS is not ideally suited to Git demands **Risks** - Many instances of NFS compatibility issues which provide very poor customer experiences | - -## Geo Capabilities - -If your availability needs to span multiple zones or multiple locations, read about [Geo](geo/index.md). - -| | Availability | Recoverability | Data Resiliency | Performance | Risks/Trade-offs| -|-|--------------|----------------|-----------------|-------------|-----------------| -|Geo| Depends on the architecture of the Geo site. It is possible to deploy secondaries in single and multiple node configurations. | Eventually consistent. Recovery point depends on replication lag, which depends on a number of factors such as network speeds. Geo supports failover from a primary to secondary site using manual commands that are scriptable. | Geo replicates 100% of planned data types and verifies 50%. See [limitations table](geo/replication/datatypes.md#limitations-on-replicationverification) for more detail. | Improves read/clone times for users of a secondary. | Geo is not intended to replace other backup/restore solutions. Because of replication lag and the possibility of replicating bad data from a primary, customers should also take regular backups of their primary site and test the restore process. | - -## Scenarios for failure modes and available mitigation paths - -The following table outlines failure modes and mitigation paths for the product offerings detailed in the tables above. Note - Gitaly Cluster install assumes an odd number replication factor of 3 or greater - -| Gitaly Mode | Loss of Single Gitaly Node | Application / Data Corruption | Regional Outage (Loss of Instance) | Notes | -| ----------- | -------------------------- | ----------------------------- | ---------------------------------- | ----- | -| Single Gitaly Node | Downtime - Must restore from backup | Downtime - Must restore from Backup | Downtime - Must wait for outage to end | | -| Single Gitaly Node + Geo Secondary | Downtime - Must restore from backup, can perform a manual failover to secondary | Downtime - Must restore from Backup, errors could have propagated to secondary | Manual intervention - failover to Geo secondary | | -| Sharded Gitaly Install | Partial Downtime - Only repositories on impacted node affected, must restore from backup | Partial Downtime - Only repositories on impacted node affected, must restore from backup | Downtime - Must wait for outage to end | | -| Sharded Gitaly Install + Geo Secondary | Partial Downtime - Only repositories on impacted node affected, must restore from backup, could perform manual failover to secondary for impacted repositories | Partial Downtime - Only repositories on impacted node affected, must restore from backup, errors could have propagated to secondary | Manual intervention - failover to Geo secondary | | -| Gitaly Cluster Install* | No Downtime - swaps repository primary to another node after 10 seconds | Not applicable; All writes are voted on by multiple Gitaly Cluster nodes | Downtime - Must wait for outage to end | Snapshot backups for Gitaly Cluster nodes not supported at this time | -| Gitaly Cluster Install* + Geo Secondary | No Downtime - swaps repository primary to another node after 10 seconds | Not applicable; All writes are voted on by multiple Gitaly Cluster nodes | Manual intervention - failover to Geo secondary | Snapshot backups for Gitaly Cluster nodes not supported at this time | +- [Consul](../administration/consul.md) +- [Environment variables](../administration/environment_variables.md) +- [File hooks](../administration/file_hooks.md) +- [Git protocol v2](../administration/git_protocol.md) +- [Incoming email](../administration/incoming_email.md) +- [Instance limits](../administration/instance_limits.md) +- [Instance Review](../administration/instance_review.md) +- [PostgreSQL](../administration/postgresql/index.md) +- [Load balancer](../administration/load_balancer.md) +- [NFS](../administration/nfs.md) +- [Postfix](../administration/reply_by_email_postfix_setup.md) +- [Redis](../administration/redis/index.md) +- [Sidekiq](../administration/sidekiq/index.md) +- [S/MIME signing](../administration/smime_signing_email.md) +- [Repository storage](../administration/repository_storage_paths.md) +- [Object storage](../administration/object_storage.md) +- [Merge request diffs storage](../administration/merge_request_diffs.md) +- [Static objects external storage](../administration/static_objects_external_storage.md) +- [Geo](../administration/geo/index.md) +- [Disaster recovery (Geo)](../administration/geo/disaster_recovery/index.md) +- [Agent server for Kubernetes](../administration/clusters/kas.md) +- [Server hooks](../administration/server_hooks.md) +- [Terraform state](../administration/terraform_state.md) +- [Terraform limits](../user/admin_area/settings/terraform_limits.md) +- [Packages](../administration/packages/index.md) +- [Web terminals](../administration/integration/terminal.md) +- [Wikis](../administration/wikis/index.md) +- [Invalidate Markdown cache](../administration/invalidate_markdown_cache.md) +- [Issue closing pattern](../administration/issue_closing_pattern.md) +- [Snippets](../administration/snippets/index.md) +- [Host the product documentation](../administration/docs_self_host.md) diff --git a/doc/administration/dedicated/index.md b/doc/administration/dedicated/index.md index 1991d0b64cc..25b53c97d8b 100644 --- a/doc/administration/dedicated/index.md +++ b/doc/administration/dedicated/index.md @@ -203,7 +203,7 @@ Prerequisites: To activate SAML for your GitLab Dedicated instance: -1. Read the [GitLab documentation about SAML](../../integration/saml.md#https://docs.gitlab.com/ee/integration/saml.html#configure-saml-on-your-idp) to gather all data your identity provider requires for configuration. You can also find some providers and their requirements in the [group SAML documentation](../../user/group/saml_sso/index.md#set-up-identity-provider). +1. Read the [GitLab documentation about SAML](../../integration/saml.md#https://docs.gitlab.com/ee/integration/saml.html#configure-saml-on-your-idp) to gather all data your identity provider requires for configuration. You can also find some providers and their requirements in the [group SAML documentation](../../user/group/saml_sso/index.md#set-up-your-identity-provider). 1. To make the necessary changes, include in your [support ticket](#configuration-changes) the desired [SAML configuration block](../../integration/saml.md#configure-saml-support-in-gitlab) that will be set on the GitLab application. At a minimum, GitLab needs the following information to enable SAML for your instance: - Assertion consumer service URL - Certificate fingerprint or certificate diff --git a/doc/administration/docs_self_host.md b/doc/administration/docs_self_host.md index 932ffb87288..a659508e37a 100644 --- a/doc/administration/docs_self_host.md +++ b/doc/administration/docs_self_host.md @@ -265,8 +265,5 @@ To upgrade to a later version [using your own web-server](#self-host-the-product If you self-host the product documentation: -- The version dropdown list displays additional versions that don't exist. Selecting - these versions displays a `404 Not Found` page. -- The search displays results from `docs.gitlab.com` and not the local site. - By default, the landing page redirects to the respective version (for example, `/14.5/`). This causes the landing page <https://docs.gitlab.com> to not be displayed. diff --git a/doc/administration/environment_variables.md b/doc/administration/environment_variables.md index b6a3a563ae1..a453d703a18 100644 --- a/doc/administration/environment_variables.md +++ b/doc/administration/environment_variables.md @@ -35,8 +35,8 @@ You can use the following environment variables to override certain values: | `GITLAB_ROOT_PASSWORD` | string | Sets the password for the `root` user on installation. | | `GITLAB_SHARED_RUNNERS_REGISTRATION_TOKEN` | string | Sets the initial registration token used for runners. | | `RAILS_ENV` | string | The Rails environment; can be one of `production`, `development`, `staging`, or `test`. | -| `UNSTRUCTURED_RAILS_LOG` | string | Enables the unstructured log in addition to JSON logs (defaults to `false`). | | `GITLAB_RAILS_CACHE_DEFAULT_TTL_SECONDS` | integer | The default TTL used for entries stored in the Rails-cache. Default is `28800`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95042) in 15.3. | +| `GITLAB_CI_CONFIG_FETCH_TIMEOUT_SECONDS` | integer | Timeout for resolving remote includes in CI config in seconds. Must be between `0` and `60`. Default is `30`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116383) in 15.11. | ## Adding more variables diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md index 99ac95cd536..ea4523c66e6 100644 --- a/doc/administration/geo/disaster_recovery/background_verification.md +++ b/doc/administration/geo/disaster_recovery/background_verification.md @@ -6,12 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Automatic background verification **(PREMIUM SELF)** -NOTE: -Automatic background verification of repositories and wikis was added in -GitLab EE 10.6 but is enabled by default only on GitLab EE 11.1. You can -disable or enable this feature manually by following -[these instructions](#disabling-or-enabling-the-automatic-background-verification). - Automatic background verification ensures that the transferred data matches a calculated checksum. If the checksum of the data on the **primary** site matches checksum of the data on the **secondary** site, the data transferred successfully. Following a planned failover, @@ -179,19 +173,5 @@ If the **primary** and **secondary** sites have a checksum verification mismatch ## Current limitations -Automatic background verification doesn't cover attachments, LFS objects, -job artifacts, and user uploads in file storage. You can keep track of the -progress to include them in [Geo: Verify all replicated data](https://gitlab.com/groups/gitlab-org/-/epics/1430). - -For now, you can verify their integrity -manually by following [these instructions](../../raketasks/check.md) on both -sites, and comparing the output between them. - -In GitLab EE 12.1, Geo calculates checksums for attachments, LFS objects, and -archived traces on secondary sites after the transfer, compares it with the -stored checksums, and rejects transfers if mismatched. Geo -currently does not support an automatic way to verify these data if they have -been synced before GitLab EE 12.1. - -Data in object storage is **not verified**, as the object store is responsible -for ensuring the integrity of the data. +For more information on what replication and verification methods are supported, +see the [supported Geo data types](../replication/datatypes.md). diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md index cef0101dd40..feb5e030b80 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md @@ -6,7 +6,7 @@ type: howto --- WARNING: -This runbook is in [**Alpha**](../../../../policy/alpha-beta-support.md#alpha-features). For complete, production-ready documentation, see the +This runbook is an [Experiment](../../../../policy/alpha-beta-support.md#experiment). For complete, production-ready documentation, see the [disaster recovery documentation](../index.md). # Disaster Recovery (Geo) promotion runbooks **(PREMIUM SELF)** diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md index 085765ec51e..19150027cca 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md @@ -6,7 +6,7 @@ type: howto --- WARNING: -This runbook is in [**Alpha**](../../../../policy/alpha-beta-support.md#alpha-features). For complete, production-ready documentation, see the +This runbook is an [Experiment](../../../../policy/alpha-beta-support.md#experiment). For complete, production-ready documentation, see the [disaster recovery documentation](../index.md). # Disaster Recovery (Geo) promotion runbooks **(PREMIUM SELF)** diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index feae2d49e8d..9fc2c05a492 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -304,7 +304,7 @@ If your **primary** site is using a [custom or self-signed certificate for inbou sudo gitlab-ctl reconfigure ``` -### Step 5. Enable Git access over HTTP/HTTPS +### Step 5. Enable Git access over HTTP/HTTPS and SSH Geo synchronizes repositories over HTTP/HTTPS, and therefore requires this clone method to be enabled. This is enabled by default, but if converting an existing site to Geo it should be checked: @@ -314,7 +314,10 @@ On the **primary** site: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility and access controls**. -1. Ensure "Enabled Git access protocols" is set to either "Both SSH and HTTP(S)" or "Only HTTP(S)". +1. If using Git over SSH, then: + 1. Ensure "Enabled Git access protocols" is set to "Both SSH and HTTP(S)". + 1. Follow [Fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md) on both primary and secondary sites. +1. If not using Git over SSH, then set "Enabled Git access protocols" to "Only HTTP(S)". ### Step 6. Verify proper functioning of the **secondary** site diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md index 95faafd073c..d6ce5ef5067 100644 --- a/doc/administration/geo/replication/datatypes.md +++ b/doc/administration/geo/replication/datatypes.md @@ -47,8 +47,8 @@ verification methods: | Blobs | Container registry _(object storage)_ | Geo with API/Managed/Docker API (*2*) | _SHA256 checksum_ | | Blobs | Package registry _(file system)_ | Geo with API | SHA256 checksum | | Blobs | Package registry _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | -| Blobs | Infrastructure registry _(file system)_ | Geo with API | SHA256 checksum | -| Blobs | Infrastructure registry _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | +| Blobs | Terraform Module Registry _(file system)_ | Geo with API | SHA256 checksum | +| Blobs | Terraform Module Registry _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | Versioned Terraform State _(file system)_ | Geo with API | SHA256 checksum | | Blobs | Versioned Terraform State _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | External Merge Request Diffs _(file system)_ | Geo with API | SHA256 checksum | @@ -192,17 +192,17 @@ successfully, you must replicate their data using some other means. |:--------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------|:---------------------------------------------------------------------------|:--------------------------------------------------------------------|:----------------------------------------------------------------|:------| |[Application data in PostgreSQL](../../postgresql/index.md) | **Yes** (10.2) | **Yes** (10.2) | N/A | N/A | | |[Project repository](../../../user/project/repository/index.md) | **Yes** (10.2) | **Yes** (10.7) | N/A | N/A | | -|[Project wiki repository](../../../user/project/wiki/index.md) | **Yes** (10.2) | **Yes** (10.7) | N/A | N/A | | +|[Project wiki repository](../../../user/project/wiki/index.md) | **Yes** (10.2)<sup>2</sup> | **Yes** (10.7)<sup>2</sup> | N/A | N/A | Migrated to [self-service framework](../../../development/geo/framework.md) in 15.11. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details.<br /><br />Behind feature flag geo_project_wiki_repository_replication, enabled by default in (15.11). | |[Group wiki repository](../../../user/project/wiki/group.md) | [**Yes** (13.10)](https://gitlab.com/gitlab-org/gitlab/-/issues/208147) | No | N/A | N/A | Behind feature flag `geo_group_wiki_repository_replication`, enabled by default. | |[Uploads](../../uploads.md) | **Yes** (10.2) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Replication is behind the feature flag `geo_upload_replication`, enabled by default. Verification was behind the feature flag `geo_upload_verification`, removed in 14.8. | |[LFS objects](../../lfs/index.md) | **Yes** (10.2) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | GitLab versions 11.11.x and 12.0.x are affected by [a bug that prevents any new LFS objects from replicating](https://gitlab.com/gitlab-org/gitlab/-/issues/32696).<br /><br />Replication is behind the feature flag `geo_lfs_object_replication`, enabled by default. Verification was behind the feature flag `geo_lfs_object_verification`, removed in 14.7. | |[Personal snippets](../../../user/snippets.md) | **Yes** (10.2) | **Yes** (10.2) | N/A | N/A | | |[Project snippets](../../../user/snippets.md) | **Yes** (10.2) | **Yes** (10.2) | N/A | N/A | | -|[CI job artifacts](../../../ci/pipelines/job_artifacts.md) | **Yes** (10.4) | **Yes** (14.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Verification is behind the feature flag `geo_job_artifact_replication`, enabled by default in 14.10. | +|[CI job artifacts](../../../ci/jobs/job_artifacts.md) | **Yes** (10.4) | **Yes** (14.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Verification is behind the feature flag `geo_job_artifact_replication`, enabled by default in 14.10. | |[CI Pipeline Artifacts](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/pipeline_artifact.rb) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Persists additional artifacts after a pipeline completes. | |[CI Secure Files](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/secure_file.rb) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [No](object_storage.md#verification-of-files-in-object-storage) | Verification is behind the feature flag `geo_ci_secure_file_replication`, enabled by default in 15.3. | -|[Container Registry](../../packages/container_registry.md) | **Yes** (12.3)* | **Yes** (15.10) | **Yes** (12.3)* | **Yes** (15.10) | See [instructions](container_registry.md) to set up the Container Registry replication. | -|[Infrastructure Registry](../../../user/packages/infrastructure_registry/index.md) | **Yes** (14.0) | **Yes** (14.0) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Behind feature flag `geo_package_file_replication`, enabled by default. | +|[Container Registry](../../packages/container_registry.md) | **Yes** (12.3)<sup>1</sup> | **Yes** (15.10) | **Yes** (12.3)<sup>1</sup> | **Yes** (15.10) | See [instructions](container_registry.md) to set up the Container Registry replication. | +|[Terraform Module Registry](../../../user/packages/terraform_module_registry/index.md) | **Yes** (14.0) | **Yes** (14.0) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Behind feature flag `geo_package_file_replication`, enabled by default. | |[Project designs repository](../../../user/project/issues/design_management.md) | **Yes** (12.7) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/32467) | N/A | N/A | Designs also require replication of LFS objects and Uploads. | |[Package Registry](../../../user/packages/package_registry/index.md) | **Yes** (13.2) | **Yes** (13.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Behind feature flag `geo_package_file_replication`, enabled by default. | |[Versioned Terraform State](../../terraform_state.md) | **Yes** (13.5) | **Yes** (13.12) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Replication is behind the feature flag `geo_terraform_state_version_replication`, enabled by default. Verification was behind the feature flag `geo_terraform_state_version_verification`, which was removed in 14.0. | @@ -217,4 +217,6 @@ successfully, you must replicate their data using some other means. |[Dependency Proxy Images](../../../user/packages/dependency_proxy/index.md) | [**Yes** (15.7)](https://gitlab.com/groups/gitlab-org/-/epics/8833) | [**Yes** (15.7)](https://gitlab.com/groups/gitlab-org/-/epics/8833) | [**Yes** (15.7)](https://gitlab.com/groups/gitlab-org/-/epics/8833) | [No](object_storage.md#verification-of-files-in-object-storage) | | |[Vulnerability Export](../../../user/application_security/vulnerability_report/index.md#export-vulnerability-details) | [Not planned](https://gitlab.com/groups/gitlab-org/-/epics/3111) | No | No | No | Not planned because they are ephemeral and sensitive information. They can be regenerated on demand. | -\* Migrated to [self-service framework](../../../development/geo/framework.md) in 15.5. See GitLab issue [#337436](https://gitlab.com/gitlab-org/gitlab/-/issues/337436) for more details. +<sup>1</sup> Migrated to [self-service framework](../../../development/geo/framework.md) in 15.5. See GitLab issue [#337436](https://gitlab.com/gitlab-org/gitlab/-/issues/337436) for more details. + +<sup>2</sup> Migrated to [self-service framework](../../../development/geo/framework.md) in 15.11. Behind feature flag `geo_project_wiki_repository_replication`, enabled by default. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details. diff --git a/doc/administration/geo/replication/img/adding_a_secondary_v13_3.png b/doc/administration/geo/replication/img/adding_a_secondary_v13_3.png Binary files differdeleted file mode 100644 index e43ace99666..00000000000 --- a/doc/administration/geo/replication/img/adding_a_secondary_v13_3.png +++ /dev/null diff --git a/doc/administration/geo/replication/single_sign_on.md b/doc/administration/geo/replication/single_sign_on.md index fc2f23552db..55e77d5657c 100644 --- a/doc/administration/geo/replication/single_sign_on.md +++ b/doc/administration/geo/replication/single_sign_on.md @@ -112,6 +112,15 @@ After configuring your SAML IdP to allow the secondary site's SAML callback URL, If you have configured SAML on the primary site correctly, then it should work on the secondary site without additional configuration. +## OpenID Connect + +If you use an [OpenID Connect (OIDC)](../../auth/oidc.md) OmniAuth provider, +in most cases, it should work without an issue: + +- **OIDC with Unified URL**: If you have configured OIDC on the primary site correctly, then it should work on the secondary site without additional configuration. +- **OIDC with separate URL with proxying disabled**: If you have configured OIDC on the primary site correctly, then it should work on the secondary site without additional configuration. +- **OIDC with separate URL with proxying enabled**: Geo with separate URL with proxying enabled does not support [OpenID Connect](../../auth/oidc.md). For more information, see [issue 396745](https://gitlab.com/gitlab-org/gitlab/-/issues/396745). + ## LDAP If you use LDAP on your **primary** site, you should also set up secondary LDAP servers on each **secondary** site. Otherwise, users cannot perform Git operations over HTTP(s) on the **secondary** site using HTTP basic authentication. However, users can still use Git with SSH and personal access tokens. diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index c18bb5b6351..8031e2f4896 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -959,7 +959,7 @@ This behavior affects only the following data types through GitLab 14.6: | Package Registry | 13.10 | | CI Pipeline Artifacts | 13.11 | | Terraform State Versions | 13.12 | -| Infrastructure Registry | 14.0 | +| Infrastructure Registry (renamed to Terraform Module Registry in GitLab 15.11) | 14.0 | | External MR diffs | 14.6 | | LFS Objects | 14.6 | | Pages Deployments | 14.6 | diff --git a/doc/administration/geo/setup/database.md b/doc/administration/geo/setup/database.md index 5f04326d49f..ff71ba4f94f 100644 --- a/doc/administration/geo/setup/database.md +++ b/doc/administration/geo/setup/database.md @@ -865,6 +865,7 @@ For each node running a Patroni instance on the secondary site: consul['configuration'] = { retry_join: %w[CONSUL_SECONDARY1_IP CONSUL_SECONDARY2_IP CONSUL_SECONDARY3_IP] } + consul['services'] = %w(postgresql) postgresql['md5_auth_cidr_addresses'] = [ 'PATRONI_SECONDARY1_IP/32', 'PATRONI_SECONDARY2_IP/32', 'PATRONI_SECONDARY3_IP/32', 'PATRONI_SECONDARY_PGBOUNCER/32', diff --git a/doc/administration/geo/setup/index.md b/doc/administration/geo/setup/index.md index 022d9c00772..64eb4fa5a6f 100644 --- a/doc/administration/geo/setup/index.md +++ b/doc/administration/geo/setup/index.md @@ -24,7 +24,6 @@ type: howto If you installed GitLab using the Omnibus packages (highly recommended): 1. [Set up the database replication](database.md) (`primary (read-write) <-> secondary (read-only)` topology). -1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). This step is required and needs to be done on **both** the **primary** and **secondary** sites. 1. [Configure GitLab](../replication/configuration.md) to set the **primary** and **secondary** sites. 1. Optional: [Configure Object storage](../../object_storage.md) 1. Optional: [Configure a secondary LDAP server](../../auth/ldap/index.md) for the **secondary** sites. See [notes on LDAP](../index.md#ldap). diff --git a/doc/administration/gitaly/configure_gitaly.md b/doc/administration/gitaly/configure_gitaly.md index 543f5b3c119..f3eb2de3f1d 100644 --- a/doc/administration/gitaly/configure_gitaly.md +++ b/doc/administration/gitaly/configure_gitaly.md @@ -6,25 +6,26 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Configure Gitaly **(FREE SELF)** -The Gitaly service itself is configured by using a [TOML configuration file](reference.md). +Configure Gitaly in one of two ways: -To change Gitaly settings: +::Tabs -**For Omnibus GitLab** +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb` and add or change the [Gitaly settings](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/1dd07197c7e5ae23626aad5a4a070a800b670380/files/gitlab-config-template/gitlab.rb.template#L1622-1676). 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). -**For installations from source** +:::TabTitle Self-compiled (source) 1. Edit `/home/git/gitaly/config.toml` and add or change the [Gitaly settings](https://gitlab.com/gitlab-org/gitaly/blob/master/config.toml.example). 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source). +::EndTabs + The following configuration options are also available: - Enabling [TLS support](#enable-tls-support). -- Configuring the [number of `gitaly-ruby` workers](#configure-number-of-gitaly-ruby-workers). - Limiting [RPC concurrency](#limit-rpc-concurrency). ## About the Gitaly token @@ -138,7 +139,11 @@ Gitaly and GitLab use two shared secrets for authentication: - _Gitaly token_: used to authenticate gRPC requests to Gitaly - _GitLab Shell token_: used for authentication callbacks from GitLab Shell to the GitLab internal API -**For Omnibus GitLab** +Configure authentication in one of two ways: + +::Tabs + +:::TabTitle Linux package (Omnibus) To configure the _Gitaly token_, edit `/etc/gitlab/gitlab.rb`: @@ -167,7 +172,7 @@ Edit `/etc/gitlab/gitlab.rb`: gitlab_shell['secret_token'] = 'shellsecret' ``` -**For installations from source** +:::TabTitle Self-compiled (source) 1. Copy `/home/git/gitlab/.gitlab_shell_secret` from the Gitaly client to the same path on the Gitaly servers (and any other Gitaly clients). @@ -189,19 +194,26 @@ Edit `/etc/gitlab/gitlab.rb`: 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source). -#### Configure Gitaly server - -**For Omnibus GitLab** +::EndTabs -1. Edit `/etc/gitlab/gitlab.rb`: +#### Configure Gitaly server <!-- Updates to example must be made at: + - https://gitlab.com/gitlab-org/charts/gitlab/blob/master/doc/advanced/external-gitaly/external-omnibus-gitaly.md#configure-omnibus-gitlab - https://gitlab.com/gitlab-org/gitlab/blob/master/doc/administration/gitaly/index.md#gitaly-server-configuration -- all reference architecture pages +- All reference architecture pages --> +Configure Gitaly server in one of two ways: + +::Tabs + +:::TabTitle Linux package (Omnibus) + +1. Edit `/etc/gitlab/gitlab.rb`: + ```ruby # Avoid running unnecessary services on the Gitaly server postgresql['enable'] = false @@ -294,7 +306,7 @@ Updates to example must be made at: - For GitLab 15.3 and later, run `sudo /opt/gitlab/embedded/bin/gitaly check /var/opt/gitlab/gitaly/config.toml`. - For GitLab 15.2 and earlier, run `sudo /opt/gitlab/embedded/bin/gitaly-hooks check /var/opt/gitlab/gitaly/config.toml`. -**For installations from source** +:::TabTitle Self-compiled (source) 1. Edit `/home/git/gitaly/config.toml`: @@ -345,6 +357,8 @@ Updates to example must be made at: - For GitLab 15.3 and later, run `sudo /opt/gitlab/embedded/bin/gitaly check /var/opt/gitlab/gitaly/config.toml`. - For GitLab 15.2 and earlier, run `sudo /opt/gitlab/embedded/bin/gitaly-hooks check /var/opt/gitlab/gitaly/config.toml`. +::EndTabs + WARNING: If directly copying repository data from a GitLab server to Gitaly, ensure that the metadata file, default path `/var/opt/gitlab/git-data/repositories/.gitaly-metadata`, is not included in the transfer. @@ -381,7 +395,11 @@ You can't define Gitaly servers with some as a local Gitaly server server (with `gitaly_address`) unless you use [mixed configuration](#mixed-configuration). -**For Omnibus GitLab** +Configure Gitaly clients in one of two ways: + +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb`: @@ -415,7 +433,7 @@ server (with `gitaly_address`) unless you use sudo gitlab-ctl tail gitaly ``` -**For installations from source** +:::TabTitle Self-compiled (source) 1. Edit `/home/git/gitlab/config/gitlab.yml`: @@ -451,6 +469,8 @@ server (with `gitaly_address`) unless you use tail -f /home/git/gitlab/log/gitaly.log ``` +::EndTabs + When you tail the Gitaly logs on your Gitaly server, you should see requests coming in. One sure way to trigger a Gitaly request is to clone a repository from GitLab over HTTP or HTTPS. @@ -532,9 +552,11 @@ Disabling Gitaly on the GitLab instance makes sense only when you run GitLab in Gitaly runs on a separate machine from the GitLab instance. Disabling Gitaly on all machines in the cluster is not a valid configuration (some machines much act as Gitaly servers). -To disable Gitaly on a GitLab server: +Disable Gitaly on a GitLab server in one of two ways: + +::Tabs -**For Omnibus GitLab** +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb`: @@ -544,7 +566,7 @@ To disable Gitaly on a GitLab server: 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). -**For installations from source** +:::TabTitle Self-compiled (source) 1. Edit `/etc/default/gitlab`: @@ -554,6 +576,8 @@ To disable Gitaly on a GitLab server: 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source). +::EndTabs + ## Enable TLS support Gitaly supports TLS encryption. To communicate with a Gitaly instance that listens for secure @@ -585,9 +609,11 @@ If you use a load balancer, it must be able to negotiate HTTP/2 using the ALPN T ### Configure Gitaly with TLS -To configure Gitaly with TLS: +Configure Gitaly with TLS in one of two ways: + +::Tabs -**For Omnibus GitLab** +:::TabTitle Linux package (Omnibus) 1. Create certificates for Gitaly servers. 1. On the Gitaly clients, copy the certificates (or their certificate authority) into @@ -651,7 +677,7 @@ To configure Gitaly with TLS: 1. Saving the file. 1. [Reconfiguring GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). -**For installations from source** +:::TabTitle Self-compiled (source) 1. Create certificates for Gitaly servers. 1. On the Gitaly clients, copy the certificates into the system trusted certificates: @@ -727,73 +753,13 @@ To configure Gitaly with TLS: 1. Saving the file. 1. [Restarting GitLab](../restart_gitlab.md#installations-from-source). +::EndTabs + ### Observe type of Gitaly connections For information on observing the type of Gitaly connections being served, see the [relevant documentation](monitoring.md#queries). -## `gitaly-ruby` - -Gitaly was developed to replace the Ruby application code in GitLab. - -To save time and avoid the risk of rewriting existing application logic, we chose to copy some -application code from GitLab into Gitaly. - -To be able to run that code, `gitaly-ruby` was created, which is a "sidecar" process for the main -Gitaly Go process. Some examples of things that are implemented in `gitaly-ruby` are: - -- RPCs that deal with wikis. -- RPCs that create commits on behalf of a user, such as merge commits. - -Recommended settings: - -- At least 300 MB memory per worker. -- No more than one worker per core. - -NOTE: -[Epic 2862](https://gitlab.com/groups/gitlab-org/-/epics/2862) proposes to remove `gitaly-ruby`. - -### Configure number of `gitaly-ruby` workers - -`gitaly-ruby` has much less capacity than Gitaly implemented in Go. If your Gitaly server has to handle lots of -requests, the default setting of having just one active `gitaly-ruby` sidecar might not be enough. - -If you see `ResourceExhausted` errors from Gitaly, it's very likely that you have not enough -`gitaly-ruby` capacity. - -You can increase the number of `gitaly-ruby` processes on your Gitaly server with the following -settings: - -**For Omnibus GitLab** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitaly['configuration'] = { - # ... - 'gitaly-ruby': { - # ... - # - # Default is 2 workers. The minimum is 2; 1 worker is always reserved as - # a passive stand-by. - num_workers: 4 - }, - } - ``` - -1. Save the file, and then [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). - -**For installations from source** - -1. Edit `/home/git/gitaly/config.toml`: - - ```toml - [gitaly-ruby] - num_workers = 4 - ``` - -1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source). - ## Limit RPC concurrency WARNING: @@ -1025,7 +991,11 @@ WARNING: Background repository optimization is an experimental feature and may place significant load on the host while running. Make sure to schedule this during off-peak hours and keep the duration short (for example, 30-60 minutes). -**For Omnibus GitLab** +Configure background repository optimization in one of two ways: + +::Tabs + +:::TabTitle Linux package (Omnibus) Edit `/etc/gitlab/gitlab.rb` and add: @@ -1042,7 +1012,7 @@ gitaly['configuration'] = { } ``` -**For installations from source** +:::TabTitle Self-compiled (source) Edit `/home/git/gitaly/config.toml` and add: @@ -1054,6 +1024,8 @@ duration = '30m' storages = ["default"] ``` +::EndTabs + ## Rotate Gitaly authentication token Rotating credentials in a production environment often requires downtime, causes outages, or both. @@ -1210,7 +1182,8 @@ automatically keep parts of the pack-objects cache files in RAM, making it faster. Because the pack-objects cache can lead to a significant increase in -disk write IO, it is off by default. +disk write IO, it is off by default. In GitLab 15.11 and later, +the write workload is approximately 50% lower, but the cache is still disabled by default. ### Configure the cache @@ -1222,6 +1195,7 @@ below. | `enabled` | `false` | Turns on the cache. When off, Gitaly runs a dedicated `git pack-objects` process for each request. | | `dir` | `<PATH TO FIRST STORAGE>/+gitaly/PackObjectsCache` | Local directory where cache files get stored. | | `max_age` | `5m` (5 minutes) | Cache entries older than this get evicted and removed from disk. | +| `min_occurrences` | 1 | Minimum times a key must occur before a cache entry is created. | In `/etc/gitlab/gitlab.rb`, set: @@ -1233,6 +1207,7 @@ gitaly['configuration'] = { enabled: true, # dir: '/var/opt/gitlab/git-data/repositories/+gitaly/PackObjectsCache', # max_age: '5m', + # min_occurrences: 1, }, } ``` @@ -1290,13 +1265,31 @@ a guarantee. Peak size may exceed this average. The `max_age` configuration setting lets you control the chance of a cache hit and the average amount of storage used by cache files. -Entries older than `max_age` get evicted from the in-memory metadata -store, and deleted from disk. +Entries older than `max_age` get deleted from the disk. Eviction does not interfere with ongoing requests. It is OK for `max_age` to be less than the time it takes to do a fetch over a slow connection because Unix filesystems do not truly delete a file until all processes that are reading the deleted file have closed it. +#### Minimum key occurrences `min_occurrences` + +> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/2222) in GitLab 15.11. + +The `min_occurrences` setting controls how often an identical request +must occur before we create a new cache entry. The default value is `1`, +meaning that unique requests do not get written into the cache. + +If you: + +- Increase this number, your cache hit rate goes down and the +cache uses less disk space. +- Decrease this number, your cache hit +rate goes up and the cache uses more disk space. + +You should set `min_occurrences` to `1`. On GitLab.com, +going from 0 to 1 saved us 50% cache disk space while barely affecting +the cache hit rate. + ### Observe the cache The cache can be observed [using metrics](monitoring.md#pack-objects-cache) and in the following logged @@ -1515,10 +1508,14 @@ By default, Gitaly doesn't sign commits made using GitLab UI. For example, commi - Web IDE. - Merge requests. -You can configure Gitaly to sign commits made using GitLab UI. The commits show as unverified and signed by an unknown user. Support for improvements is -proposed in issue [19185](https://gitlab.com/gitlab-org/gitlab/-/issues/19185). +You can configure Gitaly to sign commits made with the GitLab UI. The commits show as unverified and signed by an unknown +user. Support for improvements is proposed in [issue 19185](https://gitlab.com/gitlab-org/gitlab/-/issues/19185). + +Configure Gitaly to sign commits made with the GitLab UI in one of two ways: -**For Omnibus GitLab** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. [Create a GPG key](../../user/project/repository/gpg_signed_commits/index.md#create-a-gpg-key) and export it. For optimal performance, consider using an EdDSA key. @@ -1542,7 +1539,7 @@ proposed in issue [19185](https://gitlab.com/gitlab-org/gitlab/-/issues/19185). 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). -**For installations from source** +:::TabTitle Self-compiled (source) 1. [Create a GPG key](../../user/project/repository/gpg_signed_commits/index.md#create-a-gpg-key) and export it. For optimal performance, consider using an EdDSA key. @@ -1560,3 +1557,60 @@ proposed in issue [19185](https://gitlab.com/gitlab-org/gitlab/-/issues/19185). ``` 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source). + +::EndTabs + +## Generate configuration using an external command + +> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/4828) in GitLab 15.11. + +You can generate parts of the Gitaly configuration using an external command. You might do this: + +- To configure nodes without having to distribute the full configuration to each of them. +- To configure using auto-discovery of the node's settings. For example, using DNS entries. +- To configure secrets at startup of the node, so that don't need to be visible in plain text. + +To generate configuration using an external command, you must provide a script that dumps the +desired configuration of the Gitaly node in JSON format to its standard output. + +For example, the following command configures the HTTP password used to connect to the +GitLab internal API using an AWS secret: + +```ruby +#!/usr/bin/env ruby +require 'json' +JSON.generate({"gitlab": {"http_settings": {"password": `aws get-secret-value --secret-id ...`}}}) +``` + +You must then make the script path known to Gitaly in one of two ways: + +::Tabs + +:::TabTitle Linux package (Omnibus) + +Edit `/etc/gitlab/gitlab.rb` and configure the `config_command`: + +```ruby +gitaly['configuration'] = { + config_command: '/path/to/config_command', +} +``` + +:::TabTitle Self-compiled (source) + +Edit `/home/git/gitaly/config.toml` and configure `config_command`: + +```toml +config_command = "/path/to/config_command" +``` + +::EndTabs + +After configuration, Gitaly executes the command on startup and parses its +standard output as JSON. The resulting configuration is then merged back into +the other Gitaly configuration. + +Gitaly fails to start up if either: + +- The configuration command fails. +- The output produced by the command cannot be parsed as valid JSON. diff --git a/doc/administration/gitaly/gitaly_geo_capabilities.md b/doc/administration/gitaly/gitaly_geo_capabilities.md new file mode 100644 index 00000000000..e4147eec162 --- /dev/null +++ b/doc/administration/gitaly/gitaly_geo_capabilities.md @@ -0,0 +1,41 @@ +--- +stage: Systems +group: Gitaly +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Gitaly and Geo capabilities + +It is common to want the most available, quickly recoverable, highly performant, +and fully resilient solution for your data. However, there are tradeoffs. + +The following tables are intended to guide you to choose the right combination of capabilities based on your requirements. + +## Gitaly capabilities + +| Capability | Availability | Recoverability | Data Resiliency | Performance | Risks/Trade-offs| +|------------|--------------|----------------|-----------------|-------------|-----------------| +|Gitaly Cluster | Very high - tolerant of node failures | RTO for a single node of 10 s with no manual intervention | Data is stored on multiple nodes | Good - While writes may take slightly longer due to voting, read distribution improves read speeds | **Trade-off** - Slight decrease in write speed for redundant, strongly-consistent storage solution. **Risks** - [Does not support snapshot backups](../gitaly/index.md#snapshot-backup-and-recovery-limitations), GitLab backup task can be slow for large data sets | +|Gitaly Shards | Single storage location is a single point of failure | Would need to restore only shards which failed | Single point of failure | Good - can allocate repositories to shards to spread load | **Trade-off** - Need to manually configure repositories into different shards to balance loads / storage space **Risks** - Single point of failure relies on recovery process when single-node failure occurs | +|Gitaly + NFS | Single storage location is a single point of failure | Single node failure requires restoration from backup | Single point of failure | Average - NFS is not ideally suited to large quantities of small reads / writes which can have a detrimental impact on performance | **Trade-off** - Familiar administration though NFS is not ideally suited to Git demands **Risks** - Many instances of NFS compatibility issues which provide very poor customer experiences | + +## Geo capabilities + +If your availability needs to span multiple zones or multiple locations, read about [Geo](../geo/index.md). + +| Capability | Availability | Recoverability | Data Resiliency | Performance | Risks/Trade-offs| +|------------|--------------|----------------|-----------------|-------------|-----------------| +|Geo| Depends on the architecture of the Geo site. It is possible to deploy secondaries in single and multiple node configurations. | Eventually consistent. Recovery point depends on replication lag, which depends on a number of factors such as network speeds. Geo supports failover from a primary to secondary site using manual commands that are scriptable. | Geo replicates 100% of planned data types and verifies 50%. See [limitations table](../geo/replication/datatypes.md#limitations-on-replicationverification) for more detail. | Improves read/clone times for users of a secondary. | Geo is not intended to replace other backup/restore solutions. Because of replication lag and the possibility of replicating bad data from a primary, customers should also take regular backups of their primary site and test the restore process. | + +## Scenarios for failure modes and available mitigation paths + +The following table outlines failure modes and mitigation paths for the product offerings detailed in the tables above. Note - Gitaly Cluster install assumes an odd number replication factor of 3 or greater + +| Gitaly Mode | Loss of Single Gitaly Node | Application / Data Corruption | Regional Outage (Loss of Instance) | Notes | +| ----------- | -------------------------- | ----------------------------- | ---------------------------------- | ----- | +| Single Gitaly Node | Downtime - Must restore from backup | Downtime - Must restore from Backup | Downtime - Must wait for outage to end | | +| Single Gitaly Node + Geo Secondary | Downtime - Must restore from backup, can perform a manual failover to secondary | Downtime - Must restore from Backup, errors could have propagated to secondary | Manual intervention - failover to Geo secondary | | +| Sharded Gitaly Install | Partial Downtime - Only repositories on impacted node affected, must restore from backup | Partial Downtime - Only repositories on impacted node affected, must restore from backup | Downtime - Must wait for outage to end | | +| Sharded Gitaly Install + Geo Secondary | Partial Downtime - Only repositories on impacted node affected, must restore from backup, could perform manual failover to secondary for impacted repositories | Partial Downtime - Only repositories on impacted node affected, must restore from backup, errors could have propagated to secondary | Manual intervention - failover to Geo secondary | | +| Gitaly Cluster Install* | No Downtime - swaps repository primary to another node after 10 seconds | Not applicable; All writes are voted on by multiple Gitaly Cluster nodes | Downtime - Must wait for outage to end | Snapshot backups for Gitaly Cluster nodes not supported at this time | +| Gitaly Cluster Install* + Geo Secondary | No Downtime - swaps repository primary to another node after 10 seconds | Not applicable; All writes are voted on by multiple Gitaly Cluster nodes | Manual intervention - failover to Geo secondary | Snapshot backups for Gitaly Cluster nodes not supported at this time | diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 34e72f3eb5a..36f7456f22e 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -69,7 +69,7 @@ the current status of these issues, refer to the referenced issues and epics. | Issue | Summary | How to avoid | |:--------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------| -| Gitaly Cluster + Geo - Issues retrying failed syncs | If Gitaly Cluster is used on a Geo secondary site, repositories that have failed to sync could continue to fail when Geo tries to resync them. Recovering from this state requires assistance from support to run manual steps. | No known solution prior to GitLab 15.0. In GitLab 15.0 to 15.2, enable the [`gitaly_praefect_generated_replica_paths` feature flag](#praefect-generated-replica-paths-gitlab-150-and-later). In GitLab 15.3, the feature flag is enabled by default. | +| Gitaly Cluster + Geo - Issues retrying failed syncs | If Gitaly Cluster is used on a Geo secondary site, repositories that have failed to sync could continue to fail when Geo tries to resync them. Recovering from this state requires assistance from support to run manual steps. | No known solution prior to GitLab 15.0. In GitLab 15.0 to 15.2, enable the [`gitaly_praefect_generated_replica_paths` feature flag](#praefect-generated-replica-paths-gitlab-150-and-later) on your Geo primary site. In GitLab 15.3, the feature flag is enabled by default. | | Praefect unable to insert data into the database due to migrations not being applied after an upgrade | If the database is not kept up to date with completed migrations, then the Praefect node is unable to perform standard operation. | Make sure the Praefect database is up and running with all migrations completed (For example: `/opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-migrate-status` should show a list of all applied migrations). Consider [requesting upgrade assistance](https://about.gitlab.com/support/scheduling-upgrade-assistance/) so your upgrade plan can be reviewed by support. | | Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind results in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup:<br/><br/>1. [Shut down GitLab](../read_only_gitlab.md#shut-down-the-gitlab-ui).<br/>2. Snapshot all Gitaly Cluster nodes at the same time.<br/>3. Take a database dump of the Praefect database. | | Rebuilding or replacing an existing Gitaly Cluster node | There is no way to replace existing nodes in place because the Praefect database is relied on to determine the current state of each Gitaly node. Changing how Gitaly Cluster stores repositories is proposed in issue [4218](https://gitlab.com/gitlab-org/gitaly/-/issues/4218). Turning on [repository verification](praefect.md#repository-verification) is proposed in issue [4429](https://gitlab.com/gitlab-org/gitaly/-/issues/4429) so Praefect can detect that data is missing and needs to replicated to a new Gitaly node. | No known solution at this time. | diff --git a/doc/administration/gitaly/monitoring.md b/doc/administration/gitaly/monitoring.md index d4f3d931b03..a854a0a3b48 100644 --- a/doc/administration/gitaly/monitoring.md +++ b/doc/administration/gitaly/monitoring.md @@ -156,6 +156,7 @@ The following metrics are available from the `/metrics` endpoint: - `gitaly_praefect_replication_delay_bucket`, a histogram measuring how much time passes between when the replication job is created and when it starts. Available in GitLab 12.10 and later. - `gitaly_praefect_connections_total`, the total number of connections to Praefect. [Introduced](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4220) in GitLab 14.7. +- `gitaly_praefect_method_types`, a count of accessor and mutator RPCs per node. To monitor [strong consistency](index.md#strong-consistency), you can use the following Prometheus metrics: diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index efaa8c1ee18..62773f66796 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -1033,7 +1033,7 @@ For more information on Gitaly server configuration, see our 1. Disable all other services by editing `/etc/gitlab/gitlab.rb`: ```ruby - # Disable all other services on the Praefect node + # Disable all other services on the Gitaly node postgresql['enable'] = false redis['enable'] = false nginx['enable'] = false @@ -1181,10 +1181,7 @@ scope of the GitLab documentation. NOTE: The load balancer must be configured to accept traffic from the Gitaly nodes in -addition to the GitLab nodes. Some requests handled by -[`gitaly-ruby`](configure_gitaly.md#gitaly-ruby) sidecar processes call into the main Gitaly -process. `gitaly-ruby` uses the Gitaly address set in the GitLab server's -`git_data_dirs` setting to make this connection. +addition to the GitLab nodes. We hope that if you're managing fault-tolerant systems like GitLab, you have a load balancer of choice already. Some examples include [HAProxy](https://www.haproxy.org/) diff --git a/doc/administration/gitaly/recovery.md b/doc/administration/gitaly/recovery.md index 331f9b5a956..7d27a633512 100644 --- a/doc/administration/gitaly/recovery.md +++ b/doc/administration/gitaly/recovery.md @@ -100,7 +100,7 @@ The following parameters are available: some assigned copies that are not available. NOTE: -`dataloss` is still in [Beta](../../policy/alpha-beta-support.md#beta-features) and the output format is subject to change. +`dataloss` is still in [Beta](../../policy/alpha-beta-support.md#beta) and the output format is subject to change. To check for repositories with outdated primaries or for unavailable repositories, run: diff --git a/doc/administration/gitaly/reference.md b/doc/administration/gitaly/reference.md index 8b4fc51bcdf..cc938c95548 100644 --- a/doc/administration/gitaly/reference.md +++ b/doc/administration/gitaly/reference.md @@ -155,36 +155,6 @@ Prometheus query to see the hit rate: sum(rate(gitaly_catfile_cache_total{type="hit"}[5m])) / sum(rate(gitaly_catfile_cache_total{type=~"(hit)|(miss)"}[5m])) ``` -### `gitaly-ruby` - -A Gitaly process uses one or more `gitaly-ruby` helper processes to -execute RPCs implemented in Ruby instead of Go. The `[gitaly-ruby]` -section of the configuration file contains settings for these helper processes. - -These processes are known to occasionally suffer from memory leaks. -Gitaly restarts its `gitaly-ruby` helpers when their memory exceeds the -`max_rss` limit. - -| Name | Type | Required | Description | -| ---- | ---- | -------- | ----------- | -| `dir` | string | yes | Path to where `gitaly-ruby` is installed (needed to boot the process).| -| `max_rss` | integer | no | Resident set size limit that triggers a `gitaly-ruby` restart, in bytes. Default is `200000000` (200 MB). | -| `graceful_restart_timeout` | string | no | Grace period before a `gitaly-ruby` process is forcibly terminated after exceeding `max_rss`. Default is `10m` (10 minutes).| -| `restart_delay` | string | no |Time that `gitaly-ruby` memory must remain high before a restart. Default is `5m` (5 minutes).| -| `num_workers` | integer | no |Number of `gitaly-ruby` worker processes. Try increasing this number in case of `ResourceExhausted` errors. Default is `2`, minimum is `2`.| -| `linguist_languages_path` | string | no | Override for dynamic `languages.json` discovery. Defaults to an empty string (use of dynamic discovery).| - -Example: - -```toml -[gitaly-ruby] -dir = "/home/git/gitaly/ruby" -max_rss = 200000000 -graceful_restart_timeout = "10m" -restart_delay = "5m" -num_workers = 2 -``` - ### GitLab Shell For historical reasons @@ -232,7 +202,6 @@ The following values configure logging in Gitaly under the `[logging]` section. | `level` | string | no | Log level: `debug`, `info`, `warn`, `error`, `fatal`, or `panic`. Default: `info`. | | `sentry_dsn` | string | no | Sentry DSN (Data Source Name) for exception monitoring. | | `sentry_environment` | string | no | [Sentry Environment](https://docs.sentry.io/product/sentry-basics/environments/) for exception monitoring. | -| `ruby_sentry_dsn` | string | no | Sentry DSN (Data Source Name) for `gitaly-ruby` exception monitoring. | While the main Gitaly application logs go to `stdout`, there are some extra log files that go to a configured directory, like the GitLab Shell logs. diff --git a/doc/administration/gitaly/troubleshooting.md b/doc/administration/gitaly/troubleshooting.md index a3db28b0d7b..2bb87b20058 100644 --- a/doc/administration/gitaly/troubleshooting.md +++ b/doc/administration/gitaly/troubleshooting.md @@ -85,7 +85,7 @@ check for an SSL or TLS problem: ``` Check whether `Verify return code` field indicates a -[known Omnibus GitLab configuration problem](https://docs.gitlab.com/omnibus/settings/ssl.html). +[known Omnibus GitLab configuration problem](https://docs.gitlab.com/omnibus/settings/ssl/index.html). If `openssl` succeeds but `gitlab-rake gitlab:gitaly:check` fails, check [certificate requirements](configure_gitaly.md#certificate-requirements) for Gitaly. @@ -123,27 +123,6 @@ sudo cat /proc/$PID/environ | tr '\0' '\n' | grep ^CORRELATION_ID= This method isn't reliable for `git cat-file` processes, because Gitaly internally pools and re-uses those across RPCs. -### Observing `gitaly-ruby` traffic - -[`gitaly-ruby`](configure_gitaly.md#gitaly-ruby) is an internal implementation detail of Gitaly, -so, there's not that much visibility into what goes on inside -`gitaly-ruby` processes. - -If you have Prometheus set up to scrape your Gitaly process, you can see -request rates and error codes for individual RPCs in `gitaly-ruby` by -querying `grpc_client_handled_total`. - -All gRPC calls made by `gitaly-ruby` itself are internal calls from the main Gitaly process to one of its `gitaly-ruby` -sidecars. - -Assuming your `grpc_client_handled_total` counter only observes Gitaly, -the following query shows you RPCs are (most likely) internally -implemented as calls to `gitaly-ruby`: - -```prometheus -sum(rate(grpc_client_handled_total[5m])) by (grpc_method) > 0 -``` - ### Repository changes fail with a `401 Unauthorized` error If you run Gitaly on its own server and notice these conditions: @@ -492,7 +471,7 @@ in sync so the token check succeeds. This check helps identify the root cause of `permission denied` [errors being logged by Praefect](#permission-denied-errors-appearing-in-gitaly-or-praefect-logs-when-accessing-repositories). -For offline environments where access to public [`pool.ntp.org`](https://pool.ntp.org) servers is not possible, the Praefect `check` sub-command fails this +For offline environments where access to public `pool.ntp.org` servers is not possible, the Praefect `check` sub-command fails this check with an error message similar to: ```plaintext diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md index ea051e2067d..3b4eaeb161c 100644 --- a/doc/administration/incoming_email.md +++ b/doc/administration/incoming_email.md @@ -68,8 +68,7 @@ this method only supports replies, and not the other features of [incoming email ## Accepted headers -> - Accepting `Received` headers [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81489) in GitLab 14.9 [with a flag](feature_flags.md) named `use_received_header_for_incoming_emails`. Enabled by default. -> - Accepting `Received` headers: [feature flag](feature_flags.md) named `use_received_header_for_incoming_emails` [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/362596) in GitLab 14.1. +> Accepting `Received` headers [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81489) in GitLab 14.9. Email is processed correctly when a configured email address is present in one of the following headers (sorted in the order they are checked): diff --git a/doc/administration/index.md b/doc/administration/index.md index 68e16b56624..ded6eabdba7 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -7,235 +7,14 @@ description: 'Learn how to install, configure, update, and maintain your GitLab # Administer GitLab **(FREE SELF)** -If you use GitLab.com, only GitLab team members have access to administration tools and settings. -If you use a self-managed GitLab instance, learn how to administer it. +Learn how to administer a self-managed GitLab instance. -Only administrator users can access GitLab administration tools and settings. +- [Get started](../administration/get_started.md) +- [All feature flags](../user/feature_flags.md) +- [Configure your installation](../administration/configure.md) +- [Configure GitLab Dedicated](../administration/dedicated/index.md) +- [Maintain your installation](../administration/operations/index.md) +- [Secure your installation](../security/index.md) +- [Administer users](../user/profile/account/create_accounts.md) -## Available distributions - -GitLab has two product distributions available through [different subscriptions](https://about.gitlab.com/pricing/): - -- The open source [GitLab Community Edition (CE)](https://gitlab.com/gitlab-org/gitlab-foss). -- The open core [GitLab Enterprise Edition (EE)](https://gitlab.com/gitlab-org/gitlab). - -You can [install either GitLab CE or GitLab EE](https://about.gitlab.com/install/ce-or-ee/). -However, the features you have access to depend on your chosen subscription. - -GitLab Community Edition installations have access only to Free features. - -## Installing and maintaining GitLab - -Learn how to install, configure, update, and maintain your GitLab instance. - -### Installing GitLab - -- [Install](../install/index.md): Requirements, directory structures, and installation methods. -- [Reference architectures](reference_architectures/index.md): Add additional resources to support more users. -- [Installing GitLab on Amazon Web Services (AWS)](../install/aws/index.md): Set up GitLab on Amazon AWS. -- [Geo](geo/index.md): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. -- [Disaster Recovery](geo/disaster_recovery/index.md): Quickly fail-over to a different site with minimal effort in a disaster situation. -- [Add License](../user/admin_area/license.md): Add a license at install time to unlock features that are in paid tiers of GitLab. - -### Configuring GitLab - -- [Adjust your instance's time zone](timezone.md): Customize the default time zone of GitLab. -- [System hooks](system_hooks.md): Notifications when users, projects and keys are changed. -- [Security](../security/index.md): Learn what you can do to further secure your GitLab instance. -- [Usage statistics, version check, and Service Ping](../user/admin_area/settings/usage_statistics.md): Enable or disable information about your instance to be sent to GitLab, Inc. -- [Global user settings](user_settings.md): Configure instance-wide user permissions. -- [Polling](polling.md): Configure how often the GitLab UI polls for updates. -- [GitLab Pages configuration](pages/index.md): Enable and configure GitLab Pages. -- [GitLab Pages configuration for GitLab source installations](pages/source.md): - Enable and configure GitLab Pages on [source installations](../install/installation.md#installation-from-source). -- [Uploads administration](uploads.md): Configure GitLab uploads storage. -- [Environment variables](environment_variables.md): Supported environment - variables that can be used to override their default values to configure - GitLab. -- [File hooks](file_hooks.md): With custom file hooks, GitLab administrators can - introduce custom integrations without modifying GitLab source code. -- [Enforcing Terms of Service](../user/admin_area/settings/terms.md) -- [Customer experience improvement and third-party offers](../user/admin_area/settings/third_party_offers.md) -- [Compliance](compliance.md): A collection of features from across the - application that you may configure to help ensure that your GitLab instance - and DevOps workflow meet compliance standards. -- [Diff limits](../user/admin_area/diff_limits.md): Configure the diff rendering - size limits of branch comparison pages. -- [Merge request diffs storage](merge_request_diffs.md): Configure merge - requests diffs external storage. -- [Broadcast Messages](../user/admin_area/broadcast_messages.md): Send messages - to GitLab users through the UI. -- [Elasticsearch](../integration/advanced_search/elasticsearch.md): Enable Elasticsearch to - empower advanced search. Use when you deal with a huge amount of data. -- [External Classification Policy Authorization](../user/admin_area/settings/external_authorization.md) -- [Add a license](../user/admin_area/license.md): Add a license to unlock - features that are in paid tiers of GitLab. -- [Admin Area](../user/admin_area/index.md): for self-managed instance-wide - configuration and maintenance. -- [S/MIME Signing](smime_signing_email.md): how to sign all outgoing notification - emails with S/MIME. -- [Enabling and disabling features flags](feature_flags.md): how to enable and - disable GitLab features deployed behind feature flags. -- [Application settings cache expiry interval](application_settings_cache.md) -- [Database Load Balancing](postgresql/database_load_balancing.md): Distribute database queries among multiple database servers. -- [Omnibus support for log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only). - -#### Customizing GitLab appearance - -- [Header logo](../user/admin_area/appearance.md#top-bar): Change the logo on all pages and email headers. -- [Favicon](../user/admin_area/appearance.md#favicon): Change the default favicon to your own logo. -- [Branded login page](../user/admin_area/appearance.md#sign-in--sign-up-pages): Customize the login page with your own logo, title, and description. -- ["New Project" page](../user/admin_area/appearance.md#new-project-pages): Customize the text to be displayed on the page that opens whenever your users create a new project. -- [Additional custom email text](../user/admin_area/settings/email.md#custom-additional-text): Add additional custom text to emails sent from GitLab. - -### Maintaining GitLab - -- [Rake tasks](../raketasks/index.md): Perform various tasks for maintenance, backups, automatic webhooks setup, and more. - - [Backup and restore](../raketasks/backup_restore.md): Backup and restore your GitLab instance. -- [Operations](operations/index.md): Keeping GitLab up and running (clean up Redis sessions, moving repositories, Sidekiq MemoryKiller, Puma). -- [Restart GitLab](restart_gitlab.md): Learn how to restart GitLab and its components. -- [Invalidate Markdown cache](invalidate_markdown_cache.md): Invalidate any cached Markdown. -- [Instance review](instance_review.md): Request a free review of your GitLab instance. - -#### Updating GitLab - -- [GitLab versions and maintenance policy](../policy/maintenance.md): Understand GitLab versions and releases (Major, Minor, Patch, Security), as well as update recommendations. -- [GitLab in maintenance mode](maintenance_mode/index.md): Put GitLab in maintenance mode. -- [Update GitLab](../update/index.md): Update guides to upgrade your installation to a new version. -- [Upgrading without downtime](../update/index.md#upgrading-without-downtime): Upgrade to a newer major, minor, or patch version of GitLab without taking your GitLab instance offline. - -### Upgrading or downgrading GitLab - -- [Upgrade from GitLab CE to GitLab EE](../update/index.md#upgrading-between-editions): Learn how to upgrade GitLab Community Edition to GitLab Enterprise Editions. -- [Downgrade from GitLab EE to GitLab CE](../downgrade_ee_to_ce/index.md): Learn how to downgrade GitLab Enterprise Editions to Community Edition. - -### GitLab platform integrations - -- [Mattermost](../integration/mattermost/index.md): Integrate with [Mattermost](https://mattermost.com), an open source, private cloud workplace for web messaging. -- [PlantUML](integration/plantuml.md): Create diagrams in AsciiDoc and Markdown documents - created in snippets, wikis, and repositories. -- [Web terminals](integration/terminal.md): Provide terminal access to your applications deployed to Kubernetes from GitLab CI/CD [environments](../ci/environments/index.md#web-terminals-deprecated). - -## User settings and permissions - -- [Creating users](../user/profile/account/create_accounts.md): Create users manually or through authentication integrations. -- [Libravatar](libravatar.md): Use Libravatar instead of Gravatar for user avatars. -- [Sign-up restrictions](../user/admin_area/settings/sign_up_restrictions.md): block email addresses of specific domains, or allow only specific domains. -- [Admin mode](../user/admin_area/settings/sign_in_restrictions.md#admin-mode): require that administrators authenticate separately to use administrative access, like `sudo`. -- [Access restrictions](../user/admin_area/settings/visibility_and_access_controls.md#configure-enabled-git-access-protocols): Define which Git access protocols can be used to talk to GitLab (SSH, HTTP, HTTPS). -- [Authentication and Authorization](auth/index.md): Configure external authentication with LDAP, SAML, CAS, and additional providers. - - [Sync LDAP](auth/ldap/index.md) - - [Kerberos authentication](../integration/kerberos.md) - - See also other [authentication](../topics/authentication/index.md#gitlab-administrators) topics (for example, enforcing 2FA). -- [Email users](../user/admin_area/email_from_gitlab.md): Email GitLab users from GitLab. -- [User Cohorts](../user/admin_area/user_cohorts.md): Display the monthly cohorts of new users and their activities over time. -- [Audit events](audit_events.md): View the changes made on the GitLab server for: - - Groups and projects. - - Instances. -- [Auditor users](auditor_users.md): Users with read-only access to all projects, groups, and other resources on the GitLab instance. -- [Incoming email](incoming_email.md): Configure incoming emails to allow - users to [reply by email](reply_by_email.md), create [issues by email](../user/project/issues/create_issues.md#by-sending-an-email) and - [merge requests by email](../user/project/merge_requests/creating_merge_requests.md#by-sending-an-email), and to enable [Service Desk](../user/project/service_desk.md). - - [Postfix for incoming email](reply_by_email_postfix_setup.md): Set up a - basic Postfix mail server with IMAP authentication on Ubuntu for incoming - emails. -- [Abuse reports](../user/admin_area/review_abuse_reports.md): View and resolve abuse reports from your users. -- [Credentials Inventory](../user/admin_area/credentials_inventory.md): With Credentials inventory, GitLab administrators can keep track of the credentials used by their users in their GitLab self-managed instance. - -## Project settings - -- [Issue closing pattern](issue_closing_pattern.md): Customize how to close an issue from commit messages. -- [Gitaly](gitaly/index.md): Configuring Gitaly, the Git repository storage service for GitLab. -- [Default labels](../user/admin_area/labels.md): Create labels that are automatically added to every new project. -- [Restrict the use of public or internal projects](../user/public_access.md#restrict-use-of-public-or-internal-projects): Restrict the use of visibility levels for users when they create a project or a snippet. -- [Custom project templates](../user/admin_area/custom_project_templates.md): Configure a set of projects to be used as custom templates when creating a new project. - -## Package Registry administration - -- [Container Registry](packages/container_registry.md): Configure GitLab to act as a registry for containers. -- [Package Registry](packages/index.md): Enable GitLab to act as a registry for packages. -- [Dependency Proxy](packages/dependency_proxy.md): Configure the Dependency Proxy, a local proxy for frequently used upstream images/packages. - -### Repository settings - -- [Repository checks](repository_checks.md): Check your repository for data corruption. -- [Repository storage paths](repository_storage_paths.md): Manage the paths used to store repositories. -- [Repository storage types](repository_storage_types.md): Information about the different repository storage types. -- [Repository storage Rake tasks](raketasks/storage.md): A collection of Rake tasks to list and migrate existing projects and attachments associated with it from Legacy storage to Hashed storage. -- [Limit repository size](../user/admin_area/settings/account_and_limit_settings.md): Set a hard limit for your repositories' size. -- [Static objects external storage](static_objects_external_storage.md): Set external storage for static objects in a repository. - -## CI/CD settings - -- [Enable or disable GitLab CI/CD](cicd.md#disable-gitlab-cicd-in-new-projects): Set new projects in your instance to have GitLab CI/CD enabled or disabled by default. -- [GitLab CI/CD administration settings](../user/admin_area/settings/continuous_integration.md): Enable or disable Auto DevOps site-wide and define the artifacts' max size and expiration time. -- [External Pipeline Validation](external_pipeline_validation.md): Enable, disable, and configure external pipeline validation. -- [Job artifacts](job_artifacts.md): Enable, disable, and configure job artifacts (a set of files and directories which are outputted by a job when it completes successfully). -- [Job logs](job_logs.md): Information about the job logs. -- [Register runners](../ci/runners/runners_scope.md): Learn how to register and configure runners. -- [Shared runners quota of CI/CD minutes](../ci/pipelines/cicd_minutes.md): Limit the usage of CI/CD minutes for shared runners. -- [Enable or disable Auto DevOps](../topics/autodevops/index.md#enable-or-disable-auto-devops): Enable or disable Auto DevOps for your instance. - -## Snippet settings - -- [Setting snippet content size limit](snippets/index.md): Set a maximum content size limit for snippets. - -## Wiki settings - -- [Setting wiki page content size limit](wikis/index.md): Set a maximum content size limit for wiki pages. - -## Git configuration options - -- [Git server hooks](server_hooks.md): Git server hooks (on the file system) for when webhooks aren't enough. Previously called server hooks. -- [Git LFS configuration](lfs/index.md): Learn how to configure LFS for GitLab. -- [Housekeeping](housekeeping.md): Keep your Git repositories tidy and fast. -- [Configuring Git Protocol v2](git_protocol.md): Git protocol version 2 support. - -## Monitoring GitLab - -- [Monitoring GitLab](monitoring/index.md): - - [Monitoring uptime](../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint. - - [IP allowlist](monitoring/ip_allowlist.md): Monitor endpoints that provide health check information when probed. - - [Monitoring GitHub imports](monitoring/github_imports.md): The GitLab GitHub Importer displays Prometheus metrics to monitor the health and progress of the importer. - -### Performance Monitoring - -- [GitLab Performance Monitoring](monitoring/performance/index.md): - - [Enable Performance Monitoring](monitoring/performance/gitlab_configuration.md): Enable GitLab Performance Monitoring. - - [GitLab performance monitoring with Prometheus](monitoring/prometheus/index.md): Configure GitLab and Prometheus for measuring performance metrics. - - [GitLab performance monitoring with Grafana](monitoring/performance/grafana_configuration.md): Configure GitLab to visualize time series metrics through graphs and dashboards. - - [Performance Bar](monitoring/performance/performance_bar.md): Get performance information for the current page. - -## Troubleshooting - -- [Log system](logs/index.md): Where to look for logs. -- [Sidekiq Troubleshooting](sidekiq/sidekiq_troubleshooting.md): Debug when Sidekiq appears hung and is not processing jobs. -- [Navigating GitLab via Rails console](operations/rails_console.md) -- [GitLab application limits](instance_limits.md) -- [Responding to security incidents](../security/responding_to_security_incidents.md) - -### Support Team Documentation - -The GitLab Support Team has collected a lot of information about troubleshooting GitLab. -The following documents are used by the Support Team or by customers -with direct guidance from a Support Team member. GitLab administrators may find the -information useful for troubleshooting. However, if you are experiencing trouble with your -GitLab instance, you should check your [support options](https://about.gitlab.com/support/) -before referring to these documents. - -WARNING: -The commands in the following documentation might result in data loss or -other damage to a GitLab instance. They should be used only by experienced administrators -who are aware of the risks. - -- [Diagnostics tools](troubleshooting/diagnostics_tools.md) -- [Linux commands](troubleshooting/linux_cheat_sheet.md) -- [Troubleshooting Kubernetes](https://docs.gitlab.com/charts/troubleshooting/kubernetes_cheat_sheet.html) -- [Troubleshooting PostgreSQL](troubleshooting/postgresql.md) -- [Guide to test environments](troubleshooting/test_environments.md) (for Support Engineers) -- [Troubleshooting SSL](troubleshooting/ssl.md) -- Related links: - - [GitLab Developer Documentation](../development/index.md) - - [Repairing and recovering broken Git repositories](https://git.seveas.net/repairing-and-recovering-broken-git-repositories.html) - - [Testing with OpenSSL](https://www.feistyduck.com/library/openssl-cookbook/online/testing-with-openssl/index.html) - - [`strace` zine](https://wizardzines.com/zines/strace/) +Only GitLab team members have access to administration tools and settings on GitLab.com. diff --git a/doc/administration/instance_limits.md b/doc/administration/instance_limits.md index 2b2eefdb17c..c113d1aeff4 100644 --- a/doc/administration/instance_limits.md +++ b/doc/administration/instance_limits.md @@ -126,7 +126,7 @@ Read more about [import/export rate limits](../user/admin_area/settings/import_e Limit the maximum daily member invitations allowed per group hierarchy. -- GitLab.com: Free members may invite 20 members per day. +- GitLab.com: Free members may invite 20 members per day, Premium trial and Ultimate trial members may invite 50 members per day. - Self-managed: Invites are not limited. ### Webhook rate limit @@ -632,6 +632,42 @@ To update this limit to a new value on a self-managed installation, run the foll Plan.default.actual_limits.update!(ci_instance_level_variables: 30) ``` +### Number of group level variables + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362227) in GitLab 15.7. + +The total number of group level CI/CD variables is limited at the instance level. +This limit is checked each time a new group level variable is created. If a new variable +would cause the total number of variables to exceed the limit, the new variable is not created. + +On self-managed instances this limit is defined for the `default` plan. By default, +this limit is set to `30000`. + +To update this limit to a new value on a self-managed installation, run the following in the +[GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): + +```ruby +Plan.default.actual_limits.update!(group_ci_variables: 40000) +``` + +### Number of project level variables + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362227) in GitLab 15.7. + +The total number of project level CI/CD variables is limited at the instance level. +This limit is checked each time a new project level variable is created. If a new variable +would cause the total number of variables to exceed the limit, the new variable is not created. + +On self-managed instances this limit is defined for the `default` plan. By default, +this limit is set to `8000`. + +To update this limit to a new value on a self-managed installation, run the following in the +[GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): + +```ruby +Plan.default.actual_limits.update!(project_ci_variables: 10000) +``` + ### Maximum file size per type of artifact > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37226) in GitLab 13.3. diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 0ffb83886ed..df9bb6b6e17 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 0a4213b7e2d..e697051f100 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Jobs artifacts administration **(FREE SELF)** This is the administration documentation. To learn how to use job artifacts in your GitLab CI/CD pipeline, -see the [job artifacts configuration documentation](../ci/pipelines/job_artifacts.md). +see the [job artifacts configuration documentation](../ci/jobs/job_artifacts.md). An artifact is a list of files and directories attached to a job after it finishes. This feature is enabled by default in all GitLab installations. @@ -174,7 +174,7 @@ In a multi-server setup you must use one of the options to [eliminate local disk usage for job logs](job_logs.md#prevent-local-disk-usage), or job logs could be lost. In GitLab 13.2 and later, you should use the -[consolidated object storage settings](object_storage.md#consolidated-object-storage-configuration). +[consolidated object storage settings](object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). ### Migrating to object storage @@ -419,6 +419,8 @@ reasons are: to remove these. This script should always find work to do, as it also removes empty directories (see above). - [Artifact housekeeping was changed significantly](#artifacts-housekeeping-disabled-in-gitlab-146-to-152), and you might need to enable a feature flag to used the updated system. +- The [keep latest artifacts from most recent success jobs](../ci/jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs) + feature is enabled. In these and other cases, identify the projects most responsible for disk space usage, figure out what types of artifacts are using the most @@ -461,7 +463,7 @@ To check if the feature flags are enabled: ``` These changes include switching artifacts from `unlocked` to `locked` if -they [should be retained](../ci/pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). +they [should be retained](../ci/jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). Artifacts created before this feature was introduced have a status of `unknown`. After they expire, these artifacts are not processed by the new housekeeping jobs. @@ -673,7 +675,7 @@ If you need to manually remove job artifacts associated with multiple jobs while NOTE: This step also erases artifacts that users have chosen to - ["keep"](../ci/pipelines/job_artifacts.md#download-job-artifacts). + ["keep"](../ci/jobs/job_artifacts.md#download-job-artifacts). ```ruby builds_to_clear = builds_with_artifacts.where("finished_at < ?", 1.week.ago) @@ -779,7 +781,7 @@ In both cases, you might need to add `region` to the job artifact [object storag ### Job artifact upload fails with `500 Internal Server Error (Missing file)` -Bucket names that include folder paths are not supported with [consolidated object storage](object_storage.md#consolidated-object-storage-configuration). +Bucket names that include folder paths are not supported with [consolidated object storage](object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). For example, `bucket/path`. If a bucket name has a path in it, you might receive an error similar to: ```plaintext @@ -807,3 +809,18 @@ To work around this issue, you can try: - For older kernels, using the `nolease` mount option to disable file leasing. For more information, [see the investigation details](https://gitlab.com/gitlab-org/gitlab/-/issues/389995). + +### Usage quota shows incorrect artifact storage usage + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/238536) in GitLab 14.10. + +Sometimes the [artifacts storage usage](../user/usage_quotas.md) displays an incorrect +value for the total storage space used by artifacts. To recalculate the artifact +usage statistics for all projects in the instance, you can run this background script: + +```shell +bin/rake 'gitlab:refresh_project_statistics_build_artifacts_size[file.csv]' +``` + +The artifact usage value can fluctuate to `0` while the script is running. After +recalculation, usage should display as expected again. diff --git a/doc/administration/lfs/index.md b/doc/administration/lfs/index.md index adfba28520e..ee5cd04f3d6 100644 --- a/doc/administration/lfs/index.md +++ b/doc/administration/lfs/index.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/lfs/lfs_administration.html' --- # GitLab Git Large File Storage (LFS) Administration **(FREE SELF)** @@ -157,14 +156,14 @@ You can store LFS objects in remote object storage. This allows you to reduce reads and writes to the local disk, and free up disk space significantly. In GitLab 13.2 and later, you should use the -[consolidated object storage settings](../object_storage.md#consolidated-object-storage-configuration). +[consolidated object storage settings](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). ### Migrating to object storage You can migrate the LFS objects from local storage to object storage. The processing is done in the background and requires **no downtime**. -1. [Configure the object storage](../object_storage.md#consolidated-object-storage-configuration). +1. [Configure the object storage](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). 1. Migrate the LFS objects: ::Tabs diff --git a/doc/administration/logs/index.md b/doc/administration/logs/index.md index 1104412020a..c03971ab4c1 100644 --- a/doc/administration/logs/index.md +++ b/doc/administration/logs/index.md @@ -686,14 +686,6 @@ Unix timestamp format, and `gzip`-compressed (like `@1584057562.s`). This file is at `/var/log/gitlab/gitlab-rails/grpc.log` for Omnibus GitLab packages. Native [gRPC](https://grpc.io/) logging used by Gitaly. -### `gitaly_ruby_json.log` - -> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2678) in GitLab 13.6. - -This file is at `/var/log/gitlab/gitaly/gitaly_ruby_json.log` and is -produced by [`gitaly-ruby`](../gitaly/reference.md#gitaly-ruby). It contains an -access log of gRPC calls made by Gitaly to `gitaly-ruby`. - ### `gitaly_hooks.log` This file is at `/var/log/gitlab/gitaly/gitaly_hooks.log` and is @@ -733,7 +725,7 @@ Depending on your installation method, this file is located at: - Omnibus GitLab: `/var/log/gitlab/gitlab-rails/importer.log` - Installations from source: `/home/git/gitlab/log/importer.log` -It logs the progress of the import process. +This file logs the progress of [project imports and migrations](../../user/project/import/index.md). ## `exporter.log` @@ -829,6 +821,8 @@ Depending on your installation method, this file is located at: - Omnibus GitLab: `/var/log/gitlab/gitlab-rails/migrations.log` - Installations from source: `/home/git/gitlab/log/migrations.log` +This file logs the progress of [database migrations](../raketasks/maintenance.md#display-status-of-database-migrations). + ## `mail_room_json.log` (default) > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/19186) in GitLab 12.6. diff --git a/doc/administration/logs/log_parsing.md b/doc/administration/logs/log_parsing.md index 84c517e6120..40d6059d246 100644 --- a/doc/administration/logs/log_parsing.md +++ b/doc/administration/logs/log_parsing.md @@ -158,7 +158,7 @@ CT: 297 ROUTE: /api/:version/projects/:id/repository/tags DURS: 731.39, CT: 190 ROUTE: /api/:version/projects/:id/repository/commits DURS: 1079.02, 979.68, 958.21 ``` -### Print top API user agents +#### Print top API user agents ```shell jq --raw-output '[.route, .ua] | @tsv' api_json.log | sort | uniq -c | sort -n @@ -180,9 +180,20 @@ if the output contains many: You can also [use `fast-stats top`](#parsing-gitlab-logs-with-jq) to extract performance statistics. +### Parsing `gitlab-rails/importer.log` + +To troubleshoot [project imports](../../administration/raketasks/project_import_export.md) or +[migrations](../../user/project/import/index.md), run this command: + +```shell +jq 'select(.project_path == "<namespace>/<project>").error_messages' importer.log +``` + +For common issues, see [troubleshooting](../../administration/raketasks/project_import_export.md#troubleshooting). + ### Parsing `gitlab-workhorse/current` -### Print top Workhorse user agents +#### Print top Workhorse user agents ```shell jq --raw-output '[.uri, .user_agent] | @tsv' current | sort | uniq -c | sort -n diff --git a/doc/administration/merge_request_diffs.md b/doc/administration/merge_request_diffs.md index 85677512860..3fbcdafd886 100644 --- a/doc/administration/merge_request_diffs.md +++ b/doc/administration/merge_request_diffs.md @@ -105,7 +105,7 @@ be configured already. ### Object Storage Settings In GitLab 13.2 and later, you should use the -[consolidated object storage settings](object_storage.md#consolidated-object-storage-configuration). +[consolidated object storage settings](object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). This section describes the earlier configuration format. For source installations, these settings are nested under `external_diffs:` and diff --git a/doc/administration/monitoring/github_imports.md b/doc/administration/monitoring/github_imports.md index 097109585af..3d29626a132 100644 --- a/doc/administration/monitoring/github_imports.md +++ b/doc/administration/monitoring/github_imports.md @@ -1,6 +1,6 @@ --- -stage: Monitor -group: Respond +stage: Manage +group: Import info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/performance/index.md b/doc/administration/monitoring/performance/index.md index 89f1270532a..cbdd68804e8 100644 --- a/doc/administration/monitoring/performance/index.md +++ b/doc/administration/monitoring/performance/index.md @@ -1,6 +1,6 @@ --- -stage: Monitor -group: Respond +stage: Data Stores +group: Application Performance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index c21c92a3d63..4a523ad75ec 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -1,6 +1,6 @@ --- -stage: Monitor -group: Respond +stage: Shared responsibility based on functional area +group: Shared responsibility based on functional area info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -30,6 +30,8 @@ For enabling and viewing metrics from Sidekiq nodes, see [Sidekiq metrics](#side ## Metrics available +> `caller_id` [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/392622) from `redis_hit_miss_operations_total` and `redis_cache_generation_duration_seconds` in GitLab 15.11. + The following metrics are available: | Metric | Type | Since | Description | Labels | @@ -133,6 +135,8 @@ The following metrics are available: | `gitlab_issuable_fast_count_by_state_total` | Counter | 13.5 | Total number of row count operations on issue/merge request list pages | | | `gitlab_issuable_fast_count_by_state_failures_total` | Counter | 13.5 | Number of soft-failed row count operations on issue/merge request list pages | | | `gitlab_ci_trace_finalize_duration_seconds` | Histogram | 13.6 | Duration of build trace chunks migration to object storage | | +| `gitlab_vulnerability_report_branch_comparison_real_duration_seconds` | Histogram | 15.11 | Execution duration of vulnerability report present on default branch sql query | | +| `gitlab_vulnerability_report_branch_comparison_cpu_duration_seconds` | Histogram | 15.11 | Execution duration of vulnerability report present on default branch sql query | | | `gitlab_external_http_total` | Counter | 13.8 | Total number of HTTP calls to external systems | `controller`, `action` | | `gitlab_external_http_duration_seconds` | Counter | 13.8 | Duration in seconds spent on each HTTP call to external systems | | | `gitlab_external_http_exception_total` | Counter | 13.8 | Total number of exceptions raised when making external HTTP calls | | @@ -151,8 +155,8 @@ The following metrics are available: | `gitlab_ci_build_trace_errors_total` | Counter | 14.4 | Total amount of different error types on a build trace | `error_reason` | | `gitlab_presentable_object_cacheless_render_real_duration_seconds` | Histogram | 15.3 | Duration of real time spent caching and representing specific web request objects | `controller`, `action` | | `cached_object_operations_total` | Counter | 15.3 | Total number of objects cached for specific web requests | `controller`, `action` | -| `redis_hit_miss_operations_total` | Counter | 15.6 | Total number of Redis cache hits and misses | `cache_hit`, `caller_id`, `cache_identifier`, `feature_category`, `backing_resource` | -| `redis_cache_generation_duration_seconds` | Histogram | 15.6 | Time to generate Redis cache | `cache_hit`, `caller_id`, `cache_identifier`, `feature_category`, `backing_resource` | +| `redis_hit_miss_operations_total` | Counter | 15.6 | Total number of Redis cache hits and misses | `cache_hit`, `cache_identifier`, `feature_category`, `backing_resource` | +| `redis_cache_generation_duration_seconds` | Histogram | 15.6 | Time to generate Redis cache | `cache_hit`, `cache_identifier`, `feature_category`, `backing_resource` | | `gitlab_diffs_reorder_real_duration_seconds` | Histogram | 15.8 | Duration in seconds spend on reordering of diff files on diffs batch request | `controller`, `action` | | `gitlab_diffs_collection_real_duration_seconds` | Histogram | 15.8 | Duration in seconds spent on querying merge request diff files on diffs batch request | `controller`, `action` | | `gitlab_diffs_comparison_real_duration_seconds` | Histogram | 15.8 | Duration in seconds spent on getting comparison data on diffs batch request | `controller`, `action` | @@ -171,7 +175,7 @@ The following metrics are available: The following metrics can be controlled by feature flags: -| Metric | Feature Flag | +| Metric | Feature flag | |:---------------------------------------------------------------|:-------------------------------------------------------------------| | `gitlab_view_rendering_duration_seconds` | `prometheus_metrics_view_instrumentation` | @@ -359,9 +363,20 @@ configuration option in `gitlab.yml`. These metrics are served from the | `geo_dependency_proxy_manifests_verification_total` | Gauge | 15.6 | Number of dependency proxy manifests to attempt to verify on secondary | `url` | | `geo_dependency_proxy_manifests_verified` | Gauge | 15.6 | Number of dependency proxy manifests successfully verified on secondary | `url` | | `geo_dependency_proxy_manifests_verification_failed` | Gauge | 15.6 | Number of dependency proxy manifests that failed verification on secondary | `url` | +| `geo_project_wiki_repositories` | Gauge | 15.10 | Number of Project Wiki Repositories on primary | `url` | +| `geo_project_wiki_repositories_checksum_total` | Gauge | 15.10 | Number of Project Wiki Repositories to checksum on primary | `url` | +| `geo_project_wiki_repositories_checksummed` | Gauge | 15.10 | Number of Project Wiki Repositories that successfully calculated the checksum on primary | `url` | +| `geo_project_wiki_repositories_checksum_failed` | Gauge | 15.10 | Number of Project Wiki Repositories that failed to calculate the checksum on primary | `url` | +| `geo_project_wiki_repositories_synced` | Gauge | 15.10 | Number of syncable Project Wiki Repositories synced on secondary | `url` | +| `geo_project_wiki_repositories_failed` | Gauge | 15.10 | Number of syncable Project Wiki Repositories failed to sync on secondary | `url` | +| `geo_project_wiki_repositories_registry` | Gauge | 15.10 | Number of Project Wiki Repositories in the registry | `url` | +| `geo_project_wiki_repositories_verification_total` | Gauge | 15.10 | Number of Project Wiki Repositories to attempt to verify on secondary | `url` | +| `geo_project_wiki_repositories_verified` | Gauge | 15.10 | Number of Project Wiki Repositories successfully verified on secondary | `url` | +| `geo_project_wiki_repositories_verification_failed` | Gauge | 15.10 | Number of Project Wiki Repositories that failed verification on secondary | `url` | | `gitlab_memwd_violations_total` | Counter | 15.9 | Total number of times a Sidekiq process violated a memory threshold | | | `gitlab_memwd_violations_handled_total` | Counter | 15.9 | Total number of times Sidekiq process memory violations were handled | | | `sidekiq_watchdog_running_jobs_total` | Counter | 15.9 | Current running jobs when RSS limit was reached | `worker_class` | +| `gitlab_maintenance_mode` | Gauge | 15.11 | Is GitLab Maintenance Mode enabled? | | ## Database load balancing metrics **(PREMIUM SELF)** diff --git a/doc/administration/monitoring/prometheus/node_exporter.md b/doc/administration/monitoring/prometheus/node_exporter.md index c2f84773b3a..337ab9becaa 100644 --- a/doc/administration/monitoring/prometheus/node_exporter.md +++ b/doc/administration/monitoring/prometheus/node_exporter.md @@ -1,6 +1,6 @@ --- -stage: Monitor -group: Respond +stage: Shared responsibility based on functional area +group: Shared responsibility based on functional area info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md index 56da3155fd9..f8d48832288 100644 --- a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md +++ b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md @@ -1,6 +1,6 @@ --- -stage: Monitor -group: Respond +stage: Data Stores +group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/postgres_exporter.md b/doc/administration/monitoring/prometheus/postgres_exporter.md index 0458a4a5bae..1cae28a069f 100644 --- a/doc/administration/monitoring/prometheus/postgres_exporter.md +++ b/doc/administration/monitoring/prometheus/postgres_exporter.md @@ -1,6 +1,6 @@ --- -stage: Monitor -group: Respond +stage: Data Stores +group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/redis_exporter.md b/doc/administration/monitoring/prometheus/redis_exporter.md index b50a2a4aebf..b17bf9787a7 100644 --- a/doc/administration/monitoring/prometheus/redis_exporter.md +++ b/doc/administration/monitoring/prometheus/redis_exporter.md @@ -1,6 +1,6 @@ --- -stage: Monitor -group: Respond +stage: Shared responsibility based on functional area +group: Shared responsibility based on functional area info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/registry_exporter.md b/doc/administration/monitoring/prometheus/registry_exporter.md index b79e97bd937..3f4d6cfb14d 100644 --- a/doc/administration/monitoring/prometheus/registry_exporter.md +++ b/doc/administration/monitoring/prometheus/registry_exporter.md @@ -1,6 +1,6 @@ --- -stage: Monitor -group: Respond +stage: Package +group: Package Registry info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/administration/nfs.md b/doc/administration/nfs.md index 7349e8ad9ba..3e3712c9645 100644 --- a/doc/administration/nfs.md +++ b/doc/administration/nfs.md @@ -12,7 +12,7 @@ recommended for performance reasons. For data objects such as LFS, Uploads, and Artifacts, an [Object Storage service](object_storage.md) is recommended over NFS where possible, due to better performance. -When eliminating the usage of NFS, there are [additional steps you need to take](object_storage.md#other-alternatives-to-file-system-storage) +When eliminating the usage of NFS, there are [additional steps you need to take](object_storage.md#alternatives-to-file-system-storage) in addition to moving to Object Storage. File system performance can impact overall GitLab performance, especially for diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md index 587da749766..848ee8de951 100644 --- a/doc/administration/object_storage.md +++ b/doc/administration/object_storage.md @@ -11,11 +11,24 @@ It's recommended over NFS and in general it's better in larger setups as object storage is typically much more performant, reliable, and scalable. +To configure the object storage, you have two options: + +- Recommended. [Configure a single storage connection for all object types](#configure-a-single-storage-connection-for-all-object-types-consolidated-form): + A single credential is shared by all supported object types. This is called the consolidated form. +- [Configure each object type to define its own storage connection](#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form): + Every object defines its own object storage connection and configuration. This is called the storage-specific form. + + If you already use the storage-specific form, see how to + [transition to the consolidated form](#transition-to-consolidated-form). + +If you store data locally, see how to +[migrate to object storage](#migrate-to-object-storage). + ## Supported object storage providers -GitLab is tightly integrated with `Fog`, so you can refer to its -[documentation](https://fog.io/about/provider_documentation.html) to check -which storage services can be integrated with GitLab. +GitLab is tightly integrated with the Fog library, so you can see which +[providers](https://fog.io/about/provider_documentation.html) can be used +with GitLab. Specifically, GitLab has been tested by vendors and customers on a number of object storage providers: @@ -23,57 +36,45 @@ Specifically, GitLab has been tested by vendors and customers on a number of obj is not supported, see [issue #335775](https://gitlab.com/gitlab-org/gitlab/-/issues/335775) for more information) - [Google Cloud Storage](https://cloud.google.com/storage) -- [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces) +- [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces) (S3 compatible) - [Oracle Cloud Infrastructure](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm) - [OpenStack Swift (S3 compatible mode)](https://docs.openstack.org/swift/latest/s3_compat.html) - [Azure Blob storage](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) +- [MinIO](https://min.io/) (S3 compatible) - On-premises hardware and appliances from various storage vendors, whose list is not officially established. -- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation. - -## Configure the object storage - -There are two ways of specifying object storage configuration in GitLab: -- [Consolidated form](#consolidated-object-storage-configuration): A single credential is - shared by all supported object types. -- [Storage-specific form](#storage-specific-configuration): Every object defines its - own object storage [connection and configuration](#connection-settings). - -For more information on the differences and to transition from one form to another, see -[Transition to consolidated form](#transition-to-consolidated-form). - -If you are currently storing data locally, see -[Migrate to object storage](#migrate-to-object-storage) for migration details. - -### Consolidated object storage configuration +## Configure a single storage connection for all object types (consolidated form) > [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4368) in GitLab 13.2. -Using the consolidated object storage configuration has a number of advantages: +Most types of objects, such as CI artifacts, LFS files, and upload attachments +can be saved in object storage by specifying a single credential for object +storage with multiple buckets. + +Configuring the object storage using the consolidated form has a number of advantages: - It can simplify your GitLab configuration since the connection details are shared across object types. - It enables the use of [encrypted S3 buckets](#encrypted-s3-buckets). - It [uploads files to S3 with proper `Content-MD5` headers](https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/222). -Because [direct upload mode](../development/uploads/index.md#direct-upload) -must be enabled, only the following providers can be used: +When the consolidated form is used, +[direct upload](../development/uploads/index.md#direct-upload) is enabled +automatically. Thus, only the following providers can be used: - [Amazon S3-compatible providers](#amazon-s3) - [Google Cloud Storage](#google-cloud-storage-gcs) - [Azure Blob storage](#azure-blob-storage) -When consolidated object storage is used, direct upload is enabled -automatically. For storage-specific -configuration, [direct upload may become the default](https://gitlab.com/gitlab-org/gitlab/-/issues/27331) -because it does not require a shared folder. +The consolidated form configuration can't be used for backups or +Mattermost. Backups can be configured with +[server side encryption](../raketasks/backup_gitlab.md#s3-encrypted-buckets) +separately. See the +[table for a complete list](#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form) +of supported object storage types. -Consolidated object storage configuration can't be used for backups or -Mattermost. See the [full table for a complete list](#storage-specific-configuration). -However, backups can be configured with [server side encryption](../raketasks/backup_gitlab.md#s3-encrypted-buckets) separately. - -Enabling consolidated object storage enables object storage for all object -types. If not all buckets are specified, `sudo gitlab-ctl reconfigure` may fail with the error like: +Enabling the consolidated form enables object storage for all object +types. If not all buckets are specified, you may see an error like: ```plaintext Object storage for <object type> must have a bucket specified @@ -82,20 +83,345 @@ Object storage for <object type> must have a bucket specified If you want to use local storage for specific object types, you can [disable object storage for specific features](#disable-object-storage-for-specific-features). -Most types of objects, such as CI artifacts, LFS files, and upload -attachments can be saved in object storage by specifying a single -credential for object storage with multiple buckets. +### Configure the common parameters -When the consolidated form is: +In the consolidated form, the `object_store` section defines a +common set of parameters. -- Used with an S3-compatible object storage, Workhorse uses its internal S3 client to - upload files. -- Not used with an S3-compatible object storage, Workhorse falls back to using - pre-signed URLs. +| Setting | Description | +|-------------------|-----------------------------------| +| `enabled` | Enable or disable object storage. | +| `proxy_download` | Set to `true` to [enable proxying all files served](#proxy-download). Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data. | +| `connection` | Various [connection options](#connection-settings) described below. | +| `storage_options` | Options to use when saving new objects, such as [server side encryption](#server-side-encryption-headers). Introduced in GitLab 13.3. | +| `objects` | [Object-specific configuration](#configure-the-parameters-of-each-object). | + +The following YAML is from the source +installation, to help you see the inheritance: + +```yaml + object_store: + enabled: true + proxy_download: true + connection: + provider: AWS + aws_access_key_id: <AWS_ACCESS_KEY_ID> + aws_secret_access_key: <AWS_SECRET_ACCESS_KEY> + objects: + ... +``` + +The Omnibus configuration maps directly to this: + +```ruby +gitlab_rails['object_store']['enabled'] = true +gitlab_rails['object_store']['proxy_download'] = true +gitlab_rails['object_store']['connection'] = { + 'provider' => 'AWS', + 'aws_access_key_id' => '<AWS_ACCESS_KEY_ID', + 'aws_secret_access_key' => '<AWS_SECRET_ACCESS_KEY>' +} +``` + +### Configure the parameters of each object + +Each object type must at least define the bucket name where it will be stored. + +The following table lists the valid `objects` that can be used: + +| Type | Description | +|--------------------|----------------------------------------------------------------------------| +| `artifacts` | [CI artifacts](job_artifacts.md) | +| `external_diffs` | [Merge request diffs](merge_request_diffs.md) | +| `uploads` | [User uploads](uploads.md) | +| `lfs` | [Git Large File Storage objects](lfs/index.md) | +| `packages` | [Project packages (for example, PyPI, Maven, or NuGet)](packages/index.md) | +| `dependency_proxy` | [Dependency Proxy](packages/dependency_proxy.md) | +| `terraform_state` | [Terraform state files](terraform_state.md) | +| `pages` | [Pages](pages/index.md) | + +Within each object type, three parameters can be defined: + +| Setting | Required? | Description | +|------------------|------------------------|-------------------------------------| +| `bucket` | **{check-circle}** Yes\* | Bucket name for the object type. Not required if `enabled` is set to `false`. | +| `enabled` | **{dotted-circle}** No | Overrides the [common parameter](#configure-the-common-parameters). | +| `proxy_download` | **{dotted-circle}** No | Overrides the [common parameter](#configure-the-common-parameters). | + +The following YAML shows how the `object_store` section defines +object-specific configuration block and how the `enabled` and +`proxy_download` flags can be overridden. The `bucket` is the only +required parameter within each type: + +```yaml + object_store: + connection: + ... + objects: + artifacts: + bucket: artifacts + proxy_download: false + external_diffs: + bucket: external-diffs + lfs: + bucket: lfs-objects + uploads: + bucket: uploads + packages: + bucket: packages + dependency_proxy: + enabled: false + bucket: dependency_proxy + terraform_state: + bucket: terraform + pages: + bucket: pages +``` + +This maps to this Omnibus GitLab configuration: + +```ruby +gitlab_rails['object_store']['objects']['artifacts']['bucket'] = 'artifacts' +gitlab_rails['object_store']['objects']['artifacts']['proxy_download'] = false +gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = 'external-diffs' +gitlab_rails['object_store']['objects']['lfs']['bucket'] = 'lfs-objects' +gitlab_rails['object_store']['objects']['uploads']['bucket'] = 'uploads' +gitlab_rails['object_store']['objects']['packages']['bucket'] = 'packages' +gitlab_rails['object_store']['objects']['dependency_proxy']['enabled'] = false +gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = 'dependency-proxy' +gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = 'terraform-state' +gitlab_rails['object_store']['objects']['pages']['bucket'] = 'pages' +``` + +#### Disable object storage for specific features + +As seen above, object storage can be disabled for specific types by +setting the `enabled` flag to `false`. For example, to disable object +storage for CI artifacts: + +```ruby +gitlab_rails['object_store']['objects']['artifacts']['enabled'] = false +``` + +A bucket is not needed if the feature is disabled entirely. For example, +no bucket is needed if CI artifacts are disabled with this setting: + +```ruby +gitlab_rails['artifacts_enabled'] = false +``` + +## Configure each object type to define its own storage connection (storage-specific form) + +With the storage-specific form, every object defines its own object +storage connection and configuration. If you're using GitLab 13.2 and later, +you should [transition to the consolidated form](#transition-to-consolidated-form). + +The use of [encrypted S3 buckets](#encrypted-s3-buckets) with non-consolidated form is not supported. +You may get [ETag mismatch errors](#etag-mismatch) if you use it. + +NOTE: +For the storage-specific form, +[direct upload may become the default](https://gitlab.com/gitlab-org/gitlab/-/issues/27331) +because it does not require a shared folder. + +For configuring object storage in GitLab 13.1 and earlier, or for storage types not +supported by consolidated form, refer to the following guides: + +| Object storage type | Supported by consolidated form? | +|---------------------|------------------------------------------| +| [Backups](../raketasks/backup_gitlab.md#upload-backups-to-a-remote-cloud-storage) | **{dotted-circle}** No | +| [Container Registry](packages/container_registry.md#use-object-storage) (optional feature) | **{dotted-circle}** No | +| [Mattermost](https://docs.mattermost.com/configure/file-storage-configuration-settings.html)| **{dotted-circle}** No | +| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | **{dotted-circle}** No | +| [Job artifacts](job_artifacts.md#using-object-storage) including archived job logs | **{check-circle}** Yes | +| [LFS objects](lfs/index.md#storing-lfs-objects-in-remote-object-storage) | **{check-circle}** Yes | +| [Uploads](uploads.md#using-object-storage) | **{check-circle}** Yes | +| [Merge request diffs](merge_request_diffs.md#using-object-storage) | **{check-circle}** Yes | +| [Packages](packages/index.md#use-object-storage) (optional feature) | **{check-circle}** Yes | +| [Dependency Proxy](packages/dependency_proxy.md#using-object-storage) (optional feature) | **{check-circle}** Yes | +| [Terraform state files](terraform_state.md#using-object-storage) | **{check-circle}** Yes | +| [Pages content](pages/index.md#using-object-storage) | **{check-circle}** Yes | + +## Connection settings + +Both consolidated and storage-specific form must configure a connection. The following sections describe parameters that can be used +in the `connection` setting. + +### Amazon S3 + +The connection settings match those provided by [fog-aws](https://github.com/fog/fog-aws): + +| Setting | Description | Default | +|---------------------------------------------|------------------------------------|---------| +| `provider` | Always `AWS` for compatible hosts. | `AWS` | +| `aws_access_key_id` | AWS credentials, or compatible. | | +| `aws_secret_access_key` | AWS credentials, or compatible. | | +| `aws_signature_version` | AWS signature version to use. `2` or `4` are valid options. Digital Ocean Spaces and other providers may need `2`. | `4` | +| `enable_signature_v4_streaming` | Set to `true` to enable HTTP chunked transfers with [AWS v4 signatures](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html). Oracle Cloud S3 needs this to be `false`. | `true` | +| `region` | AWS region. | | +| `host` | DEPRECATED: Use `endpoint` instead. S3 compatible host for when not using AWS. For example, `localhost` or `storage.example.com`. HTTPS and port 443 is assumed. | `s3.amazonaws.com` | +| `endpoint` | Can be used when configuring an S3 compatible service such as [MinIO](https://min.io), by entering a URL such as `http://127.0.0.1:9000`. This takes precedence over `host`. Always use `endpoint` for consolidated form. | (optional) | +| `path_style` | Set to `true` to use `host/bucket_name/object` style paths instead of `bucket_name.host/object`. Set to `true` for using [MinIO](https://min.io). Leave as `false` for AWS S3. | `false`. | +| `use_iam_profile` | Set to `true` to use IAM profile instead of access keys. | `false` | +| `aws_credentials_refresh_threshold_seconds` | Sets the [automatic refresh threshold](https://github.com/fog/fog-aws#controlling-credential-refresh-time-with-iam-authentication) when using temporary credentials in IAM. | `15` | -See the section on [ETag mismatch errors](#etag-mismatch) for more details. +### Oracle Cloud S3 -#### Full example using Amazon S3 +Oracle Cloud S3 must be sure to use the following settings: + +| Setting | Value | +|---------------------------------|---------| +| `enable_signature_v4_streaming` | `false` | +| `path_style` | `true` | + +If `enable_signature_v4_streaming` is set to `true`, you may see the +following error in `production.log`: + +```plaintext +STREAMING-AWS4-HMAC-SHA256-PAYLOAD is not supported +``` + +### Google Cloud Storage (GCS) + +Here are the valid connection parameters for GCS: + +| Setting | Description | Example | +|------------------------------|-------------------|---------| +| `provider` | Provider name. | `Google` | +| `google_project` | GCP project name. | `gcp-project-12345` | +| `google_json_key_location` | JSON key path. | `/path/to/gcp-project-12345-abcde.json` | +| `google_json_key_string` | JSON key string. | `{ "type": "service_account", "project_id": "example-project-382839", ... }` | +| `google_application_default` | Set to `true` to use [Google Cloud Application Default Credentials](https://cloud.google.com/docs/authentication#adc) to locate service account credentials. | | + +GitLab reads the value of `google_json_key_location`, then `google_json_key_string`, and finally, `google_application_default`. +It uses the first of these settings that has a value. + +The service account must have permission to access the bucket. For more information, +see the [Cloud Storage authentication documentation](https://cloud.google.com/storage/docs/authentication). + +NOTE: +Bucket encryption with the [Cloud Key Management Service (KMS)](https://cloud.google.com/kms/docs) is not supported and results in [ETag mismatch errors](#etag-mismatch). + +#### GCS example + +For Omnibus installations, this is an example of the `connection` setting +in the consolidated form: + +```ruby +gitlab_rails['object_store']['connection'] = { + 'provider' => 'Google', + 'google_project' => '<GOOGLE PROJECT>', + 'google_json_key_location' => '<FILENAME>' +} +``` + +#### GCS example with ADC + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275979) in GitLab 13.6. + +Google Cloud Application Default Credentials (ADC) are typically +used with GitLab to use the default service account. This eliminates the +need to supply credentials for the instance. For example, in the consolidated form: + +```ruby +gitlab_rails['object_store']['connection'] = { + 'provider' => 'Google', + 'google_project' => '<GOOGLE PROJECT>', + 'google_application_default' => true +} +``` + +If you use ADC, be sure that: + +- The service account that you use has the +[`iam.serviceAccounts.signBlob` permission](https://cloud.google.com/iam/docs/reference/credentials/rest/v1/projects.serviceAccounts/signBlob). + Typically this is done by granting the `Service Account Token Creator` role to the service account. +- Your virtual machines have the [correct access scopes to access Google Cloud APIs](https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances#changeserviceaccountandscopes). If the machines do not have the right scope, the error logs may show: + + ```markdown + Google::Apis::ClientError (insufficientPermissions: Request had insufficient authentication scopes.) + ``` + +### Azure Blob storage + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25877) in GitLab 13.4. + +Although Azure uses the word `container` to denote a collection of +blobs, GitLab standardizes on the term `bucket`. Be sure to configure +Azure container names in the `bucket` settings. + +Azure Blob storage can only be used with the [consolidated form](#configure-a-single-storage-connection-for-all-object-types-consolidated-form) +because a single set of credentials are used to access multiple +containers. The [storage-specific form](#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form) +is not supported. For more details, see [how to transition to consolidated form](#transition-to-consolidated-form). + +The following are the valid connection parameters for Azure. For more information, see the +[Azure Blob Storage documentation](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction). + +| Setting | Description | Example | +|------------------------------|----------------|-----------| +| `provider` | Provider name. | `AzureRM` | +| `azure_storage_account_name` | Name of the Azure Blob Storage account used to access the storage. | `azuretest` | +| `azure_storage_access_key` | Storage account access key used to access the container. This is typically a secret, 512-bit encryption key encoded in base64. | `czV2OHkvQj9FKEgrTWJRZVRoV21ZcTN0Nnc5eiRDJkYpSkBOY1JmVWpYbjJy\nNHU3eCFBJUQqRy1LYVBkU2dWaw==\n` | +| `azure_storage_domain` | Domain name used to contact the Azure Blob Storage API (optional). Defaults to `blob.core.windows.net`. Set this if you are using Azure China, Azure Germany, Azure US Government, or some other custom Azure domain. | `blob.core.windows.net` | + +- For Omnibus installations, this is an example of the `connection` setting + in the consolidated form: + + ```ruby + gitlab_rails['object_store']['connection'] = { + 'provider' => 'AzureRM', + 'azure_storage_account_name' => '<AZURE STORAGE ACCOUNT NAME>', + 'azure_storage_access_key' => '<AZURE STORAGE ACCESS KEY>', + 'azure_storage_domain' => '<AZURE STORAGE DOMAIN>' + } + ``` + +- For source installations, Workhorse also needs to be configured with Azure + credentials. This isn't needed in Omnibus installs, because the Workhorse + settings are populated from the previous settings. + + 1. Edit `/home/git/gitlab-workhorse/config.toml` and add or amend the following lines: + + ```toml + [object_storage] + provider = "AzureRM" + + [object_storage.azurerm] + azure_storage_account_name = "<AZURE STORAGE ACCOUNT NAME>" + azure_storage_access_key = "<AZURE STORAGE ACCESS KEY>" + ``` + + If you are using a custom Azure storage domain, + `azure_storage_domain` does **not** have to be set in the Workhorse + configuration. This information is exchanged in an API call between + GitLab Rails and Workhorse. + +### Storj Gateway (SJ) + +NOTE: +The Storj Gateway [does not support](https://github.com/storj/gateway-st/blob/4b74c3b92c63b5de7409378b0d1ebd029db9337d/docs/s3-compatibility.md) multi-threaded copying (see `UploadPartCopy` in the table). +While an implementation [is planned](https://github.com/storj/roadmap/issues/40), you must [disable multi-threaded copying](#multi-threaded-copying) until completion. + +The [Storj Network](https://www.storj.io/) provides an S3-compatible API gateway. Use the following configuration example: + +```ruby +gitlab_rails['object_store']['connection'] = { + 'provider' => 'AWS', + 'endpoint' => 'https://gateway.storjshare.io', + 'path_style' => true, + 'region' => 'eu1', + 'aws_access_key_id' => 'ACCESS_KEY', + 'aws_secret_access_key' => 'SECRET_KEY', + 'aws_signature_version' => 2, + 'enable_signature_v4_streaming' => false +} +``` + +The signature version must be `2`. Using v4 results in a HTTP 411 Length Required error. +For more information, see [issue #4419](https://gitlab.com/gitlab-org/gitlab/-/issues/4419). + +## Full example using the consolidated form and Amazon S3 The following example uses AWS S3 to enable object storage for all supported services: @@ -379,309 +705,7 @@ The following example uses AWS S3 to enable object storage for all supported ser ::EndTabs -#### Common parameters - -In the consolidated configuration, the `object_store` section defines a -common set of parameters. Here we use the YAML from the source -installation because it's easier to see the inheritance: - -```yaml - object_store: - enabled: true - proxy_download: true - connection: - provider: AWS - aws_access_key_id: <AWS_ACCESS_KEY_ID> - aws_secret_access_key: <AWS_SECRET_ACCESS_KEY> - objects: - ... -``` - -The Omnibus configuration maps directly to this: - -```ruby -gitlab_rails['object_store']['enabled'] = true -gitlab_rails['object_store']['proxy_download'] = true -gitlab_rails['object_store']['connection'] = { - 'provider' => 'AWS', - 'aws_access_key_id' => '<AWS_ACCESS_KEY_ID', - 'aws_secret_access_key' => '<AWS_SECRET_ACCESS_KEY>' -} -``` - -| Setting | Description | -|-------------------|-----------------------------------| -| `enabled` | Enable or disable object storage. | -| `proxy_download` | Set to `true` to [enable proxying all files served](#proxy-download). Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data. | -| `connection` | Various [connection options](#connection-settings) described below. | -| `storage_options` | Options to use when saving new objects, such as [server side encryption](#server-side-encryption-headers). Introduced in GitLab 13.3. | -| `objects` | [Object-specific configuration](#object-specific-configuration). | - -### Connection settings - -Both consolidated configuration form and storage-specific configuration form must configure a connection. The following sections describe parameters that can be used -in the `connection` setting. - -#### Amazon S3 - -The connection settings match those provided by [fog-aws](https://github.com/fog/fog-aws): - -| Setting | Description | Default | -|---------------------------------------------|------------------------------------|---------| -| `provider` | Always `AWS` for compatible hosts. | `AWS` | -| `aws_access_key_id` | AWS credentials, or compatible. | | -| `aws_secret_access_key` | AWS credentials, or compatible. | | -| `aws_signature_version` | AWS signature version to use. `2` or `4` are valid options. Digital Ocean Spaces and other providers may need `2`. | `4` | -| `enable_signature_v4_streaming` | Set to `true` to enable HTTP chunked transfers with [AWS v4 signatures](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html). Oracle Cloud S3 needs this to be `false`. | `true` | -| `region` | AWS region. | | -| `host` | DEPRECATED: Use `endpoint` instead. S3 compatible host for when not using AWS. For example, `localhost` or `storage.example.com`. HTTPS and port 443 is assumed. | `s3.amazonaws.com` | -| `endpoint` | Can be used when configuring an S3 compatible service such as [MinIO](https://min.io), by entering a URL such as `http://127.0.0.1:9000`. This takes precedence over `host`. Always use `endpoint` for consolidated form. | (optional) | -| `path_style` | Set to `true` to use `host/bucket_name/object` style paths instead of `bucket_name.host/object`. Set to `true` for using [MinIO](https://min.io). Leave as `false` for AWS S3. | `false`. | -| `use_iam_profile` | Set to `true` to use IAM profile instead of access keys. | `false` | -| `aws_credentials_refresh_threshold_seconds` | Sets the [automatic refresh threshold](https://github.com/fog/fog-aws#controlling-credential-refresh-time-with-iam-authentication) when using temporary credentials in IAM. | `15` | - -#### Oracle Cloud S3 - -Oracle Cloud S3 must be sure to use the following settings: - -| Setting | Value | -|---------------------------------|---------| -| `enable_signature_v4_streaming` | `false` | -| `path_style` | `true` | - -If `enable_signature_v4_streaming` is set to `true`, you may see the -following error in `production.log`: - -```plaintext -STREAMING-AWS4-HMAC-SHA256-PAYLOAD is not supported -``` - -#### Google Cloud Storage (GCS) - -Here are the valid connection parameters for GCS: - -| Setting | Description | Example | -|------------------------------|-------------------|---------| -| `provider` | Provider name. | `Google` | -| `google_project` | GCP project name. | `gcp-project-12345` | -| `google_json_key_location` | JSON key path. | `/path/to/gcp-project-12345-abcde.json` | -| `google_json_key_string` | JSON key string. | `{ "type": "service_account", "project_id": "example-project-382839", ... }` | -| `google_application_default` | Set to `true` to use [Google Cloud Application Default Credentials](https://cloud.google.com/docs/authentication#adc) to locate service account credentials. | | - -GitLab reads the value of `google_json_key_location`, then `google_json_key_string`, and finally, `google_application_default`. -It uses the first of these settings that has a value. - -The service account must have permission to access the bucket. For more information, -see the [Cloud Storage authentication documentation](https://cloud.google.com/storage/docs/authentication). - -NOTE: -Bucket encryption with the [Cloud Key Management Service (KMS)](https://cloud.google.com/kms/docs) is not supported and results in [ETag mismatch errors](#etag-mismatch). - -##### GCS example - -For Omnibus installations, this is an example of the `connection` setting -in the consolidated form: - -```ruby -gitlab_rails['object_store']['connection'] = { - 'provider' => 'Google', - 'google_project' => '<GOOGLE PROJECT>', - 'google_json_key_location' => '<FILENAME>' -} -``` - -##### GCS example with ADC - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275979) in GitLab 13.6. - -Google Cloud Application Default Credentials (ADC) are typically -used with GitLab to use the default service account. This eliminates the -need to supply credentials for the instance. For example, in the consolidated form: - -```ruby -gitlab_rails['object_store']['connection'] = { - 'provider' => 'Google', - 'google_project' => '<GOOGLE PROJECT>', - 'google_application_default' => true -} -``` - -If you use ADC, be sure that: - -- The service account that you use has the -[`iam.serviceAccounts.signBlob` permission](https://cloud.google.com/iam/docs/reference/credentials/rest/v1/projects.serviceAccounts/signBlob). - Typically this is done by granting the `Service Account Token Creator` role to the service account. -- Your virtual machines have the [correct access scopes to access Google Cloud APIs](https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances#changeserviceaccountandscopes). If the machines do not have the right scope, the error logs may show: - - ```markdown - Google::Apis::ClientError (insufficientPermissions: Request had insufficient authentication scopes.) - ``` - -#### Azure Blob storage - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25877) in GitLab 13.4. - -Although Azure uses the word `container` to denote a collection of -blobs, GitLab standardizes on the term `bucket`. Be sure to configure -Azure container names in the `bucket` settings. - -Azure Blob storage can only be used with the [consolidated form](#consolidated-object-storage-configuration) -because a single set of credentials are used to access multiple -containers. The [storage-specific form](#storage-specific-configuration) -is not supported. For more details, see [how to transition to consolidated form](#transition-to-consolidated-form). - -The following are the valid connection parameters for Azure. For more information, see the -[Azure Blob Storage documentation](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction). - -| Setting | Description | Example | -|------------------------------|----------------|-----------| -| `provider` | Provider name. | `AzureRM` | -| `azure_storage_account_name` | Name of the Azure Blob Storage account used to access the storage. | `azuretest` | -| `azure_storage_access_key` | Storage account access key used to access the container. This is typically a secret, 512-bit encryption key encoded in base64. | `czV2OHkvQj9FKEgrTWJRZVRoV21ZcTN0Nnc5eiRDJkYpSkBOY1JmVWpYbjJy\nNHU3eCFBJUQqRy1LYVBkU2dWaw==\n` | -| `azure_storage_domain` | Domain name used to contact the Azure Blob Storage API (optional). Defaults to `blob.core.windows.net`. Set this if you are using Azure China, Azure Germany, Azure US Government, or some other custom Azure domain. | `blob.core.windows.net` | - -- For Omnibus installations, this is an example of the `connection` setting - in the consolidated form: - - ```ruby - gitlab_rails['object_store']['connection'] = { - 'provider' => 'AzureRM', - 'azure_storage_account_name' => '<AZURE STORAGE ACCOUNT NAME>', - 'azure_storage_access_key' => '<AZURE STORAGE ACCESS KEY>', - 'azure_storage_domain' => '<AZURE STORAGE DOMAIN>' - } - ``` - -- For source installations, Workhorse also needs to be configured with Azure - credentials. This isn't needed in Omnibus installs, because the Workhorse - settings are populated from the previous settings. - - 1. Edit `/home/git/gitlab-workhorse/config.toml` and add or amend the following lines: - - ```toml - [object_storage] - provider = "AzureRM" - - [object_storage.azurerm] - azure_storage_account_name = "<AZURE STORAGE ACCOUNT NAME>" - azure_storage_access_key = "<AZURE STORAGE ACCESS KEY>" - ``` - - If you are using a custom Azure storage domain, - `azure_storage_domain` does **not** have to be set in the Workhorse - configuration. This information is exchanged in an API call between - GitLab Rails and Workhorse. - -#### Storj Gateway (SJ) - -NOTE: -The Storj Gateway [does not support](https://github.com/storj/gateway-st/blob/4b74c3b92c63b5de7409378b0d1ebd029db9337d/docs/s3-compatibility.md) multi-threaded copying (see `UploadPartCopy` in the table). -While an implementation [is planned](https://github.com/storj/roadmap/issues/40), you must [disable multi-threaded copying](#multi-threaded-copying) until completion. - -The [Storj Network](https://www.storj.io/) provides an S3-compatible API gateway. Use the following configuration example: - -```ruby -gitlab_rails['object_store']['connection'] = { - 'provider' => 'AWS', - 'endpoint' => 'https://gateway.storjshare.io', - 'path_style' => true, - 'region' => 'eu1', - 'aws_access_key_id' => 'ACCESS_KEY', - 'aws_secret_access_key' => 'SECRET_KEY', - 'aws_signature_version' => 2, - 'enable_signature_v4_streaming' => false -} -``` - -The signature version must be `2`. Using v4 results in a HTTP 411 Length Required error. -For more information, see [issue #4419](https://gitlab.com/gitlab-org/gitlab/-/issues/4419). - -### Object-specific configuration - -The following YAML shows how the `object_store` section defines -object-specific configuration block and how the `enabled` and -`proxy_download` flags can be overridden. The `bucket` is the only -required parameter within each type: - -```yaml - object_store: - connection: - ... - objects: - artifacts: - bucket: artifacts - proxy_download: false - external_diffs: - bucket: external-diffs - lfs: - bucket: lfs-objects - uploads: - bucket: uploads - packages: - bucket: packages - dependency_proxy: - enabled: false - bucket: dependency_proxy - terraform_state: - bucket: terraform - pages: - bucket: pages -``` - -This maps to this Omnibus GitLab configuration: - -```ruby -gitlab_rails['object_store']['objects']['artifacts']['bucket'] = 'artifacts' -gitlab_rails['object_store']['objects']['artifacts']['proxy_download'] = false -gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = 'external-diffs' -gitlab_rails['object_store']['objects']['lfs']['bucket'] = 'lfs-objects' -gitlab_rails['object_store']['objects']['uploads']['bucket'] = 'uploads' -gitlab_rails['object_store']['objects']['packages']['bucket'] = 'packages' -gitlab_rails['object_store']['objects']['dependency_proxy']['enabled'] = false -gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = 'dependency-proxy' -gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = 'terraform-state' -gitlab_rails['object_store']['objects']['pages']['bucket'] = 'pages' -``` - -This is the list of valid `objects` that can be used: - -| Type | Description | -|--------------------|----------------------------------------------------------------------------| -| `artifacts` | [CI artifacts](job_artifacts.md) | -| `external_diffs` | [Merge request diffs](merge_request_diffs.md) | -| `uploads` | [User uploads](uploads.md) | -| `lfs` | [Git Large File Storage objects](lfs/index.md) | -| `packages` | [Project packages (for example, PyPI, Maven, or NuGet)](packages/index.md) | -| `dependency_proxy` | [Dependency Proxy](packages/dependency_proxy.md) | -| `terraform_state` | [Terraform state files](terraform_state.md) | -| `pages` | [Pages](pages/index.md) | - -Within each object type, three parameters can be defined: - -| Setting | Required? | Description | -|------------------|------------------------|-------------------------------------| -| `bucket` | **{check-circle}** Yes | Bucket name for the object storage. | -| `enabled` | **{dotted-circle}** No | Overrides the common parameter. | -| `proxy_download` | **{dotted-circle}** No | Overrides the common parameter. | - -#### Disable object storage for specific features - -As seen above, object storage can be disabled for specific types by -setting the `enabled` flag to `false`. For example, to disable object -storage for CI artifacts: - -```ruby -gitlab_rails['object_store']['objects']['artifacts']['enabled'] = false -``` - -A bucket is not needed if the feature is disabled entirely. For example, -no bucket is needed if CI artifacts are disabled with this setting: - -```ruby -gitlab_rails['artifacts_enabled'] = false -``` - -### Migrate to object storage +## Migrate to object storage To migrate existing local data to object storage see the following guides: @@ -694,7 +718,7 @@ To migrate existing local data to object storage see the following guides: - [Terraform state files](terraform_state.md#migrate-to-object-storage) - [Pages content](pages/index.md#migrate-pages-deployments-to-object-storage) -### Transition to consolidated form +## Transition to consolidated form Prior to GitLab 13.2: @@ -725,48 +749,11 @@ additional complexity and unnecessary redundancy. Since both GitLab Rails and Workhorse components need access to object storage, the consolidated form avoids excessive duplication of credentials. -The consolidated object storage configuration is used _only_ if all lines from +The consolidated form is used _only_ if all lines from the original form is omitted. To move to the consolidated form, remove the original configuration (for example, `artifacts_object_store_enabled`, or `uploads_object_store_connection`) -### Storage-specific configuration - -For configuring object storage in GitLab 13.1 and earlier, or for storage types not -supported by consolidated configuration form, refer to the following guides: - -| Object storage type | Supported by consolidated configuration? | -|---------------------|------------------------------------------| -| [Backups](../raketasks/backup_gitlab.md#upload-backups-to-a-remote-cloud-storage) | **{dotted-circle}** No | -| [Job artifacts](job_artifacts.md#using-object-storage) including archived job logs | **{check-circle}** Yes | -| [LFS objects](lfs/index.md#storing-lfs-objects-in-remote-object-storage) | **{check-circle}** Yes | -| [Uploads](uploads.md#using-object-storage) | **{check-circle}** Yes | -| [Container Registry](packages/container_registry.md#use-object-storage) (optional feature) | **{dotted-circle}** No | -| [Merge request diffs](merge_request_diffs.md#using-object-storage) | **{check-circle}** Yes | -| [Mattermost](https://docs.mattermost.com/configure/file-storage-configuration-settings.html)| **{dotted-circle}** No | -| [Packages](packages/index.md#use-object-storage) (optional feature) | **{check-circle}** Yes | -| [Dependency Proxy](packages/dependency_proxy.md#using-object-storage) (optional feature) | **{check-circle}** Yes | -| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | **{dotted-circle}** No | -| [Terraform state files](terraform_state.md#using-object-storage) | **{check-circle}** Yes | -| [Pages content](pages/index.md#using-object-storage) | **{check-circle}** Yes | - -WARNING: -The use of [encrypted S3 buckets](#encrypted-s3-buckets) with non-consolidated configuration is not supported. -You may start getting [ETag mismatch errors](#etag-mismatch) if you use it. - -### Other alternatives to file system storage - -If you're working to [scale out](reference_architectures/index.md) your GitLab implementation, -or add fault tolerance and redundancy, you may be -looking at removing dependencies on block or network file systems. -See the following additional guides: - -1. Make sure the [`git` user home directory](https://docs.gitlab.com/omnibus/settings/configuration.html#moving-the-home-directory-for-a-user) is on local disk. -1. Configure [database lookup of SSH keys](operations/fast_ssh_key_lookup.md) - to eliminate the need for a shared `authorized_keys` file. -1. [Prevent local disk usage for job logs](job_logs.md#prevent-local-disk-usage). -1. [Disable Pages local storage](pages/index.md#disable-pages-local-storage). - ## Use Amazon instance profiles Instead of supplying AWS access and secret keys in object storage @@ -784,10 +771,10 @@ address must be added to the `no_proxy` list. ### Encrypted S3 buckets > - [Introduced](https://gitlab.com/gitlab-org/gitlab-workhorse/-/merge_requests/466) in GitLab 13.1 for instance profiles only and [S3 default encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34460) in GitLab 13.2 for static credentials when [consolidated object storage configuration](#consolidated-object-storage-configuration) and [S3 default encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html) are used. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34460) in GitLab 13.2 for static credentials when the [consolidated form](#configure-a-single-storage-connection-for-all-object-types-consolidated-form) and [S3 default encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html) is used. When configured either with an instance profile or with the consolidated -object configuration, GitLab Workhorse properly uploads files to S3 +form, GitLab Workhorse properly uploads files to S3 buckets that have [SSE-S3 or SSE-KMS encryption enabled by default](https://docs.aws.amazon.com/kms/latest/developerguide/services-s3.html). Customer master keys (CMKs) and SSE-C encryption are [not supported since this requires sending the encryption keys in every request](https://gitlab.com/gitlab-org/gitlab/-/issues/226006). @@ -812,7 +799,7 @@ the Workhorse S3 client is enabled. One of the following two conditions must be fulfilled: - `use_iam_profile` is `true` in the connection settings. -- Consolidated object storage settings are in use. +- Consolidated form is in use. [ETag mismatch errors](#etag-mismatch) occur if server side encryption headers are used without enabling the Workhorse S3 client. @@ -889,6 +876,19 @@ After you have done at least one successful Rclone copy from the old location to 1. Perform a final `rclone sync` run, knowing that your users cannot add new objects so you do not leave any behind in the old bucket. 1. Update the object storage configuration of your GitLab server to use the new provider for `uploads`. +## Alternatives to file system storage + +If you're working to [scale out](reference_architectures/index.md) your GitLab implementation, +or add fault tolerance and redundancy, you may be +looking at removing dependencies on block or network file systems. +See the following additional guides: + +1. Make sure the [`git` user home directory](https://docs.gitlab.com/omnibus/settings/configuration.html#moving-the-home-directory-for-a-user) is on local disk. +1. Configure [database lookup of SSH keys](operations/fast_ssh_key_lookup.md) + to eliminate the need for a shared `authorized_keys` file. +1. [Prevent local disk usage for job logs](job_logs.md#prevent-local-disk-usage). +1. [Disable Pages local storage](pages/index.md#disable-pages-local-storage). + ## Troubleshooting ### Objects are not included in GitLab backups @@ -992,14 +992,14 @@ If you are seeing this ETag mismatch error with Amazon Web Services S3, it's likely this is due to [encryption settings on your bucket](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html). To fix this issue, you have two options: -- [Use the consolidated object configuration](#consolidated-object-storage-configuration). +- [Use the consolidated form](#configure-a-single-storage-connection-for-all-object-types-consolidated-form). - [Use Amazon instance profiles](#use-amazon-instance-profiles). The first option is recommended for MinIO. Otherwise, the [workaround for MinIO](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/1564#note_244497658) is to use the `--compat` parameter on the server. -Without consolidated object store configuration or instance profiles enabled, +Without the consolidated form or instance profiles enabled, GitLab Workhorse uploads files to S3 using pre-signed URLs that do not have a `Content-MD5` HTTP header computed for them. To ensure data is not corrupted, Workhorse checks that the MD5 hash of the data sent @@ -1007,9 +1007,14 @@ equals the ETag header returned from the S3 server. When encryption is enabled, this is not the case, which causes Workhorse to report an `ETag mismatch` error during an upload. -With the consolidated object configuration and instance profile, Workhorse has -S3 credentials so that it can compute the `Content-MD5` header. This -eliminates the need to compare ETag headers returned from the S3 server. +When the consolidated form is: + +- Used with an S3-compatible object storage or an istance profile, Workhorse + uses its internal S3 client which has S3 credentials so that it can compute + the `Content-MD5` header. This eliminates the need to compare ETag headers + returned from the S3 server. +- Not used with an S3-compatible object storage, Workhorse falls back to using + pre-signed URLs. Encrypting buckets with the GCS [Cloud Key Management Service (KMS)](https://cloud.google.com/kms/docs) is not supported and results in ETag mismatch errors. diff --git a/doc/administration/operations/gitlab_sshd.md b/doc/administration/operations/gitlab_sshd.md index 1153321267d..249d6232616 100644 --- a/doc/administration/operations/gitlab_sshd.md +++ b/doc/administration/operations/gitlab_sshd.md @@ -6,13 +6,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w # `gitlab-sshd` **(FREE SELF)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299109) in GitLab 14.5 as an **Alpha** release for self-managed customers. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299109) in GitLab 14.5 as an Experiment for self-managed customers. > - Ready for production use with [Cloud Native GitLab in GitLab 15.1](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/2540) and [Omnibus GitLab in GitLab 15.9](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5937). `gitlab-sshd` is [a standalone SSH server](https://gitlab.com/gitlab-org/gitlab-shell/-/tree/main/internal/sshd) written in Go. It is provided as a part of the `gitlab-shell` package. It has a lower memory use as a OpenSSH alternative, and supports -[group access restriction by IP address](../../user/group/index.md) for applications +[group access restriction by IP address](../../user/group/access_and_permissions.md#restrict-group-access-by-ip-address) for applications running behind the proxy. `gitlab-sshd` is a lightweight alternative to OpenSSH for providing @@ -27,8 +27,9 @@ If you are considering switching from OpenSSH to `gitlab-sshd`, consider these c - `gitlab-sshd` supports the PROXY protocol. It can run behind proxy servers that rely on it, such as HAProxy. The PROXY protocol is not enabled by default, but [it can be enabled](#proxy-protocol-support). -- `gitlab-sshd` **does not** support SSH certificates. For more details, read - [issue #495](https://gitlab.com/gitlab-org/gitlab-shell/-/issues/495). +- `gitlab-sshd` **does not** support SSH certificates. For more details, see the + [confidential issue](../../user/project/issues/confidential_issues.md) + `https://gitlab.com/gitlab-org/gitlab-shell/-/issues/495`. ## Enable `gitlab-sshd` diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index b02aab738ea..867ef3236ee 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -4,30 +4,21 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Performing operations in GitLab **(FREE SELF)** +# Maintain your GitLab installation **(FREE SELF)** -Keep your GitLab instance up and running smoothly. +Keep your GitLab instance up and running. -- [Upgrading GitLab](../../update/index.md). -- [Rake tasks](../../raketasks/index.md): Tasks for common administration and operational processes such as - [cleaning up unneeded items from GitLab instance](../../raketasks/cleanup.md), integrity checks, - and more. -- [Moving repositories](moving_repositories.md): Moving all repositories managed - by GitLab to another file system or another server. -- [Sidekiq MemoryKiller](../sidekiq/sidekiq_memory_killer.md): Configure Sidekiq MemoryKiller - to restart Sidekiq. -- [Multiple Sidekiq processes](../sidekiq/extra_sidekiq_processes.md): Configure multiple Sidekiq processes to ensure certain queues always have dedicated workers, no matter the number of jobs that must be processed. **(FREE SELF)** -- [Puma](puma.md): Understand Puma and puma-worker-killer. -- [`gitlab-sshd`](gitlab_sshd.md): Use GitLab SSH daemon instead of OpenSSH. -- Speed up SSH operations by - [Authorizing SSH users via a fast, indexed lookup to the GitLab database](fast_ssh_key_lookup.md), and/or - by [doing away with user SSH keys stored on GitLab entirely in favor of SSH certificates](ssh_certificates.md). -- [File System Performance Benchmarking](filesystem_benchmarking.md): File system - performance can have a big impact on GitLab performance, especially for actions - that read or write Git repositories. This information helps benchmark - file system performance against known good and bad real-world systems. -- [The Rails Console](rails_console.md): Provides a way to interact with your GitLab instance from the command line. - Used for troubleshooting a problem or retrieving some data that can only be done through direct access to GitLab. -- [ChatOps Scripts](https://gitlab.com/gitlab-com/chatops): The GitLab.com Infrastructure team uses this repository to house - common ChatOps scripts they use to troubleshoot and maintain the production instance of GitLab.com. - These scripts can be used by administrators of GitLab instances of all sizes. +- [Housekeeping](../../administration/housekeeping.md) +- [Activate GitLab EE with license](../../user/admin_area/license_file.md) +- [Fast SSH key lookup](../../administration/operations/fast_ssh_key_lookup.md) +- [File system benchmarking](../../administration/operations/filesystem_benchmarking.md) +- [`gitlab-sshd`](../../administration/operations/gitlab_sshd.md) +- [Rails console](../../administration/operations/rails_console.md) +- [Use SSH certificates](../../administration/operations/ssh_certificates.md) +- [Enable encrypted configuration](../../administration/encrypted_configuration.md) +- [Rake tasks](../../raketasks/index.md) +- [Backup and restore](../../raketasks/backup_restore.md) +- [Inactive project deletion](../../administration/inactive_project_deletion.md) +- [Move repositories](../../administration/operations/moving_repositories.md) +- [Read-only state](../../administration/read_only_gitlab.md) +- [Restart GitLab](../../administration/restart_gitlab.md) diff --git a/doc/administration/operations/puma.md b/doc/administration/operations/puma.md index f2f9f1cdcda..5ff9208ecb9 100644 --- a/doc/administration/operations/puma.md +++ b/doc/administration/operations/puma.md @@ -99,7 +99,7 @@ To change the worker timeout to 600 seconds: ## Disable Puma clustered mode in memory-constrained environments WARNING: -This is an experimental [Alpha feature](../../policy/alpha-beta-support.md#alpha-features) and subject to change without notice. The feature +This feature is an [Experiment](../../policy/alpha-beta-support.md#experiment) and subject to change without notice. The feature is not ready for production use. If you want to use this feature, you should test outside of production first. See the [known issues](#puma-single-mode-known-issues) for additional details. diff --git a/doc/administration/package_information/defaults.md b/doc/administration/package_information/defaults.md index 3b52b6bca82..932342a6a92 100644 --- a/doc/administration/package_information/defaults.md +++ b/doc/administration/package_information/defaults.md @@ -30,7 +30,7 @@ by default: | PgBouncer exporter | No | Port | X | 9188 | | GitLab Exporter | Yes | Port | X | 9168 | | Sidekiq exporter | Yes | Port | X | 8082 | -| Sidekiq health check | No | Port | X | 8092[^Sidekiq-health] | +| Sidekiq health check | Yes | Port | X | 8092[^Sidekiq-health] | | Web exporter | No | Port | X | 8083 | | Geo PostgreSQL | No | Socket | Port (5431) | X | | Redis Sentinel | No | Port | X | 26379 | diff --git a/doc/administration/package_information/supported_os.md b/doc/administration/package_information/supported_os.md index f92d57c0035..ea87dfb6e28 100644 --- a/doc/administration/package_information/supported_os.md +++ b/doc/administration/package_information/supported_os.md @@ -21,7 +21,6 @@ The following lists the currently supported OSs and their possible EOL dates. | CentOS 7 | GitLab CE / GitLab EE 7.10.0 | x86_64 | [CentOS Install Documentation](https://about.gitlab.com/install/#centos-7) | June 2024 | <https://wiki.centos.org/About/Product> | | Debian 10 | GitLab CE / GitLab EE 12.2.0 | amd64, arm64 | [Debian Install Documentation](https://about.gitlab.com/install/#debian) | 2024 | <https://wiki.debian.org/LTS> | | Debian 11 | GitLab CE / GitLab EE 14.6.0 | amd64, arm64 | [Debian Install Documentation](https://about.gitlab.com/install/#debian) | 2026 | <https://wiki.debian.org/LTS> | -| OpenSUSE 15.3 | GitLab CE / GitLab EE 14.5.0 | x86_64, aarch64 | [OpenSUSE Install Documentation](https://about.gitlab.com/install/#opensuse-leap) | Dec 2022 | <https://en.opensuse.org/Lifetime> | | OpenSUSE 15.4 | GitLab CE / GitLab EE 15.7.0 | x86_64, aarch64 | [OpenSUSE Install Documentation](https://about.gitlab.com/install/#opensuse-leap) | Nov 2023 | <https://en.opensuse.org/Lifetime> | | RHEL 8 | GitLab CE / GitLab EE 12.8.1 | x86_64, arm64 | [Use CentOS Install Documentation](https://about.gitlab.com/install/#centos-7) | May 2029 | [RHEL Details](https://access.redhat.com/support/policy/updates/errata/#Life_Cycle_Dates) | | SLES 12 | GitLab EE 9.0.0 | x86_64 | [Use OpenSUSE Install Documentation](https://about.gitlab.com/install/#opensuse-leap) | Oct 2027 | <https://www.suse.com/lifecycle/> | @@ -103,6 +102,7 @@ release for them can be found below: | Ubuntu 16.04 | [April 2021](https://ubuntu.com/info/release-end-of-life) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=gitlab-ce_13.12&dist=ubuntu%2Fxenial) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=gitlab-ee_13.12&dist=ubuntu%2Fxenial) 13.12 | | OpenSUSE 15.2 | [December 2021](https://en.opensuse.org/Lifetime#Discontinued_distributions) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=gitlab-ce-14.7&dist=opensuse%2F15.2) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=gitlab-ee-14.7&dist=opensuse%2F15.2) 14.7 | | Debian 9 "Stretch" | [June 2022](https://lists.debian.org/debian-lts-announce/2022/07/msg00002.html) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=gitlab-ce_15.2&dist=debian%2Fstretch) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=gitlab-ee_15.2&dist=debian%2Fstretch) 15.2 | +| OpenSUSE 15.3 | [December 2022](https://en.opensuse.org/Lifetime#Discontinued_distributions) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=gitlab-ce-15.10&dist=opensuse%2F15.3) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=gitlab-ee-15.10&dist=opensuse%2F15.3) 15.10 | NOTE: An exception to this deprecation policy is when we are unable to provide diff --git a/doc/administration/packages/container_registry.md b/doc/administration/packages/container_registry.md index 619ec2490a7..05d2f307f02 100644 --- a/doc/administration/packages/container_registry.md +++ b/doc/administration/packages/container_registry.md @@ -932,7 +932,7 @@ information, see the [Sidekiq configuration](../sidekiq/index.md) page. To reduce the amount of [Container Registry disk space used by a given project](#registry-disk-space-usage-by-project), -administrators can clean up image tags +administrators can setup cleanup policies and [run garbage collection](#container-registry-garbage-collection). ### Registry Disk Space Usage by Project diff --git a/doc/administration/packages/dependency_proxy.md b/doc/administration/packages/dependency_proxy.md index 78efd75bbe3..a29d6d93051 100644 --- a/doc/administration/packages/dependency_proxy.md +++ b/doc/administration/packages/dependency_proxy.md @@ -122,7 +122,7 @@ To change the local storage path: Instead of relying on the local storage, you can use an object storage to store the blobs of the Dependency Proxy. In GitLab 13.2 and later, you should use the -[consolidated object storage settings](../object_storage.md#consolidated-object-storage-configuration). +[consolidated object storage settings](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). This section describes the earlier configuration format. [Migration steps still apply](#migrate-local-dependency-proxy-blobs-and-manifests-to-object-storage). [Read more about using object storage with GitLab](../object_storage.md). diff --git a/doc/administration/packages/index.md b/doc/administration/packages/index.md index 6e53d77109f..f0f238aa5ba 100644 --- a/doc/administration/packages/index.md +++ b/doc/administration/packages/index.md @@ -124,7 +124,7 @@ Instead of relying on the local storage, you can use an object storage to store packages. For more information, see how to use the -[consolidated object storage settings](../object_storage.md#consolidated-object-storage-configuration). +[consolidated object storage settings](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). ### Migrate local packages to object storage diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index efeb19a8d5e..865d6e0bed7 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -1,8 +1,7 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -description: 'Learn how to administer GitLab Pages.' --- # GitLab Pages administration **(FREE SELF)** @@ -247,20 +246,20 @@ control over how the Pages daemon runs and serves content in your environment. | `enable` | Enable or disable GitLab Pages on the current system. | | `external_http` | Configure Pages to bind to one or more secondary IP addresses, serving HTTP requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_http`. | | `external_https` | Configure Pages to bind to one or more secondary IP addresses, serving HTTPS requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_https`. | -| `server_shutdown_timeout` | GitLab Pages server shutdown timeout in seconds (default: 30 s). | -| `gitlab_client_http_timeout` | GitLab API HTTP client connection timeout in seconds (default: 10 s). | -| `gitlab_client_jwt_expiry` | JWT Token expiry time in seconds (default: 30 s). | -| `gitlab_cache_expiry` | The maximum time a domain's configuration is stored in the cache (default: 600 s). | -| `gitlab_cache_refresh` | The interval at which a domain's configuration is set to be due to refresh (default: 60 s). | -| `gitlab_cache_cleanup` | The interval at which expired items are removed from the cache (default: 60 s). | -| `gitlab_retrieval_timeout` | The maximum time to wait for a response from the GitLab API per request (default: 30 s). | -| `gitlab_retrieval_interval` | The interval to wait before retrying to resolve a domain's configuration via the GitLab API (default: 1 s). | +| `server_shutdown_timeout` | GitLab Pages server shutdown timeout in seconds (default: `30s`). | +| `gitlab_client_http_timeout` | GitLab API HTTP client connection timeout in seconds (default: `10s`). | +| `gitlab_client_jwt_expiry` | JWT Token expiry time in seconds (default: `30s`). | +| `gitlab_cache_expiry` | The maximum time a domain's configuration is stored in the cache (default: `600s`). | +| `gitlab_cache_refresh` | The interval at which a domain's configuration is set to be due to refresh (default: `60s`). | +| `gitlab_cache_cleanup` | The interval at which expired items are removed from the cache (default: `60s`). | +| `gitlab_retrieval_timeout` | The maximum time to wait for a response from the GitLab API per request (default: `30s`). | +| `gitlab_retrieval_interval` | The interval to wait before retrying to resolve a domain's configuration via the GitLab API (default: `1s`). | | `gitlab_retrieval_retries` | The maximum number of times to retry to resolve a domain's configuration via the API (default: 3). | | `domain_config_source` | This parameter was removed in 14.0, on earlier versions it can be used to enable and test API domain configuration source | | `gitlab_id` | The OAuth application public ID. Leave blank to automatically fill when Pages authenticates with GitLab. | | `gitlab_secret` | The OAuth application secret. Leave blank to automatically fill when Pages authenticates with GitLab. | | `auth_scope` | The OAuth application scope to use for authentication. Must match GitLab Pages OAuth application settings. Leave blank to use `api` scope by default. | -| `auth_cookie_session_timeout` | Authentication cookie session timeout in seconds (default: 600 s). A value of `0` means the cookie is deleted after the browser session ends. | +| `auth_cookie_session_timeout` | Authentication cookie session timeout in seconds (default: `10m`). A value of `0` means the cookie is deleted after the browser session ends. | | `gitlab_server` | Server to use for authentication when access control is enabled; defaults to GitLab `external_url`. | | `headers` | Specify any additional http headers that should be sent to the client with each response. Multiple headers can be given as an array, header and value as one string, for example `['my-header: myvalue', 'my-other-header: my-other-value']` | | `enable_disk` | Allows the GitLab Pages daemon to serve content from disk. Shall be disabled if shared disk storage isn't available. | @@ -514,7 +513,7 @@ internet connectivity is gated by a proxy. To use a proxy for GitLab Pages: ### Using a custom Certificate Authority (CA) When using certificates issued by a custom CA, [Access Control](../../user/project/pages/pages_access_control.md) and -the [online view of HTML job artifacts](../../ci/pipelines/job_artifacts.md#download-job-artifacts) +the [online view of HTML job artifacts](../../ci/jobs/job_artifacts.md#download-job-artifacts) fails to work if the custom CA is not recognized. This usually results in this error: @@ -540,22 +539,22 @@ archive. You can modify the cache behavior by changing the following configurati | Setting | Description | | ------- | ----------- | -| `zip_cache_expiration` | The cache expiration interval of ZIP archives. Must be greater than zero to avoid serving stale content. Default is 60 s. | -| `zip_cache_cleanup` | The interval at which archives are cleaned from memory if they have already expired. Default is 30 s. | -| `zip_cache_refresh` | The time interval in which an archive is extended in memory if accessed before `zip_cache_expiration`. This works together with `zip_cache_expiration` to determine if an archive is extended in memory. See the [example below](#zip-cache-refresh-example) for important details. Default is 30 s. | +| `zip_cache_expiration` | The cache expiration interval of ZIP archives. Must be greater than zero to avoid serving stale content. Default is `60s`. | +| `zip_cache_cleanup` | The interval at which archives are cleaned from memory if they have already expired. Default is `30s`. | +| `zip_cache_refresh` | The time interval in which an archive is extended in memory if accessed before `zip_cache_expiration`. This works together with `zip_cache_expiration` to determine if an archive is extended in memory. See the [example below](#zip-cache-refresh-example) for important details. Default is `30s`. | | `zip_open_timeout` | The maximum time allowed to open a ZIP archive. Increase this time for big archives or slow network connections, as doing so may affect the latency of serving Pages. Default is 30 s. | -| `zip_http_client_timeout` | The maximum time for the ZIP HTTP client. Default is 30 m. | +| `zip_http_client_timeout` | The maximum time for the ZIP HTTP client. Default is `30m`. | #### ZIP cache refresh example Archives are refreshed in the cache (extending the time they are held in memory) if they're accessed before `zip_cache_expiration`, and the time left before expiring is less than or equal to -`zip_cache_refresh`. For example, if `archive.zip` is accessed at time 0 s, it expires in 60 s (the -default for `zip_cache_expiration`). In the example below, if the archive is opened again after 15 s -it is **not** refreshed because the time left for expiry (45 s) is greater than `zip_cache_refresh` -(default 30 s). However, if the archive is accessed again after 45 s (from the first time it was +`zip_cache_refresh`. For example, if `archive.zip` is accessed at time `0s`, it expires in `60s` (the +default for `zip_cache_expiration`). In the example below, if the archive is opened again after `15s` +it is **not** refreshed because the time left for expiry (`45s`) is greater than `zip_cache_refresh` +(default `30s`). However, if the archive is accessed again after `45s` (from the first time it was opened) it's refreshed. This extends the time the archive remains in memory from -`45s + zip_cache_expiration (60s)`, for a total of 105 s. +`45s + zip_cache_expiration (60s)`, for a total of `105s`. After an archive reaches `zip_cache_expiration`, it's marked as expired and removed on the next `zip_cache_cleanup` interval. @@ -710,6 +709,15 @@ To set the maximum number of GitLab Pages custom domains for a project: 1. Enter a value for **Maximum number of custom domains per project**. Use `0` for unlimited domains. 1. Select **Save changes**. +## Set maximum number of files per GitLab Pages website + +The total number of file entries (including directories and symlinks) is limited to `200,000` per GitLab Pages website. + +You can update the limit in your self-managed instance using the +[GitLab Rails console](../operations/rails_console.md#starting-a-rails-console-session). + +For more information, see [GitLab application limits](../instance_limits.md#number-of-files-per-gitlab-pages-website). + ## Running GitLab Pages on a separate server You can run the GitLab Pages daemon on a separate server to decrease the load on @@ -818,7 +826,7 @@ From [GitLab 13.3 to GitLab 13.12](#domain-source-configuration-before-140) GitL Starting from [GitLab 14.0](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5993) GitLab Pages uses API by default and fails to start if it can't connect to it. -For common issues, see the [troubleshooting section](#failed-to-connect-to-the-internal-gitlab-api). +For common issues, see [troubleshooting](troubleshooting.md#failed-to-connect-to-the-internal-gitlab-api). For more details see this [blog post](https://about.gitlab.com/blog/2020/08/03/how-gitlab-pages-uses-the-gitlab-api-to-serve-content/). @@ -895,7 +903,7 @@ configuration is tried to be resolved automatically before reporting an error. > [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5577) in GitLab 13.6. -[Read more about using object storage with GitLab](../object_storage.md). +For more information, see [object storage](../object_storage.md). ### Object storage settings @@ -917,7 +925,7 @@ If you want to stop using and disconnect the NFS server, you need to #### S3-compatible connection settings In GitLab 13.2 and later, you should use the -[consolidated object storage settings](../object_storage.md#consolidated-object-storage-configuration). +[consolidated object storage settings](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). This section describes the earlier configuration format. See [the available connection settings for different providers](../object_storage.md#connection-settings). @@ -1100,8 +1108,8 @@ In GitLab 14.0 a number of breaking changes were introduced which may require so The steps below describe the best way to migrate without causing any downtime for your GitLab instance. A GitLab instance running on a single server typically upgrades to 14.0 smoothly, and there should be minimal issues after the upgrade is complete. -Regardless, we recommend everyone follow the migration steps to ensure a successful upgrade. -If at any point you run into issues, consult the [troubleshooting section](#troubleshooting). +Regardless, you should follow the migration steps to ensure a successful upgrade. +For common issues, see [troubleshooting](troubleshooting.md). If your current GitLab version is lower than 13.12, then you must first update to 13.12. Updating directly to 14.0 is [not supported](../../update/index.md#upgrade-paths) @@ -1185,312 +1193,6 @@ Rate limits are enforced using the following: 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them in -the section below. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> - -## Troubleshooting - -### How to see GitLab Pages logs - -You can see Pages daemon logs by running: - -```shell -sudo gitlab-ctl tail gitlab-pages -``` - -You can also find the log file in `/var/log/gitlab/gitlab-pages/current`. - -### `unsupported protocol scheme \"\""` - -If you see the following error: - -```plaintext -{"error":"failed to connect to internal Pages API: Get \"/api/v4/internal/pages/status\": unsupported protocol scheme \"\"","level":"warning","msg":"attempted to connect to the API","time":"2021-06-23T20:03:30Z"} -``` - -It means you didn't set the HTTP(S) protocol scheme in the Pages server settings. -To fix it: - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_pages['gitlab_server'] = "https://<your_pages_domain_name>" - gitlab_pages['internal_gitlab_server'] = "https://<your_pages_domain_name>" - ``` - -1. Reconfigure GitLab: - - ```shell - sudo gitlab-ctl reconfigure - ``` - -### 502 error when connecting to GitLab Pages proxy when server does not listen over IPv6 - -In some cases, NGINX might default to using IPv6 to connect to the GitLab Pages -service even when the server does not listen over IPv6. You can identify when -this is happening if you see something similar to the log entry below in the -`gitlab_pages_error.log`: - -```plaintext -2020/02/24 16:32:05 [error] 112654#0: *4982804 connect() failed (111: Connection refused) while connecting to upstream, client: 123.123.123.123, server: ~^(?<group>.*)\.pages\.example\.com$, request: "GET /-/group/project/-/jobs/1234/artifacts/artifact.txt HTTP/1.1", upstream: "http://[::1]:8090//-/group/project/-/jobs/1234/artifacts/artifact.txt", host: "group.example.com" -``` - -To resolve this, set an explicit IP and port for the GitLab Pages `listen_proxy` setting -to define the explicit address that the GitLab Pages daemon should listen on: - -```ruby -gitlab_pages['listen_proxy'] = '127.0.0.1:8090' -``` - -### Intermittent 502 errors or after a few days - -If you run Pages on a system that uses `systemd` and -[`tmpfiles.d`](https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html), -you may encounter intermittent 502 errors trying to serve Pages with an error similar to: - -```plaintext -dial tcp: lookup gitlab.example.com on [::1]:53: dial udp [::1]:53: connect: no route to host" -``` - -GitLab Pages creates a [bind mount](https://man7.org/linux/man-pages/man8/mount.8.html) -inside `/tmp/gitlab-pages-*` that includes files like `/etc/hosts`. -However, `systemd` may clean the `/tmp/` directory on a regular basis so the DNS -configuration may be lost. - -To stop `systemd` from cleaning the Pages related content: - -1. Tell `tmpfiles.d` to not remove the Pages `/tmp` directory: - - ```shell - echo 'x /tmp/gitlab-pages-*' >> /etc/tmpfiles.d/gitlab-pages-jail.conf - ``` - -1. Restart GitLab Pages: - - ```shell - sudo gitlab-ctl restart gitlab-pages - ``` - -### Unable to access GitLab Pages - -If you can't access your GitLab Pages (such as receiving `502 Bad Gateway` errors, or a login loop) -and in your Pages log shows this error: - -```plaintext -"error":"retrieval context done: context deadline exceeded","host":"root.docs-cit.otenet.gr","level":"error","msg":"could not fetch domain information from a source" -``` - -1. Add the following to `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_pages['internal_gitlab_server'] = 'http://localhost:8080' - ``` - -1. Restart GitLab Pages: - - ```shell - sudo gitlab-ctl restart gitlab-pages - ``` - -### Failed to connect to the internal GitLab API - -If you see the following error: - -```plaintext -ERRO[0010] Failed to connect to the internal GitLab API after 0.50s error="failed to connect to internal Pages API: HTTP status: 401" -``` - -If you are [Running GitLab Pages on a separate server](#running-gitlab-pages-on-a-separate-server) -you must copy the `/etc/gitlab/gitlab-secrets.json` file -from the **GitLab server** to the **Pages server** after upgrading to GitLab 13.3, -as described in that section. - -Other reasons may include network connectivity issues between your -**GitLab server** and your **Pages server** such as firewall configurations or closed ports. -For example, if there is a connection timeout: - -```plaintext -error="failed to connect to internal Pages API: Get \"https://gitlab.example.com:3000/api/v4/internal/pages/status\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" -``` - -### Pages cannot communicate with an instance of the GitLab API - -If you use the default value for `domain_config_source=auto` and run multiple instances of GitLab -Pages, you may see intermittent 502 error responses while serving Pages content. You may also see -the following warning in the Pages logs: - -```plaintext -WARN[0010] Pages cannot communicate with an instance of the GitLab API. Please sync your gitlab-secrets.json file https://gitlab.com/gitlab-org/gitlab-pages/-/issues/535#workaround. error="pages endpoint unauthorized" -``` - -This can happen if your `gitlab-secrets.json` file is out of date between GitLab Rails and GitLab -Pages. Follow steps 8-10 of [Running GitLab Pages on a separate server](#running-gitlab-pages-on-a-separate-server), -in all of your GitLab Pages instances. - -### Intermittent 502 errors when using an AWS Network Load Balancer and GitLab Pages - -Connections will time out when using a Network Load Balancer with client IP preservation enabled and [the request is looped back to the source server](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-troubleshooting.html#loopback-timeout). -This can happen to GitLab instances with multiple servers -running both the core GitLab application and GitLab Pages. This can also happen when a single -container is running both the core GitLab application and GitLab Pages. - -AWS [recommends using an IP target type](https://aws.amazon.com/premiumsupport/knowledge-center/target-connection-fails-load-balancer/) -to resolve this issue. - -Turning off [client IP preservation](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#client-ip-preservation) -may resolve this issue when the core GitLab application and GitLab Pages run on the same host or -container. - -### 500 error with `securecookie: failed to generate random iv` and `Failed to save the session` - -This problem most likely results from an [out-dated operating system](../package_information/supported_os.md#os-versions-that-are-no-longer-supported). -The [Pages daemon uses the `securecookie` library](https://gitlab.com/search?group_id=9970&project_id=734943&repository_ref=master&scope=blobs&search=securecookie&snippets=false) to get random strings via [`crypto/rand` in Go](https://pkg.go.dev/crypto/rand#pkg-variables). -This requires the `getrandom` system call or `/dev/urandom` to be available on the host OS. -Upgrading to an [officially supported operating system](https://about.gitlab.com/install/) is recommended. - -### The requested scope is invalid, malformed, or unknown - -This problem comes from the permissions of the GitLab Pages OAuth application. To fix it: - -1. On the top bar, select **Main menu > Admin**. -1. On the left sidebar, select **Applications > GitLab Pages**. -1. Edit the application. -1. Under **Scopes**, ensure that the `api` scope is selected. -1. Save your changes. - -When running a [separate Pages server](#running-gitlab-pages-on-a-separate-server), -this setting needs to be configured on the main GitLab server. - -### Workaround in case no wildcard DNS entry can be set - -If the wildcard DNS [prerequisite](#prerequisites) can't be met, you can still use GitLab Pages in a limited fashion: - -1. [Move](../../user/project/settings/index.md#transfer-a-project-to-another-namespace) - all projects you need to use Pages with into a single group namespace, for example `pages`. -1. Configure a [DNS entry](#dns-configuration) without the `*.`-wildcard, for example `pages.example.io`. -1. Configure `pages_external_url http://example.io/` in your `gitlab.rb` file. - Omit the group namespace here, because it automatically is prepended by GitLab. - -### Pages daemon fails with permission denied errors - -If `/tmp` is mounted with `noexec`, the Pages daemon fails to start with an error like: - -```plaintext -{"error":"fork/exec /gitlab-pages: permission denied","level":"fatal","msg":"could not create pages daemon","time":"2021-02-02T21:54:34Z"} -``` - -In this case, change `TMPDIR` to a location that is not mounted with `noexec`. Add the following to -`/etc/gitlab/gitlab.rb`: - -```ruby -gitlab_pages['env'] = {'TMPDIR' => '<new_tmp_path>'} -``` - -Once added, reconfigure with `sudo gitlab-ctl reconfigure` and restart GitLab with -`sudo gitlab-ctl restart`. - -### `The redirect URI included is not valid.` when using Pages Access Control - -You may see this error if `pages_external_url` was updated at some point of time. Verify the following: - -1. The **Callback URL**/Redirect URI in the GitLab Pages [System OAuth application](../../integration/oauth_provider.md#create-an-instance-wide-application) -is using the protocol (HTTP or HTTPS) that `pages_external_url` is configured to use. -1. The domain and path components of `Redirect URI` are valid: they should look like `projects.<pages_external_url>/auth`. - -### 500 error `cannot serve from disk` - -If you get a 500 response from Pages and encounter an error similar to: - -```plaintext -ERRO[0145] cannot serve from disk error="gitlab: disk access is disabled via enable-disk=false" project_id=27 source_path="file:///shared/pages/@hashed/67/06/670671cd97404156226e507973f2ab8330d3022ca96e0c93bdbdb320c41adcaf/pages_deployments/14/artifacts.zip" source_type=zip -``` - -It means that GitLab Rails is telling GitLab Pages to serve content from a location on disk, -however, GitLab Pages was configured to disable disk access. - -To enable disk access: - -1. Enable disk access for GitLab Pages in `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_pages['enable_disk'] = true - ``` - -1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). - -### `httprange: new resource 403` - -If you see an error similar to: - -```plaintext -{"error":"httprange: new resource 403: \"403 Forbidden\"","host":"root.pages.example.com","level":"error","msg":"vfs.Root","path":"/pages1/","time":"2021-06-10T08:45:19Z"} -``` - -And you run pages on the separate server syncing files via NFS, it may mean that -the shared pages directory is mounted on a different path on the main GitLab server and the -GitLab Pages server. - -In that case, it's highly recommended you to configure -[object storage and migrate any existing pages data to it](#using-object-storage). - -Alternatively, you can mount the GitLab Pages shared directory to the same path on -both servers. - -### GitLab Pages doesn't work after upgrading to GitLab 14.0 or above - -GitLab 14.0 introduces a number of changes to GitLab Pages which may require manual intervention. - -1. Firstly [follow the migration guide](#prepare-gitlab-pages-for-140). -1. Try to upgrade to GitLab 14.3 or above. Some of the issues were fixed in GitLab 14.1, 14.2 and 14.3. -1. If it doesn't work, see [GitLab Pages logs](#how-to-see-gitlab-pages-logs), and if you see any errors there then search them on this page. - -WARNING: -In GitLab 14.0-14.2 you can temporarily enable legacy storage and configuration mechanisms. - -To do that: - -1. Describe the issue you're seeing in the [migration feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/331699). - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_pages['use_legacy_storage'] = true - ``` - -1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). - -### GitLab Pages deploy job fails with error "is not a recognized provider" - -If the **pages** job succeeds but the **deploy** job gives the error "is not a recognized provider": - - - -The error message `is not a recognized provider` could be coming from the `fog` gem that GitLab uses to connect to cloud providers for object storage. - -To fix that: - -1. Check your `gitlab.rb` file. If you have `gitlab_rails['pages_object_store_enabled']` enabled, but no bucket details have been configured, either: - - - Configure object storage for your Pages deployments, following the [S3-compatible connection settings](#s3-compatible-connection-settings) guide. - - Store your deployments locally, by commenting out that line. - -1. Save the changes you made to your `gitlab.rb` file, then [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). - -### 404 error `The page you're looking for could not be found` - -If you get a `404 Page Not Found` response from GitLab Pages: - -1. Check `.gitlab-ci.yml` contains the job `pages:`. -1. Check the current project's pipeline to confirm the job `pages:deploy` is being run. +## Related topics -Without the `pages:deploy` job, the updates to your GitLab Pages site are never published. +- [Troubleshooting GitLab Pages administration](troubleshooting.md) diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md index db76d15ec58..457e92d96bd 100644 --- a/doc/administration/pages/source.md +++ b/doc/administration/pages/source.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/administration/pages/troubleshooting.md b/doc/administration/pages/troubleshooting.md new file mode 100644 index 00000000000..6f00ae074f5 --- /dev/null +++ b/doc/administration/pages/troubleshooting.md @@ -0,0 +1,304 @@ +--- +stage: Plan +group: Knowledge +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Troubleshooting GitLab Pages administration **(FREE SELF)** + +This page contains a list of issues you might encounter when administering GitLab Pages. + +## How to see GitLab Pages logs + +You can see Pages daemon logs by running: + +```shell +sudo gitlab-ctl tail gitlab-pages +``` + +You can also find the log file in `/var/log/gitlab/gitlab-pages/current`. + +## `unsupported protocol scheme \"\""` + +If you see the following error: + +```plaintext +{"error":"failed to connect to internal Pages API: Get \"/api/v4/internal/pages/status\": unsupported protocol scheme \"\"","level":"warning","msg":"attempted to connect to the API","time":"2021-06-23T20:03:30Z"} +``` + +It means you didn't set the HTTP(S) protocol scheme in the Pages server settings. +To fix it: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['gitlab_server'] = "https://<your_gitlab_server_public_host_and_port>" + gitlab_pages['internal_gitlab_server'] = "https://<your_gitlab_server_private_host_and_port>" # optional, gitlab_pages['gitlab_server'] is used as default + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +## 502 error when connecting to GitLab Pages proxy when server does not listen over IPv6 + +In some cases, NGINX might default to using IPv6 to connect to the GitLab Pages +service even when the server does not listen over IPv6. You can identify when +this is happening if you see something similar to the log entry below in the +`gitlab_pages_error.log`: + +```plaintext +2020/02/24 16:32:05 [error] 112654#0: *4982804 connect() failed (111: Connection refused) while connecting to upstream, client: 123.123.123.123, server: ~^(?<group>.*)\.pages\.example\.com$, request: "GET /-/group/project/-/jobs/1234/artifacts/artifact.txt HTTP/1.1", upstream: "http://[::1]:8090//-/group/project/-/jobs/1234/artifacts/artifact.txt", host: "group.example.com" +``` + +To resolve this, set an explicit IP and port for the GitLab Pages `listen_proxy` setting +to define the explicit address that the GitLab Pages daemon should listen on: + +```ruby +gitlab_pages['listen_proxy'] = '127.0.0.1:8090' +``` + +## Intermittent 502 errors or after a few days + +If you run Pages on a system that uses `systemd` and +[`tmpfiles.d`](https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html), +you may encounter intermittent 502 errors trying to serve Pages with an error similar to: + +```plaintext +dial tcp: lookup gitlab.example.com on [::1]:53: dial udp [::1]:53: connect: no route to host" +``` + +GitLab Pages creates a [bind mount](https://man7.org/linux/man-pages/man8/mount.8.html) +inside `/tmp/gitlab-pages-*` that includes files like `/etc/hosts`. +However, `systemd` may clean the `/tmp/` directory on a regular basis so the DNS +configuration may be lost. + +To stop `systemd` from cleaning the Pages related content: + +1. Tell `tmpfiles.d` to not remove the Pages `/tmp` directory: + + ```shell + echo 'x /tmp/gitlab-pages-*' >> /etc/tmpfiles.d/gitlab-pages-jail.conf + ``` + +1. Restart GitLab Pages: + + ```shell + sudo gitlab-ctl restart gitlab-pages + ``` + +## Unable to access GitLab Pages + +If you can't access your GitLab Pages (such as receiving `502 Bad Gateway` errors, or a login loop) +and in your Pages log shows this error: + +```plaintext +"error":"retrieval context done: context deadline exceeded","host":"root.docs-cit.otenet.gr","level":"error","msg":"could not fetch domain information from a source" +``` + +1. Add the following to `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['internal_gitlab_server'] = 'http://localhost:8080' + ``` + +1. Restart GitLab Pages: + + ```shell + sudo gitlab-ctl restart gitlab-pages + ``` + +## Failed to connect to the internal GitLab API + +If you see the following error: + +```plaintext +ERRO[0010] Failed to connect to the internal GitLab API after 0.50s error="failed to connect to internal Pages API: HTTP status: 401" +``` + +If you are [Running GitLab Pages on a separate server](index.md#running-gitlab-pages-on-a-separate-server) +you must copy the `/etc/gitlab/gitlab-secrets.json` file +from the **GitLab server** to the **Pages server** after upgrading to GitLab 13.3, +as described in that section. + +Other reasons may include network connectivity issues between your +**GitLab server** and your **Pages server** such as firewall configurations or closed ports. +For example, if there is a connection timeout: + +```plaintext +error="failed to connect to internal Pages API: Get \"https://gitlab.example.com:3000/api/v4/internal/pages/status\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" +``` + +## Pages cannot communicate with an instance of the GitLab API + +If you use the default value for `domain_config_source=auto` and run multiple instances of GitLab +Pages, you may see intermittent 502 error responses while serving Pages content. You may also see +the following warning in the Pages logs: + +```plaintext +WARN[0010] Pages cannot communicate with an instance of the GitLab API. Please sync your gitlab-secrets.json file https://gitlab.com/gitlab-org/gitlab-pages/-/issues/535#workaround. error="pages endpoint unauthorized" +``` + +This can happen if your `gitlab-secrets.json` file is out of date between GitLab Rails and GitLab +Pages. Follow steps 8-10 of [Running GitLab Pages on a separate server](index.md#running-gitlab-pages-on-a-separate-server), +in all of your GitLab Pages instances. + +## Intermittent 502 errors when using an AWS Network Load Balancer and GitLab Pages + +Connections will time out when using a Network Load Balancer with client IP preservation enabled and [the request is looped back to the source server](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-troubleshooting.html#loopback-timeout). +This can happen to GitLab instances with multiple servers +running both the core GitLab application and GitLab Pages. This can also happen when a single +container is running both the core GitLab application and GitLab Pages. + +AWS [recommends using an IP target type](https://aws.amazon.com/premiumsupport/knowledge-center/target-connection-fails-load-balancer/) +to resolve this issue. + +Turning off [client IP preservation](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#client-ip-preservation) +may resolve this issue when the core GitLab application and GitLab Pages run on the same host or +container. + +## 500 error with `securecookie: failed to generate random iv` and `Failed to save the session` + +This problem most likely results from an [out-dated operating system](../package_information/supported_os.md#os-versions-that-are-no-longer-supported). +The [Pages daemon uses the `securecookie` library](https://gitlab.com/search?group_id=9970&project_id=734943&repository_ref=master&scope=blobs&search=securecookie&snippets=false) to get random strings via [`crypto/rand` in Go](https://pkg.go.dev/crypto/rand#pkg-variables). +This requires the `getrandom` system call or `/dev/urandom` to be available on the host OS. +Upgrading to an [officially supported operating system](https://about.gitlab.com/install/) is recommended. + +## The requested scope is invalid, malformed, or unknown + +This problem comes from the permissions of the GitLab Pages OAuth application. To fix it: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Applications > GitLab Pages**. +1. Edit the application. +1. Under **Scopes**, ensure that the `api` scope is selected. +1. Save your changes. + +When running a [separate Pages server](index.md#running-gitlab-pages-on-a-separate-server), +this setting needs to be configured on the main GitLab server. + +## Workaround in case no wildcard DNS entry can be set + +If the wildcard DNS [prerequisite](index.md#prerequisites) can't be met, you can still use GitLab Pages in a limited fashion: + +1. [Move](../../user/project/settings/index.md#transfer-a-project-to-another-namespace) + all projects you need to use Pages with into a single group namespace, for example `pages`. +1. Configure a [DNS entry](index.md#dns-configuration) without the `*.`-wildcard, for example `pages.example.io`. +1. Configure `pages_external_url http://example.io/` in your `gitlab.rb` file. + Omit the group namespace here, because it automatically is prepended by GitLab. + +## Pages daemon fails with permission denied errors + +If `/tmp` is mounted with `noexec`, the Pages daemon fails to start with an error like: + +```plaintext +{"error":"fork/exec /gitlab-pages: permission denied","level":"fatal","msg":"could not create pages daemon","time":"2021-02-02T21:54:34Z"} +``` + +In this case, change `TMPDIR` to a location that is not mounted with `noexec`. Add the following to +`/etc/gitlab/gitlab.rb`: + +```ruby +gitlab_pages['env'] = {'TMPDIR' => '<new_tmp_path>'} +``` + +Once added, reconfigure with `sudo gitlab-ctl reconfigure` and restart GitLab with +`sudo gitlab-ctl restart`. + +## `The redirect URI included is not valid.` when using Pages Access Control + +You may see this error if `pages_external_url` was updated at some point of time. Verify the following: + +1. The **Callback URL**/Redirect URI in the GitLab Pages [System OAuth application](../../integration/oauth_provider.md#create-an-instance-wide-application) +is using the protocol (HTTP or HTTPS) that `pages_external_url` is configured to use. +1. The domain and path components of `Redirect URI` are valid: they should look like `projects.<pages_external_url>/auth`. + +## 500 error `cannot serve from disk` + +If you get a 500 response from Pages and encounter an error similar to: + +```plaintext +ERRO[0145] cannot serve from disk error="gitlab: disk access is disabled via enable-disk=false" project_id=27 source_path="file:///shared/pages/@hashed/67/06/670671cd97404156226e507973f2ab8330d3022ca96e0c93bdbdb320c41adcaf/pages_deployments/14/artifacts.zip" source_type=zip +``` + +It means that GitLab Rails is telling GitLab Pages to serve content from a location on disk, +however, GitLab Pages was configured to disable disk access. + +To enable disk access: + +1. Enable disk access for GitLab Pages in `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['enable_disk'] = true + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + +## `httprange: new resource 403` + +If you see an error similar to: + +```plaintext +{"error":"httprange: new resource 403: \"403 Forbidden\"","host":"root.pages.example.com","level":"error","msg":"vfs.Root","path":"/pages1/","time":"2021-06-10T08:45:19Z"} +``` + +And you run pages on the separate server syncing files via NFS, it may mean that +the shared pages directory is mounted on a different path on the main GitLab server and the +GitLab Pages server. + +In that case, it's highly recommended you to configure +[object storage and migrate any existing pages data to it](index.md#using-object-storage). + +Alternatively, you can mount the GitLab Pages shared directory to the same path on +both servers. + +## GitLab Pages doesn't work after upgrading to GitLab 14.0 or above + +GitLab 14.0 introduces a number of changes to GitLab Pages which may require manual intervention. + +1. Firstly [follow the migration guide](index.md#prepare-gitlab-pages-for-140). +1. Try to upgrade to GitLab 14.3 or above. Some of the issues were fixed in GitLab 14.1, 14.2 and 14.3. +1. If it doesn't work, see [GitLab Pages logs](#how-to-see-gitlab-pages-logs), and if you see any errors there then search them on this page. + +WARNING: +In GitLab 14.0-14.2 you can temporarily enable legacy storage and configuration mechanisms. + +To do that: + +1. Describe the issue you're seeing in the [migration feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/331699). + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['use_legacy_storage'] = true + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + +## GitLab Pages deploy job fails with error "is not a recognized provider" + +If the **pages** job succeeds but the **deploy** job gives the error "is not a recognized provider": + + + +The error message `is not a recognized provider` could be coming from the `fog` gem that GitLab uses to connect to cloud providers for object storage. + +To fix that: + +1. Check your `gitlab.rb` file. If you have `gitlab_rails['pages_object_store_enabled']` enabled, but no bucket details have been configured, either: + + - Configure object storage for your Pages deployments, following the [S3-compatible connection settings](index.md#s3-compatible-connection-settings) guide. + - Store your deployments locally, by commenting out that line. + +1. Save the changes you made to your `gitlab.rb` file, then [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + +## 404 error `The page you're looking for could not be found` + +If you get a `404 Page Not Found` response from GitLab Pages: + +1. Check `.gitlab-ci.yml` contains the job `pages:`. +1. Check the current project's pipeline to confirm the job `pages:deploy` is being run. + +Without the `pages:deploy` job, the updates to your GitLab Pages site are never published. diff --git a/doc/administration/postgresql/database_load_balancing.md b/doc/administration/postgresql/database_load_balancing.md index 15129770888..d5cf93a135a 100644 --- a/doc/administration/postgresql/database_load_balancing.md +++ b/doc/administration/postgresql/database_load_balancing.md @@ -235,3 +235,8 @@ operation retries up to 3 times using an exponential back-off. When using load balancing, you should be able to safely restart a database server without it immediately leading to errors being presented to the users. + +### Development guide + +For detailed development guide on database load balancing, +see [the development documentation](../../development/database/load_balancing.md). diff --git a/doc/administration/postgresql/multiple_databases.md b/doc/administration/postgresql/multiple_databases.md index 188578a739c..857fd4fc9c5 100644 --- a/doc/administration/postgresql/multiple_databases.md +++ b/doc/administration/postgresql/multiple_databases.md @@ -15,7 +15,7 @@ By default, GitLab uses a single application database, referred to as the `main` To scale GitLab, you can configure GitLab to use multiple application databases. -Due to [known issues](#known-issues), configuring GitLab with multiple databases is in [**Alpha**](../../policy/alpha-beta-support.md#alpha-features). +Due to [known issues](#known-issues), configuring GitLab with multiple databases is an [Experiment](../../policy/alpha-beta-support.md#experiment). After you have set up multiple databases, GitLab uses a second application database for [CI/CD features](../../ci/index.md), referred to as the `ci` database. diff --git a/doc/administration/postgresql/replication_and_failover.md b/doc/administration/postgresql/replication_and_failover.md index 46890b0b2ca..8ac3ac40a75 100644 --- a/doc/administration/postgresql/replication_and_failover.md +++ b/doc/administration/postgresql/replication_and_failover.md @@ -1255,8 +1255,6 @@ To do the switch on **all** PgBouncer nodes: ``` 1. Run `gitlab-ctl reconfigure`. -1. You must also run `rm /var/opt/gitlab/consul/config.d/watcher_postgresql.json`. - This is a [known issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7293). #### Clean up diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index 5c258d73fdb..055cf944046 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -11,7 +11,7 @@ GitLab provides Rake tasks for general maintenance. ## Gather GitLab and system information This command gathers information about your GitLab installation and the system it runs on. -These may be useful when asking for help or reporting issues. +These may be useful when asking for help or reporting issues. In a multi-node environment, run this command on nodes running GitLab Rails to avoid PostgreSQL socket errors. **Omnibus Installation** @@ -117,7 +117,7 @@ If you're running Geo, see also the [Geo Health check Rake task](../geo/replicat You may also have a look at our troubleshooting guides for: -- [GitLab](../index.md#troubleshooting) +- [GitLab](../troubleshooting/index.md) - [Omnibus GitLab](https://docs.gitlab.com/omnibus/index.html#troubleshooting) Additionally you should also [verify database values can be decrypted using the current secrets](check.md#verify-database-values-can-be-decrypted-using-the-current-secrets). diff --git a/doc/administration/raketasks/service_desk_email.md b/doc/administration/raketasks/service_desk_email.md index 10de379b1cd..1cbdec35171 100644 --- a/doc/administration/raketasks/service_desk_email.md +++ b/doc/administration/raketasks/service_desk_email.md @@ -12,7 +12,7 @@ The following are Service Desk email-related Rake tasks. ## Secrets -GitLab can use [Service Desk email](../../user/project/service_desk.md#configuring-a-custom-mailbox) secrets read from an encrypted file instead of storing them in plaintext in the file system. The following Rake tasks are provided for updating the contents of the encrypted file. +GitLab can use [Service Desk email](../../user/project/service_desk.md#configure-a-custom-mailbox) secrets read from an encrypted file instead of storing them in plaintext in the file system. The following Rake tasks are provided for updating the contents of the encrypted file. ### Show secret diff --git a/doc/administration/redis/troubleshooting.md b/doc/administration/redis/troubleshooting.md index 29407f65efd..fbfe52570c4 100644 --- a/doc/administration/redis/troubleshooting.md +++ b/doc/administration/redis/troubleshooting.md @@ -113,15 +113,19 @@ To make sure your configuration is correct: redis.info ``` - Keep this screen open and try to simulate a failover below. + Keep this screen open, and proceed to trigger a failover as described below. -1. To simulate a failover on primary Redis, SSH into the Redis server and run: +1. To trigger a failover on the primary Redis, SSH into the Redis server and run: ```shell # port must match your primary redis port, and the sleep time must be a few seconds bigger than defined one redis-cli -h localhost -p 6379 DEBUG sleep 20 ``` + WARNING: + This action affects services, and takes the instance down for up to 20 seconds. If successful, + it should recover after that. + 1. Then back in the Rails console from the first step, run: ```ruby diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 1934ca6bff0..e70d012507f 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -31,8 +31,8 @@ full list of reference architectures, see | Gitaly<sup>5 6</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | | Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | | Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | +| Sidekiq<sup>7</sup> | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| GitLab Rails<sup>7</sup> | 3 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | | Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | | Object storage<sup>4</sup> | - | - | - | - | @@ -53,6 +53,8 @@ full list of reference architectures, see 6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to [Large repositories](index.md#large-repositories) for more information. +7. Can be placed in Auto Scaling Groups (ASGs) as the component doesn't store any [stateful data](index.md#autoscaling-of-stateful-nodes). + However, for GitLab Rails certain processes like [migrations](#gitlab-rails-post-configuration) and [Mailroom](../incoming_email.md) should be run on only one node. <!-- markdownlint-enable MD029 --> NOTE: @@ -417,6 +419,11 @@ spread connections equally in practice. ## Configure Consul +Next, we set up the Consul servers. + +NOTE: +Consul must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + The following IPs will be used as an example: - `10.6.0.11`: Consul 1 @@ -801,6 +808,9 @@ Using [Redis](https://redis.io/) in scalable environment is possible using a **P topology with a [Redis Sentinel](https://redis.io/docs/manual/sentinel/) service to watch and automatically start the failover procedure. +NOTE: +Redis clusters must each be deployed in an odd number of 3 nodes or higher. This is to ensure Redis Sentinel can take votes as part of a quorum. This does not apply when configuring Redis externally, such as a cloud provider service. + Redis requires authentication if used with Sentinel. See [Redis Security](https://redis.io/docs/manual/security/) documentation for more information. We recommend using a combination of a Redis password and tight @@ -1336,6 +1346,9 @@ This is how this would work with a Omnibus GitLab PostgreSQL setup: Praefect is the router and transaction manager for Gitaly Cluster and all connections to Gitaly go through it. This section details how to configure it. +NOTE: +Consul must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + Praefect requires several secret tokens to secure communications across the Cluster: - `<praefect_external_token>`: Used for repositories hosted on your Gitaly cluster and can only be accessed by Gitaly clients that carry this token. @@ -1510,9 +1523,7 @@ Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should have a throughput of at least 8,000 input/output operations per second (IOPS) for read operations and 2,000 IOPS for -write operations. These IOPS values are initial recommendations, and may be -adjusted to greater or lesser values depending on the scale of your -environment's workload. If you're running the environment on a Cloud provider, +write operations. If you're running the environment on a Cloud provider, refer to their documentation about how to configure IOPS correctly. Gitaly servers must not be exposed to the public internet, as Gitaly's network @@ -2089,11 +2100,9 @@ the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). sudo gitlab-rake gitlab:db:configure ``` - If you encounter a `rake aborted!` error message stating that PgBouncer is - failing to connect to PostgreSQL, it may be that your PgBouncer node's IP - address is missing from PostgreSQL's `trust_auth_cidr_addresses` in `gitlab.rb` - on your database nodes. Before proceeding, see - [PgBouncer error `ERROR: pgbouncer cannot connect to server`](../postgresql/replication_and_failover.md#pgbouncer-error-error-pgbouncer-cannot-connect-to-server). + Note that this requires the Rails node to be configured to connect to the primary database + directly, [bypassing PgBouncer](../postgresql/pgbouncer.md#procedure-for-bypassing-pgbouncer). + After migrations have completed, you must configure the node to pass through PgBouncer again. 1. [Configure fast lookup of authorized SSH keys in the database](../operations/fast_ssh_key_lookup.md). @@ -2189,15 +2198,15 @@ and scalable. There are two ways of specifying object storage configuration in GitLab: -- [Consolidated form](../object_storage.md#consolidated-object-storage-configuration): A single credential is +- [Consolidated form](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form): A single credential is shared by all supported object types. -- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its +- [Storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). The consolidated form is used in the following examples when available. NOTE: -When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +When using the [storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, which was deprecated in 14.9, requires shared storage such as NFS. diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 08cb6e2cdff..19a79657218 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -31,8 +31,8 @@ full list of reference architectures, see | Gitaly<sup>5 6</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | | Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | | Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| GitLab Rails | 5 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | +| Sidekiq<sup>7</sup> | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| GitLab Rails<sup>7</sup> | 5 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | | Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | | Object storage<sup>4</sup> | - | - | - | - | @@ -53,6 +53,8 @@ full list of reference architectures, see 6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to [Large repositories](index.md#large-repositories) for more information. +7. Can be placed in Auto Scaling Groups (ASGs) as the component doesn't store any [stateful data](index.md#autoscaling-of-stateful-nodes). + However, for GitLab Rails certain processes like [migrations](#gitlab-rails-post-configuration) and [Mailroom](../incoming_email.md) should be run on only one node. <!-- markdownlint-enable MD029 --> NOTE: @@ -434,6 +436,11 @@ spread connections equally in practice. ## Configure Consul +Next, we set up the Consul servers. + +NOTE: +Consul must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + The following IPs will be used as an example: - `10.6.0.11`: Consul 1 @@ -818,6 +825,9 @@ Using [Redis](https://redis.io/) in scalable environment is possible using a **P topology with a [Redis Sentinel](https://redis.io/docs/manual/sentinel/) service to watch and automatically start the failover procedure. +NOTE: +Redis clusters must each be deployed in an odd number of 3 nodes or higher. This is to ensure Redis Sentinel can take votes as part of a quorum. This does not apply when configuring Redis externally, such as a cloud provider service. + Redis requires authentication if used with Sentinel. See [Redis Security](https://redis.io/docs/manual/security/) documentation for more information. We recommend using a combination of a Redis password and tight @@ -1353,6 +1363,9 @@ This is how this would work with a Omnibus GitLab PostgreSQL setup: Praefect is the router and transaction manager for Gitaly Cluster and all connections to Gitaly go through it. This section details how to configure it. +NOTE: +Praefect must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + Praefect requires several secret tokens to secure communications across the Cluster: - `<praefect_external_token>`: Used for repositories hosted on your Gitaly cluster and can only be accessed by Gitaly clients that carry this token. @@ -1527,9 +1540,7 @@ Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should have a throughput of at least 8,000 input/output operations per second (IOPS) for read operations and 2,000 IOPS for -write operations. These IOPS values are initial recommendations, and may be -adjusted to greater or lesser values depending on the scale of your -environment's workload. If you're running the environment on a Cloud provider, +write operations. If you're running the environment on a Cloud provider, refer to their documentation about how to configure IOPS correctly. Gitaly servers must not be exposed to the public internet, as Gitaly's network @@ -2108,11 +2119,9 @@ the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). sudo gitlab-rake gitlab:db:configure ``` - If you encounter a `rake aborted!` error message stating that PgBouncer is - failing to connect to PostgreSQL, it may be that your PgBouncer node's IP - address is missing from PostgreSQL's `trust_auth_cidr_addresses` in `gitlab.rb` - on your database nodes. Before proceeding, see - [PgBouncer error `ERROR: pgbouncer cannot connect to server`](../postgresql/replication_and_failover.md#pgbouncer-error-error-pgbouncer-cannot-connect-to-server). + Note that this requires the Rails node to be configured to connect to the primary database + directly, [bypassing PgBouncer](../postgresql/pgbouncer.md#procedure-for-bypassing-pgbouncer). + After migrations have completed, you must configure the node to pass through PgBouncer again. 1. [Configure fast lookup of authorized SSH keys in the database](../operations/fast_ssh_key_lookup.md). @@ -2207,15 +2216,15 @@ and scalable. There are two ways of specifying object storage configuration in GitLab: -- [Consolidated form](../object_storage.md#consolidated-object-storage-configuration): A single credential is +- [Consolidated form](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form): A single credential is shared by all supported object types. -- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its +- [Storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). The consolidated form is used in the following examples when available. NOTE: -When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +When using the [storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, which was deprecated in 14.9, requires shared storage such as NFS. diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index d87e8270dcb..f31cf6f4ab6 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -26,7 +26,7 @@ For a full list of reference architectures, see | PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | | Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` | `D2s v3` | | Gitaly<sup>5</sup> | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| GitLab Rails | 2 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | +| GitLab Rails<sup>6</sup> | 2 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | | Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | | Object storage<sup>4</sup> | - | - | - | - | - | @@ -45,6 +45,8 @@ For a full list of reference architectures, see 5. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to [Large repositories](index.md#large-repositories) for more information. +6. Can be placed in Auto Scaling Groups (ASGs) as the component doesn't store any [stateful data](index.md#autoscaling-of-stateful-nodes). + However, for GitLab Rails certain processes like [migrations](#gitlab-rails-post-configuration) and [Mailroom](../incoming_email.md) should be run on only one node. <!-- markdownlint-enable MD029 --> NOTE: @@ -423,9 +425,7 @@ Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should have a throughput of at least 8,000 input/output operations per second (IOPS) for read operations and 2,000 IOPS -for write operations. These IOPS values are initial recommendations, and may be -adjusted to greater or lesser values depending on the scale of your -environment's workload. If you're running the environment on a Cloud provider, +for write operations. If you're running the environment on a Cloud provider, refer to their documentation about how to configure IOPS correctly. Be sure to note the following items: @@ -774,11 +774,9 @@ the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). sudo gitlab-rake gitlab:db:configure ``` - If you encounter a `rake aborted!` error message stating that PgBouncer is - failing to connect to PostgreSQL, it may be that your PgBouncer node's IP - address is missing from PostgreSQL's `trust_auth_cidr_addresses` in `gitlab.rb` - on your database nodes. Before proceeding, see - [PgBouncer error `ERROR: pgbouncer cannot connect to server`](../postgresql/replication_and_failover.md#pgbouncer-error-error-pgbouncer-cannot-connect-to-server). + Note that this requires the Rails node to be configured to connect to the primary database + directly, [bypassing PgBouncer](../postgresql/pgbouncer.md#procedure-for-bypassing-pgbouncer). + After migrations have completed, you must configure the node to pass through PgBouncer again. 1. [Configure fast lookup of authorized SSH keys in the database](../operations/fast_ssh_key_lookup.md). @@ -904,15 +902,15 @@ and scalable. There are two ways of specifying object storage configuration in GitLab: -- [Consolidated form](../object_storage.md#consolidated-object-storage-configuration): A single credential is +- [Consolidated form](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form): A single credential is shared by all supported object types. -- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its +- [Storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). The consolidated form is used in the following examples when available. NOTE: -When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +When using the [storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, which was deprecated in 14.9, requires shared storage such as NFS. diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index c28799ed459..a01eaee10d3 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -40,8 +40,8 @@ For a full list of reference architectures, see | Gitaly<sup>5 6</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | | Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | | Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | -| GitLab Rails | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | +| Sidekiq<sup>7</sup> | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | +| GitLab Rails<sup>7</sup> | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | | Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | | Object storage<sup>4</sup> | - | - | - | - | @@ -62,6 +62,8 @@ For a full list of reference architectures, see 6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to [Large repositories](index.md#large-repositories) for more information. +7. Can be placed in Auto Scaling Groups (ASGs) as the component doesn't store any [stateful data](index.md#autoscaling-of-stateful-nodes). + However, for GitLab Rails certain processes like [migrations](#gitlab-rails-post-configuration) and [Mailroom](../incoming_email.md) should be run on only one node. <!-- markdownlint-enable MD029 --> NOTE: @@ -640,8 +642,13 @@ are supported and can be added if needed. ## Configure Consul and Sentinel -Now that the Redis servers are all set up, let's configure the Sentinel -servers. The following IPs will be used as an example: +Now that the Redis servers are all set up, let's configure the Consul + Sentinel +servers. + +NOTE: +Consul and Redis Sentinel must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + +The following IPs will be used as an example: - `10.6.0.11`: Consul/Sentinel 1 - `10.6.0.12`: Consul/Sentinel 2 @@ -1288,6 +1295,9 @@ This is how this would work with a Omnibus GitLab PostgreSQL setup: Praefect is the router and transaction manager for Gitaly Cluster and all connections to Gitaly go through it. This section details how to configure it. +NOTE: +Praefect must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + Praefect requires several secret tokens to secure communications across the Cluster: - `<praefect_external_token>`: Used for repositories hosted on your Gitaly cluster and can only be accessed by Gitaly clients that carry this token. @@ -1462,9 +1472,7 @@ Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should have a throughput of at least 8,000 input/output operations per second (IOPS) for read operations and 2,000 IOPS for -write operations. These IOPS values are initial recommendations, and may be -adjusted to greater or lesser values depending on the scale of your -environment's workload. If you're running the environment on a Cloud provider, +write operations. If you're running the environment on a Cloud provider, refer to their documentation about how to configure IOPS correctly. Gitaly servers must not be exposed to the public internet, as Gitaly's network @@ -2073,11 +2081,9 @@ the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). gitlab-rake gitlab:db:configure ``` - If you encounter a `rake aborted!` error message stating that PgBouncer is - failing to connect to PostgreSQL, it may be that your PgBouncer node's IP - address is missing from PostgreSQL's `trust_auth_cidr_addresses` in `gitlab.rb` - on your database nodes. Before proceeding, see - [PgBouncer error `ERROR: pgbouncer cannot connect to server`](../postgresql/replication_and_failover.md#pgbouncer-error-error-pgbouncer-cannot-connect-to-server). + Note that this requires the Rails node to be configured to connect to the primary database + directly, [bypassing PgBouncer](../postgresql/pgbouncer.md#procedure-for-bypassing-pgbouncer). + After migrations have completed, you must configure the node to pass through PgBouncer again. 1. [Configure fast lookup of authorized SSH keys in the database](../operations/fast_ssh_key_lookup.md). @@ -2180,15 +2186,15 @@ and scalable. There are two ways of specifying object storage configuration in GitLab: -- [Consolidated form](../object_storage.md#consolidated-object-storage-configuration): A single credential is +- [Consolidated form](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form): A single credential is shared by all supported object types. -- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its +- [Storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). The consolidated form is used in the following examples when available. NOTE: -When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +When using the [storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, which was deprecated in 14.9, requires shared storage such as NFS. diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 3f2771dda29..d4423ef4f9f 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -31,8 +31,8 @@ full list of reference architectures, see | Gitaly<sup>5 6</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` | | Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | | Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| GitLab Rails | 12 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | +| Sidekiq<sup>7</sup> | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| GitLab Rails<sup>7</sup> | 12 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | | Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | | Object storage<sup>4</sup> | - | - | - | - | @@ -53,6 +53,8 @@ full list of reference architectures, see 6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to [Large repositories](index.md#large-repositories) for more information. +7. Can be placed in Auto Scaling Groups (ASGs) as the component doesn't store any [stateful data](index.md#autoscaling-of-stateful-nodes). + However, for GitLab Rails certain processes like [migrations](#gitlab-rails-post-configuration) and [Mailroom](../incoming_email.md) should be run on only one node. <!-- markdownlint-enable MD029 --> NOTE: @@ -426,6 +428,11 @@ spread connections equally in practice. ## Configure Consul +Next, we set up the Consul servers. + +NOTE: +Consul must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + The following IPs will be used as an example: - `10.6.0.11`: Consul 1 @@ -811,6 +818,9 @@ Using [Redis](https://redis.io/) in scalable environment is possible using a **P topology with a [Redis Sentinel](https://redis.io/docs/manual/sentinel/) service to watch and automatically start the failover procedure. +NOTE: +Redis clusters must each be deployed in an odd number of 3 nodes or higher. This is to ensure Redis Sentinel can take votes as part of a quorum. This does not apply when configuring Redis externally, such as a cloud provider service. + Redis requires authentication if used with Sentinel. See [Redis Security](https://redis.io/docs/manual/security/) documentation for more information. We recommend using a combination of a Redis password and tight @@ -1349,6 +1359,9 @@ This is how this would work with a Omnibus GitLab PostgreSQL setup: Praefect is the router and transaction manager for Gitaly Cluster and all connections to Gitaly go through it. This section details how to configure it. +NOTE: +Praefect must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + Praefect requires several secret tokens to secure communications across the Cluster: - `<praefect_external_token>`: Used for repositories hosted on your Gitaly cluster and can only be accessed by Gitaly clients that carry this token. @@ -1523,9 +1536,7 @@ Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should have a throughput of at least 8,000 input/output operations per second (IOPS) for read operations and 2,000 IOPS for -write operations. These IOPS values are initial recommendations, and may be -adjusted to greater or lesser values depending on the scale of your -environment's workload. If you're running the environment on a Cloud provider, +write operations. If you're running the environment on a Cloud provider, refer to their documentation about how to configure IOPS correctly. Gitaly servers must not be exposed to the public internet, as Gitaly's network @@ -2107,11 +2118,9 @@ the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). sudo gitlab-rake gitlab:db:configure ``` - If you encounter a `rake aborted!` error message stating that PgBouncer is - failing to connect to PostgreSQL, it may be that your PgBouncer node's IP - address is missing from PostgreSQL's `trust_auth_cidr_addresses` in `gitlab.rb` - on your database nodes. Before proceeding, see - [PgBouncer error `ERROR: pgbouncer cannot connect to server`](../postgresql/replication_and_failover.md#pgbouncer-error-error-pgbouncer-cannot-connect-to-server). + Note that this requires the Rails node to be configured to connect to the primary database + directly, [bypassing PgBouncer](../postgresql/pgbouncer.md#procedure-for-bypassing-pgbouncer). + After migrations have completed, you must configure the node to pass through PgBouncer again. 1. [Configure fast lookup of authorized SSH keys in the database](../operations/fast_ssh_key_lookup.md). @@ -2206,15 +2215,15 @@ and scalable. There are two ways of specifying object storage configuration in GitLab: -- [Consolidated form](../object_storage.md#consolidated-object-storage-configuration): A single credential is +- [Consolidated form](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form): A single credential is shared by all supported object types. -- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its +- [Storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). The consolidated form is used in the following examples when available. NOTE: -When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +When using the [storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, which was deprecated in 14.9, requires shared storage such as NFS. diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 43d63d4f08e..627812f5647 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -37,8 +37,8 @@ costly-to-operate environment by using the | Gitaly<sup>5 6</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | | Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | | Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | -| GitLab Rails | 3 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | +| Sidekiq<sup>7</sup> | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | +| GitLab Rails<sup>7</sup> | 3 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | | Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | | Object storage<sup>4</sup> | - | - | - | - | @@ -59,6 +59,8 @@ costly-to-operate environment by using the 6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to [Large repositories](index.md#large-repositories) for more information. +7. Can be placed in Auto Scaling Groups (ASGs) as the component doesn't store any [stateful data](index.md#autoscaling-of-stateful-nodes). + However, for GitLab Rails certain processes like [migrations](#gitlab-rails-post-configuration) and [Mailroom](../incoming_email.md) should be run on only one node. <!-- markdownlint-enable MD029 --> NOTE: @@ -637,8 +639,13 @@ are supported and can be added if needed. ## Configure Consul and Sentinel -Now that the Redis servers are all set up, let's configure the Sentinel -servers. The following IPs are used as an example: +Now that the Redis servers are all set up, let's configure the Consul + Sentinel +servers. + +NOTE: +Consul and Redis Sentinel must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + +The following IPs will be used as an example: - `10.6.0.11`: Consul/Sentinel 1 - `10.6.0.12`: Consul/Sentinel 2 @@ -1285,6 +1292,9 @@ This is how this would work with a Omnibus GitLab PostgreSQL setup: Praefect is the router and transaction manager for Gitaly Cluster and all connections to Gitaly go through it. This section details how to configure it. +NOTE: +Praefect must be deployed in an odd number of 3 nodes or higher. This is to ensure the nodes can take votes as part of a quorum. + Praefect requires several secret tokens to secure communications across the Cluster: - `<praefect_external_token>`: Used for repositories hosted on your Gitaly cluster and can only be accessed by Gitaly clients that carry this token. @@ -1459,9 +1469,7 @@ Due to Gitaly having notable input and output requirements, we strongly recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should have a throughput of at least 8,000 input/output operations per second (IOPS) for read operations and 2,000 IOPS for -write operations. These IOPS values are initial recommendations, and may be -adjusted to greater or lesser values depending on the scale of your -environment's workload. If you're running the environment on a Cloud provider, +write operations. If you're running the environment on a Cloud provider, refer to their documentation about how to configure IOPS correctly. Gitaly servers must not be exposed to the public internet, as Gitaly's network @@ -2068,11 +2076,9 @@ the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). gitlab-rake gitlab:db:configure ``` - If you encounter a `rake aborted!` error message stating that PgBouncer is - failing to connect to PostgreSQL, it may be that your PgBouncer node's IP - address is missing from PostgreSQL's `trust_auth_cidr_addresses` in `gitlab.rb` - on your database nodes. Before proceeding, see - [PgBouncer error `ERROR: pgbouncer cannot connect to server`](../postgresql/replication_and_failover.md#pgbouncer-error-error-pgbouncer-cannot-connect-to-server). + Note that this requires the Rails node to be configured to connect to the primary database + directly, [bypassing PgBouncer](../postgresql/pgbouncer.md#procedure-for-bypassing-pgbouncer). + After migrations have completed, you must configure the node to pass through PgBouncer again. 1. [Configure fast lookup of authorized SSH keys in the database](../operations/fast_ssh_key_lookup.md). @@ -2175,15 +2181,15 @@ and scalable. There are two ways of specifying object storage configuration in GitLab: -- [Consolidated form](../object_storage.md#consolidated-object-storage-configuration): A single credential is +- [Consolidated form](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form): A single credential is shared by all supported object types. -- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its +- [Storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). The consolidated form is used in the following examples when available. NOTE: -When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +When using the [storage-specific form](../object_storage.md#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, which was deprecated in 14.9, requires shared storage such as NFS. diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index e4d07558306..f40c8fc3c67 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -47,9 +47,9 @@ The following Cloud Native Hybrid reference architectures, where select recommen The first choice to consider is whether a Self Managed approach is correct for you and your requirements. -Running any application in production is complex, and the same applies for GitLab. While we aim to make this as smooth as possible, there are still the general complexities. This depends on the design chosen, but typically you'll need to manage all aspects such as hardware, operating systems, networking, storage, security, GitLab itself, and more. +Running any application in production is complex, and the same applies for GitLab. While we aim to make this as smooth as possible, there are still the general complexities. This depends on the design chosen, but typically you'll need to manage all aspects such as hardware, operating systems, networking, storage, security, GitLab itself, and more. This includes both the initial setup of the environment and the longer term maintenance. -As such, it's recommended that you have a working knowledge of running applications in production when deciding on going down this route. For users who want a more managed solution it's recommended to instead explore our other offerings such as [GitLab SaaS](../../subscriptions/gitlab_com/index.md) or [GitLab Dedicated](../../subscriptions/gitlab_dedicated/index.md). +As such, it's recommended that you have a working knowledge of running and maintaining applications in production when deciding on going down this route. If you aren't in this position, our [Professional Services](https://about.gitlab.com/services/#implementation-services) team offers implementation services, but for those who want a more managed solution long term, it's recommended to instead explore our other offerings such as [GitLab SaaS](../../subscriptions/gitlab_com/index.md) or [GitLab Dedicated](../../subscriptions/gitlab_dedicated/index.md). ## Deciding which architecture to use @@ -72,15 +72,13 @@ With standalone setups, especially single node environments, there are [various ### High Availability (HA) -High Availability ensures every component in the GitLab setup can handle failures through various mechanisms. To achieve this however is complex, and the environments required can be sizable. +High Availability ensures every component in the GitLab setup can handle failures through various mechanisms. However, to achieve this is complex and the environments required can be sizable. For environments serving 3,000 or more users we generally recommend that a HA strategy is used as at this level outages have a bigger impact against more users. All the architectures in this range have HA built in by design for this reason. -For users who still need to have HA for a lower number of users this can also be achieved with an adjusted [3K architecture](3k_users.md#supported-modifications-for-lower-user-counts-ha). - #### Do you need High Availability (HA)? -As mentioned above, achieving HA does come at a cost. The environment's required are sizable as each component needs to be multiplied, which comes with additional actual and maintenance costs. +As mentioned above, achieving HA does come at a cost. The environment requirements are sizable as each component needs to be multiplied, which comes with additional actual and maintenance costs. For a lot of our customers with fewer than 3,000 users, we've found a backup strategy is sufficient and even preferable. While this does have a slower recovery time, it also means you have a much smaller architecture and less maintenance costs as a result. @@ -89,13 +87,17 @@ In general then, we'd only recommend you employ HA in the following scenarios: - When you have 3,000 or more users. - When GitLab being down would critically impact your workflow. +#### Scaled-down High Availability (HA) approaches + +If you still need to have HA for a lower number of users, this can be achieved with an adjusted [3K architecture](3k_users.md#supported-modifications-for-lower-user-counts-ha). + #### Zero Downtime Upgrades [Zero Downtime Upgrades](../../update/zero_downtime.md) are available for standard Reference Architecture environments with HA (Cloud Native Hybrid is not supported at this time). This allows for an environment to stay up during an upgrade, but the process is more complex as a result and has some limitations as detailed in the documentation. -When going through this process it's worth noting that there may still be brief moments of downtime when the HA mechanisms tale effect. +When going through this process it's worth noting that there may still be brief moments of downtime when the HA mechanisms take effect. -In most cases the downtime required for doing an upgrade in general shouldn't be substantial, so this is only recommended if it's a key requirement for you. +In most cases the downtime required for doing an upgrade shouldn't be substantial, so this is only recommended if it's a key requirement for you. ### Cloud Native Hybrid (Kubernetes HA) @@ -163,7 +165,7 @@ Before implementing a reference architecture, refer to the following requirement These reference architectures were built and tested on Google Cloud Platform (GCP) using the [Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms) -CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)). +CPU platform as a lowest common denominator baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)). Newer, similarly-sized CPUs are supported and may have improved performance as a result. For Omnibus GitLab environments, ARM-based equivalents are also supported. @@ -237,8 +239,8 @@ for more information and guidance. [Praefect requires its own database server](../gitaly/praefect.md#postgresql) and that to achieve full High Availability, a third-party PostgreSQL database solution is required. -We hope to offer a built in solutions for these restrictions in the future but, in the meantime, a non HA PostgreSQL server -can be set up using Omnibus GitLab, the specifications reflect. Refer to the following issues for more information: +We hope to offer a built-in solution for these restrictions in the future. In the meantime, a non-HA PostgreSQL server +can be set up using Omnibus GitLab as the specifications reflect. Refer to the following issues for more information: - [`omnibus-gitlab#5919`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5919). - [`gitaly#3398`](https://gitlab.com/gitlab-org/gitaly/-/issues/3398). @@ -324,10 +326,12 @@ When selecting a database service, it should run a standard, performant, and [su - Read Replicas for [Database Load Balancing](../postgresql/database_load_balancing.md). - Cross Region replication for [GitLab Geo](../geo/index.md). -Several cloud provider services are known not to support the above or have been found to have other issues and aren't recommended: +#### Unsupported database services + +Several database cloud provider services are known not to support the above or have been found to have other issues and aren't recommended: - [Amazon Aurora](https://aws.amazon.com/rds/aurora/) is incompatible and not supported. See [14.4.0](../../update/index.md#1440) for more details. -- [Azure Database for PostgreSQL Single Server](https://azure.microsoft.com/en-gb/products/postgresql/#overview) (Single / Flexible) is **strongly not recommended** for use due to notable performance / stability issues or missing functionality. See [Recommendation Notes for Azure](#recommendation-notes-for-azure) for more details. +- [Azure Database for PostgreSQL Single Server](https://azure.microsoft.com/en-gb/products/postgresql/#overview) (Single / Flexible) is not supported for use due to notable performance / stability issues or missing functionality. See [Recommendation Notes for Azure](#recommendation-notes-for-azure) for more details. - [Google AlloyDB](https://cloud.google.com/alloydb) and [Amazon RDS Multi-AZ DB cluster](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) have not been tested and are not recommended. Both solutions are specifically not expected to work with GitLab Geo. - [Amazon RDS Multi-AZ DB instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZSingleStandby.html) is a separate product and is supported. @@ -337,10 +341,55 @@ Due to performance issues that we found with several key Azure services, we only In addition to the above, you should be aware of the additional specific guidance for Azure: -- **We outright strongly do not recommend [Azure Database for PostgreSQL Single Server](https://learn.microsoft.com/en-us/azure/postgresql/single-server/overview-single-server)** specifically due to significant performance and stability issues found. **For GitLab 14.0 and higher the service is not supported** due to it only supporting up to PostgreSQL 11. - - A new service, [Azure Database for PostgreSQL Flexible Server](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/) has been released. [Internal testing](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/91) has shown that it does look to perform as expected, but this hasn't been validated in production, so generally isn't recommended at this time. Additionally, as it's a new service, you may find that it's missing some functionality depending on your requirements. +- [Azure Database for PostgreSQL Single Server](https://azure.microsoft.com/en-gb/products/postgresql/#overview) (Single / Flexible) is not supported for use due to notable performance / stability issues or missing functionality. +- A new service, [Azure Database for PostgreSQL Flexible Server](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/) has been released. [Internal testing](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/91) has shown that it does look to perform as expected, but this hasn't been validated in production, so generally isn't recommended at this time. Additionally, as it's a new service, you may find that it's missing some functionality depending on your requirements. - [Azure Blob Storage](https://azure.microsoft.com/en-gb/products/storage/blobs/) has been found to have [performance limits that can impact production use at certain times](https://gitlab.com/gitlab-org/gitlab/-/issues/344861). However, this has only been seen in our largest architectures (25k+) so far. +## Deviating from the suggested reference architectures + +As a general guideline, the further away you move from the reference architectures, +the harder it is to get support for it. With any deviation, you're introducing +a layer of complexity that adds challenges to finding out where potential +issues might lie. + +The reference architectures use the official GitLab Linux packages (Omnibus +GitLab) or [Helm Charts](https://docs.gitlab.com/charts/) to install and configure the various components. The components are +installed on separate machines (virtualized or bare metal), with machine hardware +requirements listed in the "Configuration" column and equivalent VM standard sizes listed +in GCP/AWS/Azure columns of each [available reference architecture](#available-reference-architectures). + +Running components on Docker (including Docker Compose) with the same specs should be fine, as Docker is well known in terms of support. +However, it is still an additional layer and may still add some support complexities, such as not being able to run `strace` easily in containers. + +### Unsupported designs + +While we endeavour to try and have a good range of support for GitLab environment designs, there are certain approaches we know definitively not to work, and as a result are not supported. Those approaches are detailed in the following sections. + +#### Stateful components in Kubernetes + +[Running stateful components in Kubernetes, such as Gitaly Cluster, is not supported](https://docs.gitlab.com/charts/installation/#configure-the-helm-chart-to-use-external-stateful-data). + +Gitaly Cluster is only supported to be run on VMs as Git itself doesn't match well with the Kubernetes design and attempting to run it can lead to significant and complex issues. +[Refer to epic 6127 for more information](https://gitlab.com/groups/gitlab-org/-/epics/6127). + +This also applies to other third-party stateful components such as Postgres and Redis, but you can explore other third-party solutions for those components if desired such as supported Cloud Provider services unless called out specifically as unsupported. + +#### Autoscaling of stateful nodes + +As a general guidance, only _stateless_ components of GitLab can be run in Autoscaling groups, namely GitLab Rails +and Sidekiq. + +Other components that have state, such as Gitaly, are not supported in this fashion (for more information, see [issue 2997](https://gitlab.com/gitlab-org/gitaly/-/issues/2997)). + +This also applies to other third-party stateful components such as Postgres and Redis, but you can explore other third-party solutions for those components if desired such as supported Cloud Provider services unless called out specifically as unsupported. + +#### Spreading one environment over multiple data centers + +Deploying one GitLab environment over multiple data centers is not supported due to potential split brain edge cases +if a data center were to go down. For example, several components of the GitLab setup, namely Consul, Redis Sentinel and Praefect require an odd number quorum to function correctly and splitting over multiple data centers can impact this notably. + +For deploying GitLab over multiple data centers or regions we offer [GitLab Geo](https://about.gitlab.com/solutions/geo/) as a comprehensive solution. + ## Validation and test results The [Quality Engineering team](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/) @@ -427,34 +476,34 @@ table.test-coverage th { <tr> <th scope="row">1k</th> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k">Weekly</a></td> - <td></td> - <td></td> - <td></td> - <td></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> </tr> <tr> <th scope="row">2k</th> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k">Weekly</a></td> - <td></td> - <td></td> - <td></td> - <td></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td><i>Planned</i></td> </tr> <tr> <th scope="row">3k</th> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k">Weekly</a></td> - <td></td> - <td></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k_hybrid_aws_services">Weekly</a></td> - <td></td> + <td style="background-color:lightgrey"></td> </tr> <tr> <th scope="row">5k</th> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k">Weekly</a></td> - <td></td> - <td></td> - <td></td> - <td></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> </tr> <tr> <th scope="row">10k</th> @@ -462,29 +511,33 @@ table.test-coverage th { <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k_hybrid">Weekly</a></td> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k_aws">Weekly</a></td> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k_hybrid_aws_services">Weekly</a></td> - <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k">Ad-Hoc</a></td> + <td style="background-color:lightgrey"></td> </tr> <tr> <th scope="row">25k</th> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k">Weekly</a></td> - <td></td> - <td></td> - <td></td> - <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k">Ad-Hoc</a></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> </tr> <tr> <th scope="row">50k</th> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k">Weekly</a></td> - <td></td> - <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k">Ad-Hoc (inc Cloud Services)</a></td> - <td></td> - <td></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> + <td style="background-color:lightgrey"></td> </tr> </table> ## Cost to run -The following table details the cost to run the different reference architectures across GCP, AWS, and Azure. Bare-metal costs are not included here as it varies widely depending on each customer configuration. +As a starting point, the following table details initial costs for the different reference architectures across GCP, AWS, and Azure via Omnibus. + +NOTE: +Due to the nature of Cloud Native Hybrid, it's not possible to give a static cost calculation. +Bare-metal costs are also not included here as it varies widely depending on each configuration. <table class="test-coverage"> <col> @@ -492,91 +545,93 @@ The following table details the cost to run the different reference architecture <colgroup span="2"></colgroup> <tr> <th rowspan="2">Reference<br/>Architecture</th> - <th style="text-align: center" colspan="2" scope="colgroup">GCP</th> - <th style="text-align: center" colspan="2" scope="colgroup">AWS</th> - <th style="text-align: center" colspan="2" scope="colgroup">Azure</th> + <th style="text-align: center" scope="colgroup">GCP</th> + <th style="text-align: center" scope="colgroup">AWS</th> + <th style="text-align: center" scope="colgroup">Azure</th> </tr> <tr> <th scope="col">Omnibus</th> - <th scope="col">Cloud Native Hybrid</th> <th scope="col">Omnibus</th> - <th scope="col">Cloud Native Hybrid</th> <th scope="col">Omnibus</th> </tr> <tr> <th scope="row">1k</th> <td><a href="https://cloud.google.com/products/calculator#id=a6d6a94a-c7dc-4c22-85c4-7c5747f272ed">Calculated cost</a></td> - <td></td> <td><a href="https://calculator.aws/#/estimate?id=b51f178f4403b69a63f6eb33ea425f82de3bf249">Calculated cost</a></td> - <td></td> <td><a href="https://azure.microsoft.com/en-us/pricing/calculator/?shared-estimate=1adf30bef7e34ceba9efa97c4470417b">Calculated cost</a></td> </tr> <tr> <th scope="row">2k</th> <td><a href="https://cloud.google.com/products/calculator#id=0d3aff1f-ea3d-43f9-aa59-df49d27c35ca">Calculated cost</a></td> - <td></td> <td><a href="https://calculator.aws/#/estimate?id=3b3e3b81953737132789591d3a5179521943f1c0">Calculated cost</a></td> - <td></td> <td><a href="https://azure.microsoft.com/en-us/pricing/calculator/?shared-estimate=25f66c35ba454bb98fb4034a8a50bb8c">Calculated cost</a></td> </tr> <tr> <th scope="row">3k</th> <td><a href="https://cloud.google.com/products/calculator/#id=15fc2bd9-5b1c-479d-bc46-d5ce096b8107">Calculated cost</a></td> - <td></td> <td><a href="https://calculator.aws/#/estimate?id=7e94eb8712f6845fdeb05e61f459598a91dac3cb">Calculated cost</a></td> - <td></td> <td><a href="https://azure.microsoft.com/en-us/pricing/calculator/?shared-estimate=24ac11fd947a4985ae9c9a5142649ad3">Calculated cost</a></td> </tr> <tr> <th scope="row">5k</th> <td><a href="https://cloud.google.com/products/calculator/#id=9a798136-53f2-4c35-be43-8e1e975a6663">Calculated cost</a></td> - <td></td> <td><a href="https://calculator.aws/#/estimate?id=ad4c9db623a214c92d780cd9dff33f444d62cf02">Calculated cost</a></td> - <td></td> <td><a href="https://azure.microsoft.com/en-us/pricing/calculator/?shared-estimate=bcf23017ddfd40649fdc885cacd08d0c">Calculated cost</a></td> </tr> <tr> <th scope="row">10k</th> <td><a href="https://cloud.google.com/products/calculator#id=cbe61840-31a1-487f-88fa-631251c2fde5">Calculated cost</a></td> - <td></td> <td><a href="https://calculator.aws/#/estimate?id=3e2970f919915a6337acea76a9f07655a1ecda4a">Calculated cost</a></td> - <td></td> <td><a href="https://azure.microsoft.com/en-us/pricing/calculator/?shared-estimate=5748068be4864af6a34efb1cde685fa1">Calculated cost</a></td> </tr> <tr> <th scope="row">25k</th> <td><a href="https://cloud.google.com/products/calculator#id=b4b8b587-508a-4433-adc8-dc506bbe924f">Calculated cost</a></td> - <td></td> <td><a href="https://calculator.aws/#/estimate?id=32acaeaa93366110cd5fbf98a66a8a141db7adcb">Calculated cost</a></td> - <td></td> <td><a href="https://azure.microsoft.com/en-us/pricing/calculator/?shared-estimate=24f878f20ee64b5cb64de459d34c8128">Calculate cost</a></td> </tr> <tr> <th scope="row">50k</th> <td><a href="https://cloud.google.com/products/calculator/#id=48b4d817-d6cd-44b8-b069-0ba9a5d123ea">Calculated cost</a></td> - <td></td> <td><a href="https://calculator.aws/#/estimate?id=5a0bba1338e3577d627ec97833dbc80ac9615562">Calculated cost</a></td> - <td></td> <td><a href="https://azure.microsoft.com/en-us/pricing/calculator/?shared-estimate=4dd065eea2194d70b44d6d897e81f460">Calculated cost</a></td> </tr> </table> -## Deviating from the suggested reference architectures +## Maintaining a Reference Architecture environment -As a general guideline, the further away you move from the Reference Architectures, -the harder it is to get support for it. With any deviation, you're introducing -a layer of complexity that adds challenges to finding out where potential -issues might lie. +Maintaining a Reference Architecture environment is generally the same as any other GitLab environment is generally covered in other sections of this documentation. -The reference architectures use the official GitLab Linux packages (Omnibus -GitLab) or [Helm Charts](https://docs.gitlab.com/charts/) to install and configure the various components. The components are -installed on separate machines (virtualized or bare metal), with machine hardware -requirements listed in the "Configuration" column and equivalent VM standard sizes listed -in GCP/AWS/Azure columns of each [available reference architecture](#available-reference-architectures). +In this section you'll find links to documentation for relevant areas as well as any specific Reference Architecture notes. -Running components on Docker (including Compose) with the same specs should be fine, as Docker is well known in terms of support. -However, it is still an additional layer and may still add some support complexities, such as not being able to run `strace` easily in containers. +### Upgrades + +Upgrades for a Reference Architecture environment is the same as any other GitLab environment. +The main [Upgrade GitLab](../../update/index.md) section has detailed steps on how to approach this. + +[Zero-downtime upgrades](#zero-downtime-upgrades) are also available. + +NOTE: +You should upgrade a Reference Architecture in the same order as you created it. + +### Scaling an environment + +Scaling a GitLab environment is designed to be as seamless as possible. + +In terms of the Reference Architectures, you would look to the next size and adjust accordingly. +Most setups would only need vertical scaling, but there are some specific areas that can be adjusted depending on the setup: + +- If you're scaling from a non-HA environment to an HA environment, various components are recommended to be deployed in their HA forms: + - Redis to multi-node Redis w/ Redis Sentinel + - Postgres to multi-node Postgres w/ Consul + PgBouncer + - Gitaly to Gitaly Cluster w/ Praefect +- From 10k users and higher, Redis is recommended to be split into multiple HA servers as it's single threaded. + +Conversely, if you have robust metrics in place that show the environment is over-provisioned you can apply the same process for +scaling downloads. It's recommended to take an iterative approach when scaling downwards however to ensure there are no issues. + +### How to monitor your environment -Other technologies, like [Docker swarm](https://docs.docker.com/engine/swarm/) -are not officially supported, but can be implemented at your own risk. In that -case, GitLab Support is not able to help you. +To monitor your GitLab environment, you can use the tools +[bundled with GitLab](../monitoring/index.md), but it's also possible to use third-party +options if desired. diff --git a/doc/administration/restart_gitlab.md b/doc/administration/restart_gitlab.md index 7996db3d1e1..e86541b7ced 100644 --- a/doc/administration/restart_gitlab.md +++ b/doc/administration/restart_gitlab.md @@ -126,3 +126,17 @@ kubectl delete pods -l release=<helm release name>,app=<component name> ``` The release name can be obtained from the output of the `helm list` command. + +## Docker installation + +If you change the configuration on your [Docker installation](../install/docker.md), for that change to take effect you must restart: + +- The main `gitlab` container. +- Any separate component containers. + +For example, if you deployed Sidekiq on a separate container, to restart the containers, run: + +```shell +sudo docker restart gitlab +sudo docker restart sidekiq +``` diff --git a/doc/administration/server_hooks.md b/doc/administration/server_hooks.md index 87ae6fdb003..5e9863447a1 100644 --- a/doc/administration/server_hooks.md +++ b/doc/administration/server_hooks.md @@ -2,7 +2,6 @@ stage: Systems group: Gitaly info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/administration/custom_hooks.html' --- # Git server hooks **(FREE SELF)** @@ -28,7 +27,43 @@ alternatives to server hooks include: [Geo](geo/index.md) doesn't replicate server hooks to secondary nodes. -## Create server hooks for a repository +## Set server hooks for a repository + +::Tabs + +:::TabTitle GitLab 15.11 and later + +> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/4629) in GitLab 15.11, `hooks set` command replaces direct file system access. + +Prerequisites: + +- The [storage name and relative path](repository_storage_types.md#from-project-name-to-hashed-path) for the repository. + +To set server hooks for a repository: + +1. Create tarball containing custom hooks: + 1. Write the code to make the server hook function as expected. Git server hooks can be in any programming language. + Ensure the [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) at the top reflects the language type. For + example, if the script is in Ruby the shebang is probably `#!/usr/bin/env ruby`. + + - To create a single server hook, create a file with a name that matches the hook type. For example, for a + `pre-receive` server hook, the filename should be `pre-receive` with no extension. + - To create many server hooks, create a directory for the hooks that matches the hook type. For example, for a + `pre-receive` server hook, the directory name should be `pre-receive.d`. Put the files for the hook in that + directory. + + 1. Ensure the server hook files are executable and do not match the backup file pattern (`*~`). The server hooks be + in a `custom_hooks` directory that is at the root of the tarball. + 1. Create the custom hooks archive with the tar command. For example, `tar -cf custom_hooks.tar custom_hooks`. +1. Run the `hooks set` command with required options to set the Git hooks for the repository. For example, + `cat hooks.tar | gitaly hooks set --storage <storage> --repository <relative path> --config <config path>`. + + - A path to a valid Gitaly configuration for the node is required to connect to the node and provided to the `--config` flag. + - Custom hooks tarball must be passed via `stdin`. For example, `cat hooks.tar | gitaly hooks set --storage <storage> --repository <relative path> --config <config path>`. + +If you implemented the server hook code correctly, it should execute when the Git hook is next triggered. + +:::TabTitle GitLab 15.10 and earlier To create server hooks for a repository: @@ -56,6 +91,8 @@ To create server hooks for a repository: If the server hook code is properly implemented, it should execute when the Git hook is next triggered. +::EndTabs + ### Gitaly Cluster If you use [Gitaly Cluster](gitaly/index.md), the scripts must be copied to every Gitaly node that has a replica of the repository. Every Gitaly node @@ -113,6 +150,33 @@ To create a global server hook for all repositories: If the server hook code is properly implemented, it should execute when the Git hook is next triggered. Hooks are executed in alphabetical order by filename in the hook type subdirectories. +## Remove server hooks for a repository + +::Tabs + +:::TabTitle GitLab 15.11 and later + +> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/4629) in GitLab 15.11, `hooks set` command replaces direct file system access. + +Prerequisites: + +- The [storage name and relative path](repository_storage_types.md#from-project-name-to-hashed-path) for the repository. + +To remove server hooks, pass an empty tarball to `hook set` to indicate that the repository should contain no hooks. For example: + +```shell +cat empty_hooks.tar | gitaly hooks set --storage <storage> --repository <relative path> --config <config path>`. +``` + +:::TabTitle GitLab 15.10 and earlier + +To remove server hooks: + +1. Go to the location of the repository on disk. +1. Delete the server hooks in the `custom_hooks` directory. + +::EndTabs + ## Chained server hooks GitLab can execute server hooks in a chain. GitLab searches for and executes server hooks in the following order: diff --git a/doc/administration/sidekiq/index.md b/doc/administration/sidekiq/index.md index 8645df55a62..6fb12aa6ef9 100644 --- a/doc/administration/sidekiq/index.md +++ b/doc/administration/sidekiq/index.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can configure an external Sidekiq instance by using the Sidekiq that's bundled in the GitLab package. Sidekiq requires connection to the Redis, PostgreSQL, and Gitaly instances. -## Configure TCP access for PostgreSQL, Gitaly, and Redis +## Configure TCP access for PostgreSQL, Gitaly, and Redis on the GitLab instance By default, GitLab uses UNIX sockets and is not set up to communicate via TCP. To change this: @@ -56,7 +56,6 @@ By default, GitLab uses UNIX sockets and is not set up to communicate via TCP. T redis['password'] = 'redis-password-goes-here' gitlab_rails['redis_password'] = 'redis-password-goes-here' - gitlab_rails['auto_migrate'] = false ``` 1. Run `reconfigure`: @@ -71,18 +70,6 @@ By default, GitLab uses UNIX sockets and is not set up to communicate via TCP. T sudo gitlab-ctl restart postgresql ``` -1. After the restart, set `auto_migrate` to `true` or comment to use the default settings: - - ```ruby - gitlab_rails['auto_migrate'] = true - ``` - -1. Run `reconfigure` again: - - ```shell - sudo gitlab-ctl reconfigure - ``` - ## Set up Sidekiq instance 1. SSH into the Sidekiq server. diff --git a/doc/administration/sidekiq/processing_specific_job_classes.md b/doc/administration/sidekiq/processing_specific_job_classes.md index 0e4a0662277..26c2804f130 100644 --- a/doc/administration/sidekiq/processing_specific_job_classes.md +++ b/doc/administration/sidekiq/processing_specific_job_classes.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: These are advanced settings. While they are used on GitLab.com, most GitLab -instances should add more processes that all listen to all queues. This is the +instances should only add more processes that listen to all queues. This is the same approach we take in our [Reference Architectures](../reference_architectures/index.md). GitLab has two options for creating Sidekiq processes that only handle specific diff --git a/doc/administration/silent_mode/index.md b/doc/administration/silent_mode/index.md new file mode 100644 index 00000000000..f07f1dde323 --- /dev/null +++ b/doc/administration/silent_mode/index.md @@ -0,0 +1,64 @@ +--- +stage: Systems +group: Geo +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# GitLab Silent Mode (Alpha) **(FREE SELF)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/9826) in GitLab 15.11. This feature is an [Experiment](../../policy/alpha-beta-support.md#experiment). + +Silent Mode allows you to suppress outbound communication, such as emails, from GitLab. Silent Mode is not intended to be used on environments which are in-use. Two use-cases are: + +- Validating Geo site promotion. You have a secondary Geo site as part of your [disaster recovery](../geo/disaster_recovery/index.md) solution. You want to regularly test promoting it to become a primary Geo site, as a best practice to ensure your disaster recovery plan actually works. But you don't want to actually perform an entire failover, since the primary site lives in a region which provides the lowest latency to your users. And you don't want to take downtime during every regular test. So, you let the primary site remain up, while you promote the secondary site. You start smoke testing the promoted site. But, the promoted site starts emailing users, the push mirrors push changes to external Git repositories, etc. This is where Silent Mode comes in. You can enable it as part of site promotion, to avoid this issue. +- Validating GitLab backups. You set up a testing instance to test that your backups restore successfully. As part of the restore, you enable Silent Mode, for example to avoid sending invalid emails to users. + +## Enable Silent Mode + +Prerequisites: + +- You must have administrator access. + +There are two ways to enable Silent Mode: + +- [**API**](../../api/settings.md): + + ```shell + curl --request PUT --header "PRIVATE-TOKEN:$ADMIN_TOKEN" "<gitlab-url>/api/v4/application/settings?silent_mode_enabled=true" + ``` + +- [**Rails console**](../operations/rails_console.md#starting-a-rails-console-session): + + ```ruby + ::Gitlab::CurrentSettings.update!(silent_mode_enabled: true) + ``` + +## Disable Silent Mode + +Prerequisites: + +- You must have administrator access. + +There are two ways to disable Silent Mode: + +- [**API**](../../api/settings.md): + + ```shell + curl --request PUT --header "PRIVATE-TOKEN:$ADMIN_TOKEN" "<gitlab-url>/api/v4/application/settings?silent_mode_enabled=false" + ``` + +- [**Rails console**](../operations/rails_console.md#starting-a-rails-console-session): + + ```ruby + ::Gitlab::CurrentSettings.update!(silent_mode_enabled: false) + ``` + +## Behavior of GitLab features in Silent Mode + +### Service Desk + +Incoming emails still raise issues, but the users who sent the emails to [Service Desk](../../user/project/service_desk.md) are not notified of issue creation or comments on their issues. + +### Outbound emails + +Outbound emails are suppressed. It may take up to a minute to take effect after enabling Silent Mode. [Issue 405433](https://gitlab.com/gitlab-org/gitlab/-/issues/405433) proposes removing this delay. diff --git a/doc/administration/static_objects_external_storage.md b/doc/administration/static_objects_external_storage.md index ef89e38be6f..73f44ed3889 100644 --- a/doc/administration/static_objects_external_storage.md +++ b/doc/administration/static_objects_external_storage.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" type: reference --- diff --git a/doc/administration/terraform_state.md b/doc/administration/terraform_state.md index 12d88c7c74a..9a54eb86eeb 100644 --- a/doc/administration/terraform_state.md +++ b/doc/administration/terraform_state.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -175,7 +175,7 @@ sudo find /var/opt/gitlab/gitlab-rails/shared/terraform_state -type f | grep -v ### S3-compatible connection settings In GitLab 13.2 and later, you should use the -[consolidated object storage settings](object_storage.md#consolidated-object-storage-configuration). +[consolidated object storage settings](object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). This section describes the earlier configuration format. See [the available connection settings for different providers](object_storage.md#connection-settings). diff --git a/doc/administration/troubleshooting/index.md b/doc/administration/troubleshooting/index.md index db572f202ea..8be6abc180d 100644 --- a/doc/administration/troubleshooting/index.md +++ b/doc/administration/troubleshooting/index.md @@ -27,3 +27,28 @@ that you can check for feature-specific help, including helpful Rails commands. If you need a testing environment to troubleshoot, see the [apps for a testing environment](test_environments.md). + +## Support team troubleshooting info + +The GitLab Support Team has collected a lot of information about troubleshooting GitLab. +The following documents are used by the Support Team or by customers +with direct guidance from a Support Team member. GitLab administrators may find the +information useful for troubleshooting. However, if you are experiencing trouble with your +GitLab instance, you should check your [support options](https://about.gitlab.com/support/) +before referring to these documents. + +WARNING: +The commands in the following documentation might result in data loss or +other damage to a GitLab instance. They should be used only by experienced administrators +who are aware of the risks. + +- [Diagnostics tools](diagnostics_tools.md) +- [Linux commands](linux_cheat_sheet.md) +- [Troubleshooting Kubernetes](https://docs.gitlab.com/charts/troubleshooting/kubernetes_cheat_sheet.html) +- [Troubleshooting PostgreSQL](postgresql.md) +- [Guide to test environments](test_environments.md) (for Support Engineers) +- [Troubleshooting SSL](ssl.md) +- Related links: + - [Repairing and recovering broken Git repositories](https://git.seveas.net/repairing-and-recovering-broken-git-repositories.html) + - [Testing with OpenSSL](https://www.feistyduck.com/library/openssl-cookbook/online/testing-with-openssl/index.html) + - [`strace` zine](https://wizardzines.com/zines/strace/) diff --git a/doc/administration/uploads.md b/doc/administration/uploads.md index c4e2c352e9b..e614f0d38e3 100644 --- a/doc/administration/uploads.md +++ b/doc/administration/uploads.md @@ -65,7 +65,7 @@ This configuration relies on valid AWS credentials to be configured already. ### Object Storage Settings In GitLab 13.2 and later, you should use the -[consolidated object storage settings](object_storage.md#consolidated-object-storage-configuration). +[consolidated object storage settings](object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). This section describes the earlier configuration format. For source installations the following settings are nested under `uploads:` and then `object_store:`. On Omnibus GitLab installs they are prefixed by `uploads_object_store_`. diff --git a/doc/administration/wikis/index.md b/doc/administration/wikis/index.md index f3442e23160..330e41ee880 100644 --- a/doc/administration/wikis/index.md +++ b/doc/administration/wikis/index.md @@ -1,7 +1,7 @@ --- type: reference, howto -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/api_resources.md b/doc/api/api_resources.md index b7c1def0ba4..7304c0a86be 100644 --- a/doc/api/api_resources.md +++ b/doc/api/api_resources.md @@ -44,8 +44,8 @@ The following API resources are available in the project context: | [Environments](environments.md) | `/projects/:id/environments` | | [Error Tracking](error_tracking.md) | `/projects/:id/error_tracking/settings` | | [Events](events.md) | `/projects/:id/events` (also available for users and standalone) | -| [Feature Flag User Lists](feature_flag_user_lists.md) | `/projects/:id/feature_flags_user_lists` | -| [Feature Flags](feature_flags.md) | `/projects/:id/feature_flags` | +| [Feature flag User Lists](feature_flag_user_lists.md) | `/projects/:id/feature_flags_user_lists` | +| [Feature flags](feature_flags.md) | `/projects/:id/feature_flags` | | [Freeze Periods](freeze_periods.md) | `/projects/:id/freeze_periods` | | [Go Proxy](packages/go_proxy.md) | `/projects/:id/packages/go` | | [Helm repository](packages/helm.md) | `/projects/:id/packages/helm_repository` | @@ -163,7 +163,8 @@ The following API resources are available outside of project and group contexts | [Geo Nodes](geo_nodes.md) **(PREMIUM SELF)** | `/geo_nodes` | | [Group Activity Analytics](group_activity_analytics.md) | `/analytics/group_activity/{issues_count}` | | [Group repository storage moves](group_repository_storage_moves.md) **(PREMIUM SELF)** | `/group_repository_storage_moves` | -| [Import repository from GitHub](import.md) | `/import/github` | +| [Import repository from GitHub](import.md#import-repository-from-github) | `/import/github` | +| [Import repository from Bitbucket Server](import.md#import-repository-from-bitbucket-server) | `/import/bitbucket_server` | | [Instance clusters](instance_clusters.md) **(FREE SELF)** | `/admin/clusters` | | [Instance-level CI/CD variables](instance_level_ci_variables.md) **(FREE SELF)** | `/admin/ci/variables` | | [Issues Statistics](issues_statistics.md) | `/issues_statistics` (also available for groups and projects) | diff --git a/doc/api/audit_events.md b/doc/api/audit_events.md index f42ace8c1de..c4b3d99c742 100644 --- a/doc/api/audit_events.md +++ b/doc/api/audit_events.md @@ -8,6 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/121) in GitLab 12.4. > - [Author Email added to the response body](https://gitlab.com/gitlab-org/gitlab/-/issues/386322) in GitLab 15.9. +> - Support for keyset pagination [added](https://gitlab.com/gitlab-org/gitlab/-/issues/367528) in GitLab 15.11. ## Instance Audit Events **(PREMIUM SELF)** @@ -29,8 +30,8 @@ GET /audit_events | `entity_type` | string | no | Return audit events for the given entity type. Valid values are: `User`, `Group`, or `Project`. | | `entity_id` | integer | no | Return audit events for the given entity ID. Requires `entity_type` attribute to be present. | -By default, `GET` requests return 20 results at a time because the API results -are paginated. +This endpoint supports both offset-based and [keyset-based](rest/index.md#keyset-based-pagination) pagination. You should use keyset-based +pagination when requesting consecutive pages of results. Read more on [pagination](rest/index.md#pagination). @@ -137,7 +138,7 @@ Example response: ## Group Audit Events > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34078) in GitLab 12.5. -> - [Support for keyset pagination added](https://gitlab.com/gitlab-org/gitlab/-/issues/333968) in GitLab 15.2. +> - Support for keyset pagination [added](https://gitlab.com/gitlab-org/gitlab/-/issues/333968) in GitLab 15.2. The Group Audit Events API allows you to retrieve [group audit events](../administration/audit_events.md#group-events). This API cannot retrieve project audit events. @@ -254,7 +255,7 @@ Example response: ## Project Audit Events > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/219238) in GitLab 13.1. -> - [Support for keyset pagination added](https://gitlab.com/gitlab-org/gitlab/-/issues/367528) in GitLab 15.10. +> - Support for keyset pagination [added](https://gitlab.com/gitlab-org/gitlab/-/issues/367528) in GitLab 15.10. The Project Audit Events API allows you to retrieve [project audit events](../administration/audit_events.md#project-events). diff --git a/doc/api/bulk_imports.md b/doc/api/bulk_imports.md index 6a33bd1bc95..2dd244cb18d 100644 --- a/doc/api/bulk_imports.md +++ b/doc/api/bulk_imports.md @@ -4,16 +4,28 @@ group: Import info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Group migration by direct transfer API **(FREE)** +# Group and project migration by direct transfer API **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64335) in GitLab 14.1. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64335) in GitLab 14.1. +> - Project migration [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390515) in GitLab 15.11. With the group migration by direct transfer API, you can start and view the progress of migrations initiated with [group migration by direct transfer](../user/group/import/index.md#migrate-groups-by-direct-transfer-recommended). -## Start a new group migration +## Prerequisites -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66353) in GitLab 14.2. +For information on prerequisites for group migration by direct transfer API, see +prerequisites for [migrating groups by direct transfer](../user/group/import/index.md#prerequisites). + +## Start a new group or project migration + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66353) in GitLab 14.2. +> - `project_entity` source type [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390515) in GitLab 15.11. + +Use this endpoint to start a new group or project migration. Specify: + +- `entities[group_entity]` to migrate a group. +- `entities[project_entity]` to migrate a project. ```plaintext POST /bulk_imports @@ -25,7 +37,7 @@ POST /bulk_imports | `configuration[url]` | String | yes | Source GitLab instance URL. | | `configuration[access_token]` | String | yes | Access token to the source GitLab instance. | | `entities` | Array | yes | List of entities to import. | -| `entities[source_type]` | String | yes | Source entity type (only `group_entity` is supported). | +| `entities[source_type]` | String | yes | Source entity type. Valid values are `group_entity` (GitLab 14.2 and later) and `project_entity` (GitLab 15.11 and later). | | `entities[source_full_path]` | String | yes | Source full path of the entity to import. | | `entities[destination_name]` | String | yes | Deprecated: Use :destination_slug instead. Destination slug for the entity. | | `entities[destination_slug]` | String | yes | Destination slug for the entity. | @@ -54,18 +66,18 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitla { "id": 1, "status": "created", "source_type": "gitlab", "created_at": "2021-06-18T09:45:55.358Z", "updated_at": "2021-06-18T09:46:27.003Z" } ``` -## List all group migrations +## List all group or project migrations ```plaintext GET /bulk_imports ``` -| Attribute | Type | Required | Description | -|:-----------|:--------|:---------|:--------------------------------------------------------------------------------------------| -| `per_page` | integer | no | Number of records to return per page. | -| `page` | integer | no | Page to retrieve. | -| `sort` | string | no | Return GitLab migration sorted in `asc` or `desc` order by creation date. Default is `desc` | -| `status` | string | no | Import status. | +| Attribute | Type | Required | Description | +|:-----------|:--------|:---------|:-----------------------------------------------------------------------------------| +| `per_page` | integer | no | Number of records to return per page. | +| `page` | integer | no | Page to retrieve. | +| `sort` | string | no | Return records sorted in `asc` or `desc` order by creation date. Default is `desc` | +| `status` | string | no | Import status. | The status can be one of the following: @@ -97,18 +109,18 @@ curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab ] ``` -## List all group migrations' entities +## List all group or project migrations' entities ```plaintext GET /bulk_imports/entities ``` -| Attribute | Type | Required | Description | -|:-----------|:--------|:---------|:-----------------------------------------------------------------------------------------------------| -| `per_page` | integer | no | Number of records to return per page. | -| `page` | integer | no | Page to retrieve. | -| `sort` | string | no | Return GitLab migration entities sorted in `asc` or `desc` order by creation date. Default is `desc` | -| `status` | string | no | Import status. | +| Attribute | Type | Required | Description | +|:-----------|:--------|:---------|:-----------------------------------------------------------------------------------| +| `per_page` | integer | no | Number of records to return per page. | +| `page` | integer | no | Page to retrieve. | +| `sort` | string | no | Return records sorted in `asc` or `desc` order by creation date. Default is `desc` | +| `status` | string | no | Import status. | The status can be one of the following: @@ -165,7 +177,7 @@ curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab ] ``` -## Get group migration details +## Get group or project migration details ```plaintext GET /bulk_imports/:id @@ -185,18 +197,18 @@ curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab } ``` -## List group migration entities +## List group or project migration entities ```plaintext GET /bulk_imports/:id/entities ``` -| Attribute | Type | Required | Description | -|:-----------|:--------|:---------|:--------------------------------------------------------------------------------------------| -| `per_page` | integer | no | Number of records to return per page. | -| `page` | integer | no | Page to retrieve. | -| `sort` | string | no | Return GitLab migration sorted in `asc` or `desc` order by creation date. Default is `desc` | -| `status` | string | no | Import status. | +| Attribute | Type | Required | Description | +|:-----------|:--------|:---------|:-----------------------------------------------------------------------------------| +| `per_page` | integer | no | Number of records to return per page. | +| `page` | integer | no | Page to retrieve. | +| `sort` | string | no | Return records sorted in `asc` or `desc` order by creation date. Default is `desc` | +| `status` | string | no | Import status. | The status can be one of the following: @@ -221,7 +233,7 @@ curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab ] ``` -## Get group migration entity details +## Get group or project migration entity details ```plaintext GET /bulk_imports/:id/entities/:entity_id diff --git a/doc/api/cluster_agents.md b/doc/api/cluster_agents.md index 2622d6782aa..69cbbb9b964 100644 --- a/doc/api/cluster_agents.md +++ b/doc/api/cluster_agents.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/dependencies.md b/doc/api/dependencies.md index b99d9e2d91d..5ae36926868 100644 --- a/doc/api/dependencies.md +++ b/doc/api/dependencies.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Dependencies API **(ULTIMATE)** WARNING: -This API is in an [Alpha](../policy/alpha-beta-support.md#alpha-features) stage and considered unstable. +This API is in an [Experiment](../policy/alpha-beta-support.md#experiment) and considered unstable. The response payload may be subject to change or breakage across GitLab releases. @@ -34,7 +34,7 @@ GET /projects/:id/dependencies?package_manager=yarn,bundler | Attribute | Type | Required | Description | | ------------- | -------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `package_manager` | string array | no | Returns dependencies belonging to specified package manager. Valid values: `bundler`, `composer`, `conan`, `go`, `gradle`, `maven`, `npm`, `nuget`, `pip`, `pipenv`, `yarn`, `sbt`, or `setuptools`. | +| `package_manager` | string array | no | Returns dependencies belonging to specified package manager. Valid values: `bundler`, `composer`, `conan`, `go`, `gradle`, `maven`, `npm`, `nuget`, `pip`, `pipenv`, `pnpm`, `yarn`, `sbt`, or `setuptools`. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/4/dependencies" diff --git a/doc/api/deploy_keys.md b/doc/api/deploy_keys.md index 9b02f30b7ee..7bf9fb0827a 100644 --- a/doc/api/deploy_keys.md +++ b/doc/api/deploy_keys.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -43,6 +43,7 @@ Example response: "fingerprint": "4a:9d:64:15:ed:3a:e6:07:6e:89:36:b3:3b:03:05:d9", "fingerprint_sha256": "SHA256:Jrs3LD1Ji30xNLtTVf9NDCj7kkBgPBb2pjvTZ3HfIgU", "created_at": "2013-10-02T10:12:29Z", + "expires_at": null, "projects_with_write_access": [ { "id": 73, @@ -71,6 +72,7 @@ Example response: "fingerprint": "0b:cf:58:40:b9:23:96:c7:ba:44:df:0e:9e:87:5e:75", "fingerprint_sha256": "SHA256:lGI/Ys/Wx7PfMhUO1iuBH92JQKYN+3mhJZvWO4Q5ims", "created_at": "2013-10-02T11:12:29Z", + "expires_at": null, "projects_with_write_access": [] } ] @@ -103,6 +105,7 @@ Example response: "fingerprint": "4a:9d:64:15:ed:3a:e6:07:6e:89:36:b3:3b:03:05:d9", "fingerprint_sha256": "SHA256:Jrs3LD1Ji30xNLtTVf9NDCj7kkBgPBb2pjvTZ3HfIgU", "created_at": "2013-10-02T10:12:29Z", + "expires_at": null, "can_push": false }, { @@ -112,6 +115,7 @@ Example response: "fingerprint": "0b:cf:58:40:b9:23:96:c7:ba:44:df:0e:9e:87:5e:75", "fingerprint_sha256": "SHA256:lGI/Ys/Wx7PfMhUO1iuBH92JQKYN+3mhJZvWO4Q5ims", "created_at": "2013-10-02T11:12:29Z", + "expires_at": null, "can_push": false } ] @@ -205,6 +209,7 @@ Example response: "fingerprint": "4a:9d:64:15:ed:3a:e6:07:6e:89:36:b3:3b:03:05:d9", "fingerprint_sha256": "SHA256:Jrs3LD1Ji30xNLtTVf9NDCj7kkBgPBb2pjvTZ3HfIgU", "created_at": "2013-10-02T10:12:29Z", + "expires_at": null, "can_push": false } ``` @@ -220,12 +225,13 @@ project only if the original one is accessible by the same user. POST /projects/:id/deploy_keys ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `title` | string | yes | New deploy key's title | -| `key` | string | yes | New deploy key | -| `can_push` | boolean | no | Can deploy key push to the project's repository | +| Attribute | Type | Required | Description | +| ----------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | +| `title` | string | yes | New deploy key's title | +| `key` | string | yes | New deploy key | +| `expires_at` | datetime | no | Expiration date for the deploy key. Does not expire if no value is provided. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | +| `can_push` | boolean | no | Can deploy key push to the project's repository | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --header "Content-Type: application/json" \ @@ -241,7 +247,8 @@ Example response: "id" : 12, "title" : "My deploy key", "can_push": true, - "created_at" : "2015-08-29T12:44:31.550Z" + "created_at" : "2015-08-29T12:44:31.550Z", + "expires_at": null } ``` @@ -272,6 +279,7 @@ Example response: "title": "New deploy key", "key": "ssh-rsa AAAA...", "created_at": "2015-08-29T12:44:31.550Z", + "expires_at": null, "can_push": true } ``` @@ -317,7 +325,8 @@ Example response: "key" : "ssh-rsa AAAA...", "id" : 12, "title" : "My deploy key", - "created_at" : "2015-08-29T12:44:31.550Z" + "created_at" : "2015-08-29T12:44:31.550Z", + "expires_at": null } ``` diff --git a/doc/api/deploy_tokens.md b/doc/api/deploy_tokens.md index 0b7ea13c341..33b59d7c87f 100644 --- a/doc/api/deploy_tokens.md +++ b/doc/api/deploy_tokens.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/deployments.md b/doc/api/deployments.md index eb718d80b78..9cd5381da23 100644 --- a/doc/api/deployments.md +++ b/doc/api/deployments.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/api/dora/metrics.md b/doc/api/dora/metrics.md index a3865213714..a1515413bed 100644 --- a/doc/api/dora/metrics.md +++ b/doc/api/dora/metrics.md @@ -11,6 +11,8 @@ type: reference, api > - The legacy key/value pair `{ "<date>" => "<value>" }` was removed from the payload in GitLab 14.0. > `time_to_restore_service` metric was introduced in GitLab 14.9. +You can also retrieve [DORA metrics](../../user/analytics/dora_metrics.md) with the [GraphQL API](../../api/graphql/reference/index.md). + All methods require at least the Reporter role. ## Get project-level DORA metrics diff --git a/doc/api/draft_notes.md b/doc/api/draft_notes.md index 079b08781ae..e532de6a502 100644 --- a/doc/api/draft_notes.md +++ b/doc/api/draft_notes.md @@ -169,3 +169,20 @@ PUT /projects/:id/merge_requests/:merge_request_iid/draft_notes/:draft_note_id/p ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/14/merge_requests/11/draft_notes/5/publish" ``` + +## Publish all pending draft notes + +Bulk publishes all existing draft notes for a given merge request that belong to the user. + +```plaintext +POST /projects/:id/merge_requests/:merge_request_iid/draft_notes/bulk_publish +``` + +| Attribute | Type | Required | Description | +| ------------------- | ----------------- | -------- | --------------------- | +| `id` | integer or string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). +| `merge_request_iid` | integer | yes | The IID of a project merge request. + +```shell +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/14/merge_requests/11/draft_notes/bulk_publish" +``` diff --git a/doc/api/environments.md b/doc/api/environments.md index 2bd7b28d0d0..745abdd7ab3 100644 --- a/doc/api/environments.md +++ b/doc/api/environments.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: concepts, howto --- @@ -339,7 +339,7 @@ curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://git > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296625) in GitLab 14.2. It schedules for deletion multiple environments that have already been -[stopped](../ci/environments/index.md#stop-an-environment) and +[stopped](../ci/environments/index.md#stopping-an-environment) and are [in the review app folder](../ci/review_apps/index.md). The actual deletion is performed after 1 week from the time of execution. By default, it only deletes environments 30 days or older. You can change this default using the `before` parameter. @@ -415,7 +415,7 @@ Example response: ## Stop stale environments -Issue stop request to all environments that were last modified or deployed to before a specified date. Excludes protected environments. Returns `200` if stop request was successful and `400` if the before date is invalid. For details of exactly when the environment is stopped, see [Stop an environment](../ci/environments/index.md#stop-an-environment). +Issue stop request to all environments that were last modified or deployed to before a specified date. Excludes protected environments. Returns `200` if stop request was successful and `400` if the before date is invalid. For details of exactly when the environment is stopped, see [Stop an environment](../ci/environments/index.md#stopping-an-environment). ```plaintext POST /projects/:id/environments/stop_stale diff --git a/doc/api/feature_flag_user_lists.md b/doc/api/feature_flag_user_lists.md index 0d48f8daf40..ee8ec3f9595 100644 --- a/doc/api/feature_flag_user_lists.md +++ b/doc/api/feature_flag_user_lists.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -9,9 +9,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/205409) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212318) to GitLab Free in 13.5. -API for accessing GitLab Feature Flag User Lists. +API for accessing GitLab feature flag user lists. -Users with Developer or higher [permissions](../user/permissions.md) can access the Feature Flag User Lists API. +Users with Developer or higher [permissions](../user/permissions.md) can access the feature flag user lists API. NOTE: `GET` requests return twenty results at a time because the API results diff --git a/doc/api/feature_flags.md b/doc/api/feature_flags.md index f62d51ea427..2e8e232876e 100644 --- a/doc/api/feature_flags.md +++ b/doc/api/feature_flags.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/features.md b/doc/api/features.md index 2b0bf656664..f83f445d67d 100644 --- a/doc/api/features.md +++ b/doc/api/features.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/freeze_periods.md b/doc/api/freeze_periods.md index cb9bf81e961..c2cf2d84056 100644 --- a/doc/api/freeze_periods.md +++ b/doc/api/freeze_periods.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/api/geo_nodes.md b/doc/api/geo_nodes.md index ed413c358de..038b7d633a6 100644 --- a/doc/api/geo_nodes.md +++ b/doc/api/geo_nodes.md @@ -439,6 +439,18 @@ Example response: "snippet_repositories_verification_failed_count": null, "snippet_repositories_synced_in_percentage": "0.00%", "snippet_repositories_verified_in_percentage": "0.00%", + "project_wiki_repositories_count": 3, + "project_wiki_repositories_checksum_total_count": 3, + "project_wiki_repositories_checksummed_count": 3, + "project_wiki_repositories_checksum_failed_count": 0, + "project_wiki_repositories_synced_count": null, + "project_wiki_repositories_failed_count": null, + "project_wiki_repositories_registry_count": null, + "project_wiki_repositories_verification_total_count": null, + "project_wiki_repositories_verified_count": null, + "project_wiki_repositories_verification_failed_count": null, + "project_wiki_repositories_synced_in_percentage": "0.00%", + "project_wiki_repositories_verified_in_percentage": "0.00%", "group_wiki_repositories_count": 5, "group_wiki_repositories_checksum_total_count": 5, "group_wiki_repositories_checksummed_count": 5, diff --git a/doc/api/graphql/branch_rules.md b/doc/api/graphql/branch_rules.md index c38ebd673d0..5709ce9fafb 100644 --- a/doc/api/graphql/branch_rules.md +++ b/doc/api/graphql/branch_rules.md @@ -130,7 +130,7 @@ Instead of requesting access, it may be easier for you to run the query in the project(fullPath: "flightjs/Flight") { ``` -1. Visit the [GraphiQL explorer tool](http://gdk.test:3000/-/graphql-explorer) for your GDK instance. +1. In your GDK instance, visit the GraphiQL explorer tool: `http://gdk.test:3000/-/graphql-explorer`. 1. Paste the `query` listed above into the left window of your GraphiQL explorer tool. 1. Select **Play** to view the result. diff --git a/doc/api/graphql/getting_started.md b/doc/api/graphql/getting_started.md index df1b1cdca7c..d45228c34c8 100644 --- a/doc/api/graphql/getting_started.md +++ b/doc/api/graphql/getting_started.md @@ -286,6 +286,24 @@ We've asked for the note details, but it doesn't exist anymore, so we get `null` More about mutations: [GraphQL Documentation](https://graphql.org/learn/queries/#mutations). +### Update project settings + +You can update multiple project settings in a single GraphQL mutation. +This example is a workaround for [the major change](../../update/deprecations.md#default-cicd-job-token-ci_job_token-scope-changed) +in `CI_JOB_TOKEN` scoping behavior. + +```graphql +mutation DisableCI_JOB_TOKENscope { + projectCiCdSettingsUpdate(input:{fullPath: "<namespace>/<project-name>", inboundJobTokenScopeEnabled: false, jobTokenScopeEnabled: false}) { + ciCdSettings { + inboundJobTokenScopeEnabled + jobTokenScopeEnabled + } + errors + } +} +``` + ### Introspective queries Clients can query the GraphQL endpoint for information about its own schema. diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index bcd7c9dde44..68813fe13b5 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -55,6 +55,26 @@ CI related settings that apply to the entire instance. Returns [`CiApplicationSettings`](#ciapplicationsettings). +### `Query.ciCatalogResources` + +CI Catalog resources visible to the current user. + +WARNING: +**Introduced** in 15.11. +This feature is an Experiment. It can be changed or removed at any time. + +Returns [`CiCatalogResourceConnection`](#cicatalogresourceconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="querycicatalogresourcesprojectpath"></a>`projectPath` | [`ID`](#id) | Project with the namespace catalog. | + ### `Query.ciConfig` Linted and processed contents of a CI config. @@ -85,6 +105,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="queryciminutesusagedate"></a>`date` | [`Date`](#date) | Date for which to retrieve the usage data, should be the first day of a month. | | <a id="queryciminutesusagenamespaceid"></a>`namespaceId` | [`NamespaceID`](#namespaceid) | Global ID of the Namespace for the monthly CI/CD minutes usage. | ### `Query.ciVariables` @@ -242,7 +263,7 @@ Find issues visible to the current user. At least one filter must be provided. WARNING: **Introduced** in 15.6. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`IssueConnection`](#issueconnection). @@ -374,7 +395,7 @@ Find a note. WARNING: **Introduced** in 15.9. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`Note`](#note). @@ -542,7 +563,7 @@ Find a synthetic note. WARNING: **Introduced** in 15.9. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`Note`](#note). @@ -717,7 +738,7 @@ Find a work item. WARNING: **Introduced** in 15.1. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`WorkItem`](#workitem). @@ -750,7 +771,7 @@ mutation($id: NoteableID!, $body: String!) { WARNING: **Introduced** in 15.10. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `AchievementsAwardInput` @@ -774,7 +795,7 @@ Input type: `AchievementsAwardInput` WARNING: **Introduced** in 15.8. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `AchievementsCreateInput` @@ -796,11 +817,34 @@ Input type: `AchievementsCreateInput` | <a id="mutationachievementscreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationachievementscreateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +### `Mutation.achievementsDelete` + +WARNING: +**Introduced** in 15.11. +This feature is an Experiment. It can be changed or removed at any time. + +Input type: `AchievementsDeleteInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationachievementsdeleteachievementid"></a>`achievementId` | [`AchievementsAchievementID!`](#achievementsachievementid) | Global ID of the achievement being deleted. | +| <a id="mutationachievementsdeleteclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationachievementsdeleteachievement"></a>`achievement` | [`Achievement`](#achievement) | Achievement. | +| <a id="mutationachievementsdeleteclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationachievementsdeleteerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | + ### `Mutation.achievementsRevoke` WARNING: **Introduced** in 15.10. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `AchievementsRevokeInput` @@ -819,6 +863,32 @@ Input type: `AchievementsRevokeInput` | <a id="mutationachievementsrevokeerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationachievementsrevokeuserachievement"></a>`userAchievement` | [`UserAchievement`](#userachievement) | Achievement award. | +### `Mutation.achievementsUpdate` + +WARNING: +**Introduced** in 15.11. +This feature is an Experiment. It can be changed or removed at any time. + +Input type: `AchievementsUpdateInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationachievementsupdateachievementid"></a>`achievementId` | [`AchievementsAchievementID!`](#achievementsachievementid) | Global ID of the achievement being updated. | +| <a id="mutationachievementsupdateavatar"></a>`avatar` | [`Upload`](#upload) | Avatar for the achievement. | +| <a id="mutationachievementsupdateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationachievementsupdatedescription"></a>`description` | [`String`](#string) | Description of or notes for the achievement. | +| <a id="mutationachievementsupdatename"></a>`name` | [`String`](#string) | Name for the achievement. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationachievementsupdateachievement"></a>`achievement` | [`Achievement`](#achievement) | Achievement. | +| <a id="mutationachievementsupdateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationachievementsupdateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | + ### `Mutation.addProjectToSecurityDashboard` Input type: `AddProjectToSecurityDashboardInput` @@ -855,6 +925,7 @@ Input type: `AdminSidekiqQueuesDeleteJobsInput` | <a id="mutationadminsidekiqqueuesdeletejobsclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationadminsidekiqqueuesdeletejobsfeaturecategory"></a>`featureCategory` | [`String`](#string) | Delete jobs matching feature_category in the context metadata. | | <a id="mutationadminsidekiqqueuesdeletejobsjobid"></a>`jobId` | [`String`](#string) | Delete jobs matching job_id in the context metadata. | +| <a id="mutationadminsidekiqqueuesdeletejobsmergeactionstatus"></a>`mergeActionStatus` | [`String`](#string) | Delete jobs matching merge_action_status in the context metadata. | | <a id="mutationadminsidekiqqueuesdeletejobspipelineid"></a>`pipelineId` | [`String`](#string) | Delete jobs matching pipeline_id in the context metadata. | | <a id="mutationadminsidekiqqueuesdeletejobsproject"></a>`project` | [`String`](#string) | Delete jobs matching project in the context metadata. | | <a id="mutationadminsidekiqqueuesdeletejobsqueuename"></a>`queueName` | [`String!`](#string) | Name of the queue to delete jobs from. | @@ -875,6 +946,30 @@ Input type: `AdminSidekiqQueuesDeleteJobsInput` | <a id="mutationadminsidekiqqueuesdeletejobserrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationadminsidekiqqueuesdeletejobsresult"></a>`result` | [`DeleteJobsResponse`](#deletejobsresponse) | Information about the status of the deletion request. | +### `Mutation.aiAction` + +WARNING: +**Introduced** in 15.11. +This feature is an Experiment. It can be changed or removed at any time. + +Input type: `AiActionInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationaiactionclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationaiactionexplaincode"></a>`explainCode` | [`AiExplainCodeInput`](#aiexplaincodeinput) | Input for explain_code AI action. | +| <a id="mutationaiactionexplainvulnerability"></a>`explainVulnerability` | [`AiExplainVulnerabilityInput`](#aiexplainvulnerabilityinput) | Input for explain_vulnerability AI action. | +| <a id="mutationaiactionsummarizecomments"></a>`summarizeComments` | [`AiSummarizeCommentsInput`](#aisummarizecommentsinput) | Input for summarize_comments AI action. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationaiactionclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationaiactionerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | + ### `Mutation.alertSetAssignees` Input type: `AlertSetAssigneesInput` @@ -1224,7 +1319,7 @@ Input type: `BoardListUpdateLimitMetricsInput` WARNING: **Introduced** in 15.10. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `BulkDestroyJobArtifactsInput` @@ -1271,7 +1366,7 @@ Input type: `BulkEnableDevopsAdoptionNamespacesInput` WARNING: **Introduced** in 15.3. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `BulkRunnerDeleteInput` @@ -1291,6 +1386,28 @@ Input type: `BulkRunnerDeleteInput` | <a id="mutationbulkrunnerdeletedeletedids"></a>`deletedIds` | [`[CiRunnerID!]`](#cirunnerid) | IDs of records effectively deleted. Only present if operation was performed synchronously. | | <a id="mutationbulkrunnerdeleteerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +### `Mutation.catalogResourcesCreate` + +WARNING: +**Introduced** in 15.11. +This feature is an Experiment. It can be changed or removed at any time. + +Input type: `CatalogResourcesCreateInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationcatalogresourcescreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationcatalogresourcescreateprojectpath"></a>`projectPath` | [`ID!`](#id) | Project to convert to a catalog resource. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationcatalogresourcescreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationcatalogresourcescreateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | + ### `Mutation.ciCdSettingsUpdate` WARNING: @@ -1729,7 +1846,7 @@ Input type: `CreateComplianceFrameworkInput` WARNING: **Introduced** in 13.6. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `CreateCustomEmojiInput` @@ -2619,7 +2736,7 @@ Input type: `DestroyContainerRepositoryTagsInput` WARNING: **Introduced** in 13.6. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `DestroyCustomEmojiInput` @@ -3738,7 +3855,7 @@ Allows updating several properties for a set of issues. Does nothing if the `bul WARNING: **Introduced** in 15.9. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `IssuesBulkUpdateInput` @@ -4765,6 +4882,8 @@ Input type: `ProjectInitializeProductAnalyticsInput` ### `Mutation.projectMemberBulkUpdate` +Updates multiple members of a project. To use this mutation, you must have at least the Maintainer role. + Input type: `ProjectMemberBulkUpdateInput` #### Arguments @@ -4832,7 +4951,7 @@ Input type: `ProjectSetLockedInput` WARNING: **Introduced** in 15.9. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `ProjectSyncForkInput` @@ -5116,7 +5235,7 @@ Input type: `RepositionImageDiffNoteInput` WARNING: **Introduced** in 15.10. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `RunnerCreateInput` @@ -5125,14 +5244,16 @@ Input type: `RunnerCreateInput` | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mutationrunnercreateaccesslevel"></a>`accessLevel` | [`CiRunnerAccessLevel`](#cirunneraccesslevel) | Access level of the runner. | -| <a id="mutationrunnercreateassociatedprojects"></a>`associatedProjects` | [`[ProjectID!]`](#projectid) | Projects associated with the runner. Available only for project runners. | | <a id="mutationrunnercreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationrunnercreatedescription"></a>`description` | [`String`](#string) | Description of the runner. | +| <a id="mutationrunnercreategroupid"></a>`groupId` | [`GroupID`](#groupid) | Global ID of the group that the runner is created in (valid only for group runner). | | <a id="mutationrunnercreatelocked"></a>`locked` | [`Boolean`](#boolean) | Indicates the runner is locked. | | <a id="mutationrunnercreatemaintenancenote"></a>`maintenanceNote` | [`String`](#string) | Runner's maintenance notes. | | <a id="mutationrunnercreatemaximumtimeout"></a>`maximumTimeout` | [`Int`](#int) | Maximum timeout (in seconds) for jobs processed by the runner. | | <a id="mutationrunnercreatepaused"></a>`paused` | [`Boolean`](#boolean) | Indicates the runner is not allowed to receive jobs. | +| <a id="mutationrunnercreateprojectid"></a>`projectId` | [`ProjectID`](#projectid) | Global ID of the project that the runner is created in (valid only for project runner). | | <a id="mutationrunnercreaterununtagged"></a>`runUntagged` | [`Boolean`](#boolean) | Indicates the runner is able to run untagged jobs. | +| <a id="mutationrunnercreaterunnertype"></a>`runnerType` | [`CiRunnerType!`](#cirunnertype) | Type of the runner to create. | | <a id="mutationrunnercreatetaglist"></a>`tagList` | [`[String!]`](#string) | Tags associated with the runner. | #### Fields @@ -5317,6 +5438,25 @@ Input type: `SecurityFindingCreateIssueInput` | <a id="mutationsecurityfindingcreateissueerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationsecurityfindingcreateissueissue"></a>`issue` | [`Issue`](#issue) | Issue created after mutation. | +### `Mutation.securityFindingCreateMergeRequest` + +Input type: `SecurityFindingCreateMergeRequestInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsecurityfindingcreatemergerequestclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsecurityfindingcreatemergerequestuuid"></a>`uuid` | [`String!`](#string) | UUID of the security finding to be used to create a merge request. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsecurityfindingcreatemergerequestclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsecurityfindingcreatemergerequesterrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationsecurityfindingcreatemergerequestmergerequest"></a>`mergeRequest` | [`MergeRequest`](#mergerequest) | Merge Request created after mutation. | + ### `Mutation.securityFindingDismiss` Input type: `SecurityFindingDismissInput` @@ -5503,7 +5643,7 @@ Input type: `TerraformStateUnlockInput` WARNING: **Introduced** in 15.6. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `TimelineEventCreateInput` @@ -6256,7 +6396,7 @@ Input type: `VulnerabilityConfirmInput` | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mutationvulnerabilityconfirmclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | -| <a id="mutationvulnerabilityconfirmcomment"></a>`comment` | [`String`](#string) | Comment why vulnerability was marked as confirmed (max. 50 000 characters). | +| <a id="mutationvulnerabilityconfirmcomment"></a>`comment` | [`String`](#string) | Comment why vulnerability was confirmed (maximum 50,000 characters). | | <a id="mutationvulnerabilityconfirmid"></a>`id` | [`VulnerabilityID!`](#vulnerabilityid) | ID of the vulnerability to be confirmed. | #### Fields @@ -6308,7 +6448,7 @@ Input type: `VulnerabilityDismissInput` | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mutationvulnerabilitydismissclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | -| <a id="mutationvulnerabilitydismisscomment"></a>`comment` | [`String`](#string) | Comment why vulnerability should be dismissed (max. 50 000 characters). | +| <a id="mutationvulnerabilitydismisscomment"></a>`comment` | [`String`](#string) | Comment why vulnerability was dismissed (maximum 50,000 characters). | | <a id="mutationvulnerabilitydismissdismissalreason"></a>`dismissalReason` | [`VulnerabilityDismissalReason`](#vulnerabilitydismissalreason) | Reason why vulnerability should be dismissed. | | <a id="mutationvulnerabilitydismissid"></a>`id` | [`VulnerabilityID!`](#vulnerabilityid) | ID of the vulnerability to be dismissed. | @@ -6318,7 +6458,7 @@ Input type: `VulnerabilityDismissInput` | ---- | ---- | ----------- | | <a id="mutationvulnerabilitydismissclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationvulnerabilitydismisserrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | -| <a id="mutationvulnerabilitydismissvulnerability"></a>`vulnerability` | [`Vulnerability`](#vulnerability) | Vulnerability after dismissal. | +| <a id="mutationvulnerabilitydismissvulnerability"></a>`vulnerability` | [`Vulnerability`](#vulnerability) | Vulnerability after state change. | ### `Mutation.vulnerabilityExternalIssueLinkCreate` @@ -6385,6 +6525,26 @@ Input type: `VulnerabilityFindingDismissInput` | <a id="mutationvulnerabilityfindingdismisserrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationvulnerabilityfindingdismissfinding"></a>`finding` | [`PipelineSecurityReportFinding`](#pipelinesecurityreportfinding) | Finding after dismissal. | +### `Mutation.vulnerabilityIssueLinkCreate` + +Input type: `VulnerabilityIssueLinkCreateInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationvulnerabilityissuelinkcreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationvulnerabilityissuelinkcreateissueid"></a>`issueId` | [`IssueID!`](#issueid) | ID of the issue to link to. | +| <a id="mutationvulnerabilityissuelinkcreatevulnerabilityids"></a>`vulnerabilityIds` | [`[VulnerabilityID!]!`](#vulnerabilityid) | IDs of vulnerabilities to link to the given issue. Up to 100 can be provided. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationvulnerabilityissuelinkcreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationvulnerabilityissuelinkcreateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationvulnerabilityissuelinkcreateissuelinks"></a>`issueLinks` | [`[VulnerabilityIssueLink!]`](#vulnerabilityissuelink) | Created issue links. | + ### `Mutation.vulnerabilityResolve` Input type: `VulnerabilityResolveInput` @@ -6394,7 +6554,7 @@ Input type: `VulnerabilityResolveInput` | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mutationvulnerabilityresolveclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | -| <a id="mutationvulnerabilityresolvecomment"></a>`comment` | [`String`](#string) | Comment why vulnerability was reverted to detected (max. 50 000 characters). | +| <a id="mutationvulnerabilityresolvecomment"></a>`comment` | [`String`](#string) | Comment why vulnerability was resolved (maximum 50,000 characters). | | <a id="mutationvulnerabilityresolveid"></a>`id` | [`VulnerabilityID!`](#vulnerabilityid) | ID of the vulnerability to be resolved. | #### Fields @@ -6414,8 +6574,8 @@ Input type: `VulnerabilityRevertToDetectedInput` | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mutationvulnerabilityreverttodetectedclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | -| <a id="mutationvulnerabilityreverttodetectedcomment"></a>`comment` | [`String`](#string) | Comment why vulnerability was reverted to detected (max. 50 000 characters). | -| <a id="mutationvulnerabilityreverttodetectedid"></a>`id` | [`VulnerabilityID!`](#vulnerabilityid) | ID of the vulnerability to be reverted. | +| <a id="mutationvulnerabilityreverttodetectedcomment"></a>`comment` | [`String`](#string) | Comment why vulnerability was reverted to detected (maximum 50,000 characters). | +| <a id="mutationvulnerabilityreverttodetectedid"></a>`id` | [`VulnerabilityID!`](#vulnerabilityid) | ID of the vulnerability to be reverted to detected. | #### Fields @@ -6423,7 +6583,33 @@ Input type: `VulnerabilityRevertToDetectedInput` | ---- | ---- | ----------- | | <a id="mutationvulnerabilityreverttodetectedclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationvulnerabilityreverttodetectederrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | -| <a id="mutationvulnerabilityreverttodetectedvulnerability"></a>`vulnerability` | [`Vulnerability`](#vulnerability) | Vulnerability after revert. | +| <a id="mutationvulnerabilityreverttodetectedvulnerability"></a>`vulnerability` | [`Vulnerability`](#vulnerability) | Vulnerability after state change. | + +### `Mutation.workItemConvert` + +Converts the work item to a new type. + +WARNING: +**Introduced** in 15.11. +This feature is an Experiment. It can be changed or removed at any time. + +Input type: `WorkItemConvertInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationworkitemconvertclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationworkitemconvertid"></a>`id` | [`WorkItemID!`](#workitemid) | Global ID of the work item. | +| <a id="mutationworkitemconvertworkitemtypeid"></a>`workItemTypeId` | [`WorkItemsTypeID!`](#workitemstypeid) | Global ID of the new work item type. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationworkitemconvertclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationworkitemconverterrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationworkitemconvertworkitem"></a>`workItem` | [`WorkItem`](#workitem) | Updated work item. | ### `Mutation.workItemCreate` @@ -6431,7 +6617,7 @@ Creates a work item. WARNING: **Introduced** in 15.1. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `WorkItemCreateInput` @@ -6445,7 +6631,8 @@ Input type: `WorkItemCreateInput` | <a id="mutationworkitemcreatehierarchywidget"></a>`hierarchyWidget` | [`WorkItemWidgetHierarchyCreateInput`](#workitemwidgethierarchycreateinput) | Input for hierarchy widget. | | <a id="mutationworkitemcreateiterationwidget"></a>`iterationWidget` | [`WorkItemWidgetIterationInput`](#workitemwidgetiterationinput) | Iteration widget of the work item. | | <a id="mutationworkitemcreatemilestonewidget"></a>`milestoneWidget` | [`WorkItemWidgetMilestoneInput`](#workitemwidgetmilestoneinput) | Input for milestone widget. | -| <a id="mutationworkitemcreateprojectpath"></a>`projectPath` | [`ID!`](#id) | Full path of the project the work item is associated with. | +| <a id="mutationworkitemcreatenamespacepath"></a>`namespacePath` | [`ID`](#id) | Full path of the namespace(project or group) the work item is created in. | +| <a id="mutationworkitemcreateprojectpath"></a>`projectPath` **{warning-solid}** | [`ID`](#id) | **Deprecated:** Please use namespace_path instead. That will cover for both projects and groups. Deprecated in 15.10. | | <a id="mutationworkitemcreatetitle"></a>`title` | [`String!`](#string) | Title of the work item. | | <a id="mutationworkitemcreateworkitemtypeid"></a>`workItemTypeId` | [`WorkItemsTypeID!`](#workitemstypeid) | Global ID of a work item type. | @@ -6463,7 +6650,7 @@ Creates a work item from a task in another work item's description. WARNING: **Introduced** in 15.1. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `WorkItemCreateFromTaskInput` @@ -6490,7 +6677,7 @@ Deletes a work item. WARNING: **Introduced** in 15.1. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `WorkItemDeleteInput` @@ -6515,7 +6702,7 @@ Deletes a task in a work item's description. WARNING: **Introduced** in 15.1. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `WorkItemDeleteTaskInput` @@ -6540,7 +6727,7 @@ Input type: `WorkItemDeleteTaskInput` WARNING: **Introduced** in 15.10. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `WorkItemExportInput` @@ -6548,7 +6735,7 @@ Input type: `WorkItemExportInput` | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="mutationworkitemexportauthorusername"></a>`authorUsername` **{warning-solid}** | [`String`](#string) | **Deprecated:** This feature is in Alpha. It can be changed or removed at any time. Introduced in 15.9. | +| <a id="mutationworkitemexportauthorusername"></a>`authorUsername` **{warning-solid}** | [`String`](#string) | **Deprecated:** This feature is an Experiment. It can be changed or removed at any time. Introduced in 15.9. | | <a id="mutationworkitemexportclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationworkitemexportiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of work items. For example, `["1", "2"]`. | | <a id="mutationworkitemexportin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | @@ -6571,7 +6758,7 @@ Updates a work item by Global ID. WARNING: **Introduced** in 15.1. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `WorkItemUpdateInput` @@ -6580,8 +6767,10 @@ Input type: `WorkItemUpdateInput` | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mutationworkitemupdateassigneeswidget"></a>`assigneesWidget` | [`WorkItemWidgetAssigneesInput`](#workitemwidgetassigneesinput) | Input for assignees widget. | +| <a id="mutationworkitemupdateawardemojiwidget"></a>`awardEmojiWidget` | [`WorkItemWidgetAwardEmojiUpdateInput`](#workitemwidgetawardemojiupdateinput) | Input for award emoji widget. | | <a id="mutationworkitemupdateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationworkitemupdateconfidential"></a>`confidential` | [`Boolean`](#boolean) | Sets the work item confidentiality. | +| <a id="mutationworkitemupdatecurrentusertodoswidget"></a>`currentUserTodosWidget` | [`WorkItemWidgetCurrentUserTodosInput`](#workitemwidgetcurrentusertodosinput) | Input for to-dos widget. | | <a id="mutationworkitemupdatedescriptionwidget"></a>`descriptionWidget` | [`WorkItemWidgetDescriptionInput`](#workitemwidgetdescriptioninput) | Input for description widget. | | <a id="mutationworkitemupdatehealthstatuswidget"></a>`healthStatusWidget` | [`WorkItemWidgetHealthStatusInput`](#workitemwidgethealthstatusinput) | Input for health status widget. | | <a id="mutationworkitemupdatehierarchywidget"></a>`hierarchyWidget` | [`WorkItemWidgetHierarchyUpdateInput`](#workitemwidgethierarchyupdateinput) | Input for hierarchy widget. | @@ -6611,7 +6800,7 @@ Updates a work item's task by Global ID. WARNING: **Introduced** in 15.1. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Input type: `WorkItemUpdateTaskInput` @@ -6994,6 +7183,30 @@ The edge type for [`CiBuildNeed`](#cibuildneed). | <a id="cibuildneededgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="cibuildneededgenode"></a>`node` | [`CiBuildNeed`](#cibuildneed) | The item at the end of the edge. | +#### `CiCatalogResourceConnection` + +The connection type for [`CiCatalogResource`](#cicatalogresource). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="cicatalogresourceconnectioncount"></a>`count` | [`Int!`](#int) | Total count of collection. | +| <a id="cicatalogresourceconnectionedges"></a>`edges` | [`[CiCatalogResourceEdge]`](#cicatalogresourceedge) | A list of edges. | +| <a id="cicatalogresourceconnectionnodes"></a>`nodes` | [`[CiCatalogResource]`](#cicatalogresource) | A list of nodes. | +| <a id="cicatalogresourceconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `CiCatalogResourceEdge` + +The edge type for [`CiCatalogResource`](#cicatalogresource). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="cicatalogresourceedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="cicatalogresourceedgenode"></a>`node` | [`CiCatalogResource`](#cicatalogresource) | The item at the end of the edge. | + #### `CiConfigGroupConnection` The connection type for [`CiConfigGroup`](#ciconfiggroup). @@ -7336,29 +7549,29 @@ The edge type for [`CiRunner`](#cirunner). | <a id="cirunneredgenode"></a>`node` | [`CiRunner`](#cirunner) | The item at the end of the edge. | | <a id="cirunneredgeweburl"></a>`webUrl` | [`String`](#string) | Web URL of the runner. The value depends on where you put this field in the query. You can use it for projects or groups. | -#### `CiRunnerMachineConnection` +#### `CiRunnerManagerConnection` -The connection type for [`CiRunnerMachine`](#cirunnermachine). +The connection type for [`CiRunnerManager`](#cirunnermanager). ##### Fields | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="cirunnermachineconnectioncount"></a>`count` | [`Int!`](#int) | Total count of collection. | -| <a id="cirunnermachineconnectionedges"></a>`edges` | [`[CiRunnerMachineEdge]`](#cirunnermachineedge) | A list of edges. | -| <a id="cirunnermachineconnectionnodes"></a>`nodes` | [`[CiRunnerMachine]`](#cirunnermachine) | A list of nodes. | -| <a id="cirunnermachineconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | +| <a id="cirunnermanagerconnectioncount"></a>`count` | [`Int!`](#int) | Total count of collection. | +| <a id="cirunnermanagerconnectionedges"></a>`edges` | [`[CiRunnerManagerEdge]`](#cirunnermanageredge) | A list of edges. | +| <a id="cirunnermanagerconnectionnodes"></a>`nodes` | [`[CiRunnerManager]`](#cirunnermanager) | A list of nodes. | +| <a id="cirunnermanagerconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | -#### `CiRunnerMachineEdge` +#### `CiRunnerManagerEdge` -The edge type for [`CiRunnerMachine`](#cirunnermachine). +The edge type for [`CiRunnerManager`](#cirunnermanager). ##### Fields | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="cirunnermachineedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | -| <a id="cirunnermachineedgenode"></a>`node` | [`CiRunnerMachine`](#cirunnermachine) | The item at the end of the edge. | +| <a id="cirunnermanageredgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="cirunnermanageredgenode"></a>`node` | [`CiRunnerManager`](#cirunnermanager) | The item at the end of the edge. | #### `CiSecureFileRegistryConnection` @@ -9563,6 +9776,29 @@ The edge type for [`ProjectMember`](#projectmember). | <a id="projectmemberedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="projectmemberedgenode"></a>`node` | [`ProjectMember`](#projectmember) | The item at the end of the edge. | +#### `ProjectWikiRepositoryRegistryConnection` + +The connection type for [`ProjectWikiRepositoryRegistry`](#projectwikirepositoryregistry). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectwikirepositoryregistryconnectionedges"></a>`edges` | [`[ProjectWikiRepositoryRegistryEdge]`](#projectwikirepositoryregistryedge) | A list of edges. | +| <a id="projectwikirepositoryregistryconnectionnodes"></a>`nodes` | [`[ProjectWikiRepositoryRegistry]`](#projectwikirepositoryregistry) | A list of nodes. | +| <a id="projectwikirepositoryregistryconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `ProjectWikiRepositoryRegistryEdge` + +The edge type for [`ProjectWikiRepositoryRegistry`](#projectwikirepositoryregistry). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectwikirepositoryregistryedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="projectwikirepositoryregistryedgenode"></a>`node` | [`ProjectWikiRepositoryRegistry`](#projectwikirepositoryregistry) | The item at the end of the edge. | + #### `ProtectedEnvironmentApprovalRuleConnection` The connection type for [`ProtectedEnvironmentApprovalRule`](#protectedenvironmentapprovalrule). @@ -10903,7 +11139,7 @@ Representation of a GitLab user. | <a id="achievementname"></a>`name` | [`String!`](#string) | Name of the achievement. | | <a id="achievementnamespace"></a>`namespace` | [`Namespace!`](#namespace) | Namespace of the achievement. | | <a id="achievementupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp the achievement was last updated. | -| <a id="achievementuserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Recipients for the achievement. | +| <a id="achievementuserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Recipients for the achievement. | ### `AgentConfiguration` @@ -10928,6 +11164,15 @@ Information about a connected Agent. | <a id="agentmetadatapodnamespace"></a>`podNamespace` | [`String`](#string) | Namespace of the pod running the Agent. | | <a id="agentmetadataversion"></a>`version` | [`String`](#string) | Agent version tag. | +### `AiResponse` + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="airesponseerrors"></a>`errors` | [`[String!]`](#string) | Errors return by AI API as response. | +| <a id="airesponseresponsebody"></a>`responseBody` | [`String`](#string) | Response body from AI API. | + ### `AlertManagementAlert` Describes an alert from the project's Alert Management. @@ -11107,6 +11352,7 @@ Describes a rule for who can approve merge requests. | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="approvalruleallowmergewheninvalid"></a>`allowMergeWhenInvalid` | [`Boolean`](#boolean) | Indicates if the rule can be ignored if it is invalid. | | <a id="approvalruleapprovalsrequired"></a>`approvalsRequired` | [`Int`](#int) | Number of required approvals. | | <a id="approvalruleapproved"></a>`approved` | [`Boolean`](#boolean) | Indicates if the rule is satisfied. | | <a id="approvalruleapprovedby"></a>`approvedBy` | [`UserCoreConnection`](#usercoreconnection) | List of users defined in the rule that approved the merge request. (see [Connections](#connections)) | @@ -11115,6 +11361,7 @@ Describes a rule for who can approve merge requests. | <a id="approvalruleeligibleapprovers"></a>`eligibleApprovers` | [`[UserCore!]`](#usercore) | List of all users eligible to approve the merge request (defined explicitly and from associated groups). | | <a id="approvalrulegroups"></a>`groups` | [`GroupConnection`](#groupconnection) | List of groups added as approvers for the rule. (see [Connections](#connections)) | | <a id="approvalruleid"></a>`id` | [`GlobalID!`](#globalid) | ID of the rule. | +| <a id="approvalruleinvalid"></a>`invalid` | [`Boolean`](#boolean) | Indicates if the rule is invalid and cannot be approved. | | <a id="approvalrulename"></a>`name` | [`String`](#string) | Name of the rule. | | <a id="approvalruleoverridden"></a>`overridden` | [`Boolean`](#boolean) | Indicates if the rule was overridden for the merge request. | | <a id="approvalrulesection"></a>`section` | [`String`](#string) | Named section of the Code Owners file that the rule applies to. | @@ -11358,7 +11605,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="boardepicancestorsmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Filter epics by milestone title, computed from epic's issues. | | <a id="boardepicancestorsmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. | | <a id="boardepicancestorsnot"></a>`not` | [`NegatedEpicFilterInput`](#negatedepicfilterinput) | Negated epic arguments. | -| <a id="boardepicancestorsor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | +| <a id="boardepicancestorsor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | | <a id="boardepicancestorssearch"></a>`search` | [`String`](#string) | Search query for title or description. | | <a id="boardepicancestorssort"></a>`sort` | [`EpicSort`](#epicsort) | List epics by sort order. | | <a id="boardepicancestorsstartdate"></a>`startDate` **{warning-solid}** | [`Time`](#time) | **Deprecated** in 13.5. Use timeframe.start. | @@ -11397,7 +11644,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="boardepicchildrenmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Filter epics by milestone title, computed from epic's issues. | | <a id="boardepicchildrenmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. | | <a id="boardepicchildrennot"></a>`not` | [`NegatedEpicFilterInput`](#negatedepicfilterinput) | Negated epic arguments. | -| <a id="boardepicchildrenor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | +| <a id="boardepicchildrenor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | | <a id="boardepicchildrensearch"></a>`search` | [`String`](#string) | Search query for title or description. | | <a id="boardepicchildrensort"></a>`sort` | [`EpicSort`](#epicsort) | List epics by sort order. | | <a id="boardepicchildrenstartdate"></a>`startDate` **{warning-solid}** | [`Time`](#time) | **Deprecated** in 13.5. Use timeframe.start. | @@ -11558,6 +11805,17 @@ Represents the total number of issues and their weights for a particular day. | <a id="cibuildneedid"></a>`id` | [`ID!`](#id) | ID of the BuildNeed. | | <a id="cibuildneedname"></a>`name` | [`String`](#string) | Name of the job we need to complete. | +### `CiCatalogResource` + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="cicatalogresourcedescription"></a>`description` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. Description of the catalog resource. | +| <a id="cicatalogresourceicon"></a>`icon` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. Icon for the catalog resource. | +| <a id="cicatalogresourceid"></a>`id` **{warning-solid}** | [`ID!`](#id) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. ID of the catalog resource. | +| <a id="cicatalogresourcename"></a>`name` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. Name of the catalog resource. | + ### `CiConfig` #### Fields @@ -11752,7 +12010,7 @@ CI/CD variables for a GitLab instance. | <a id="cijobrefpath"></a>`refPath` | [`String`](#string) | Path to the ref. | | <a id="cijobretried"></a>`retried` | [`Boolean`](#boolean) | Indicates that the job has been retried. | | <a id="cijobretryable"></a>`retryable` | [`Boolean!`](#boolean) | Indicates the job can be retried. | -| <a id="cijobrunnermachine"></a>`runnerMachine` **{warning-solid}** | [`CiRunnerMachine`](#cirunnermachine) | **Introduced** in 15.11. This feature is in Alpha. It can be changed or removed at any time. Runner machine assigned to the job. | +| <a id="cijobrunnermanager"></a>`runnerManager` **{warning-solid}** | [`CiRunnerManager`](#cirunnermanager) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. Runner manager assigned to the job. | | <a id="cijobscheduled"></a>`scheduled` | [`Boolean!`](#boolean) | Indicates the job is scheduled. | | <a id="cijobscheduledat"></a>`scheduledAt` | [`Time`](#time) | Schedule for the build. | | <a id="cijobschedulingtype"></a>`schedulingType` | [`String`](#string) | Type of job scheduling. Value is `dag` if the job uses the `needs` keyword, and `stage` otherwise. | @@ -11762,6 +12020,7 @@ CI/CD variables for a GitLab instance. | <a id="cijobstatus"></a>`status` | [`CiJobStatus`](#cijobstatus) | Status of the job. | | <a id="cijobstuck"></a>`stuck` | [`Boolean!`](#boolean) | Indicates the job is stuck. | | <a id="cijobtags"></a>`tags` | [`[String!]`](#string) | Tags for the current job. | +| <a id="cijobtrace"></a>`trace` | [`CiJobTrace`](#cijobtrace) | Trace generated by the job. | | <a id="cijobtriggered"></a>`triggered` | [`Boolean`](#boolean) | Whether the job was triggered. | | <a id="cijobuserpermissions"></a>`userPermissions` | [`JobPermissions!`](#jobpermissions) | Permissions for the current user on the resource. | | <a id="cijobwebpath"></a>`webPath` | [`String`](#string) | Web path of the job. | @@ -11789,6 +12048,14 @@ CI/CD variables for a GitLab instance. | <a id="cijobtokenscopetypeoutboundallowlist"></a>`outboundAllowlist` | [`ProjectConnection!`](#projectconnection) | Allow list of projects that are accessible using the current project's CI Job tokens. (see [Connections](#connections)) | | <a id="cijobtokenscopetypeprojects"></a>`projects` **{warning-solid}** | [`ProjectConnection!`](#projectconnection) | **Deprecated** in 15.9. The `projects` attribute is being deprecated. Use `outbound_allowlist`. | +### `CiJobTrace` + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="cijobtracehtmlsummary"></a>`htmlSummary` **{warning-solid}** | [`String!`](#string) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. HTML summary containing the last 10 lines of the trace. | + ### `CiJobsDurationStatistics` Representation of duration statistics for a group of CI jobs. @@ -11797,11 +12064,11 @@ Representation of duration statistics for a group of CI jobs. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="cijobsdurationstatisticsp50"></a>`p50` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is in Alpha. It can be changed or removed at any time. 50th percentile. 50% of the durations are lower than this value. | -| <a id="cijobsdurationstatisticsp75"></a>`p75` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is in Alpha. It can be changed or removed at any time. 75th percentile. 75% of the durations are lower than this value. | -| <a id="cijobsdurationstatisticsp90"></a>`p90` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is in Alpha. It can be changed or removed at any time. 90th percentile. 90% of the durations are lower than this value. | -| <a id="cijobsdurationstatisticsp95"></a>`p95` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is in Alpha. It can be changed or removed at any time. 95th percentile. 95% of the durations are lower than this value. | -| <a id="cijobsdurationstatisticsp99"></a>`p99` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is in Alpha. It can be changed or removed at any time. 99th percentile. 99% of the durations are lower than this value. | +| <a id="cijobsdurationstatisticsp50"></a>`p50` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is an Experiment. It can be changed or removed at any time. 50th percentile. 50% of the durations are lower than this value. | +| <a id="cijobsdurationstatisticsp75"></a>`p75` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is an Experiment. It can be changed or removed at any time. 75th percentile. 75% of the durations are lower than this value. | +| <a id="cijobsdurationstatisticsp90"></a>`p90` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is an Experiment. It can be changed or removed at any time. 90th percentile. 90% of the durations are lower than this value. | +| <a id="cijobsdurationstatisticsp95"></a>`p95` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is an Experiment. It can be changed or removed at any time. 95th percentile. 95% of the durations are lower than this value. | +| <a id="cijobsdurationstatisticsp99"></a>`p99` **{warning-solid}** | [`Duration`](#duration) | **Introduced** in 15.8. This feature is an Experiment. It can be changed or removed at any time. 99th percentile. 99% of the durations are lower than this value. | ### `CiJobsStatistics` @@ -11811,7 +12078,7 @@ Statistics for a group of CI jobs. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="cijobsstatisticsqueuedduration"></a>`queuedDuration` **{warning-solid}** | [`CiJobsDurationStatistics`](#cijobsdurationstatistics) | **Introduced** in 15.8. This feature is in Alpha. It can be changed or removed at any time. Statistics for amount of time that jobs were waiting to be picked up. The calculation is performed based on the most recent 100 jobs executed by the 5000 most recently created runners in context. If no filter is applied to runners, the calculation is performed based on the most recent 100 jobs globally. | +| <a id="cijobsstatisticsqueuedduration"></a>`queuedDuration` **{warning-solid}** | [`CiJobsDurationStatistics`](#cijobsdurationstatistics) | **Introduced** in 15.8. This feature is an Experiment. It can be changed or removed at any time. Statistics for amount of time that jobs were waiting to be picked up. The calculation is performed based on the most recent 100 jobs executed by the 5000 most recently created runners in context. If no filter is applied to runners, the calculation is performed based on the most recent 100 jobs globally. | ### `CiManualVariable` @@ -11883,17 +12150,18 @@ CI/CD variables for a project. | <a id="cirunnercreatedby"></a>`createdBy` | [`UserCore`](#usercore) | User that created this runner. | | <a id="cirunnerdescription"></a>`description` | [`String`](#string) | Description of the runner. | | <a id="cirunnereditadminurl"></a>`editAdminUrl` | [`String`](#string) | Admin form URL of the runner. Only available for administrators. | -| <a id="cirunnerephemeralauthenticationtoken"></a>`ephemeralAuthenticationToken` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. Ephemeral authentication token used for runner machine registration. Only available for the creator of the runner for a limited time during registration. | +| <a id="cirunnerephemeralauthenticationtoken"></a>`ephemeralAuthenticationToken` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. Ephemeral authentication token used for runner manager registration. Only available for the creator of the runner for a limited time during registration. | +| <a id="cirunnerephemeralregisterurl"></a>`ephemeralRegisterUrl` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. URL of the registration page of the runner manager. Only available for the creator of the runner for a limited time during registration. | | <a id="cirunnerexecutorname"></a>`executorName` | [`String`](#string) | Executor last advertised by the runner. | | <a id="cirunnergroups"></a>`groups` | [`GroupConnection`](#groupconnection) | Groups the runner is associated with. For group runners only. (see [Connections](#connections)) | | <a id="cirunnerid"></a>`id` | [`CiRunnerID!`](#cirunnerid) | ID of the runner. | | <a id="cirunneripaddress"></a>`ipAddress` | [`String`](#string) | IP address of the runner. | | <a id="cirunnerjobcount"></a>`jobCount` | [`Int`](#int) | Number of jobs processed by the runner (limited to 1000, plus one to indicate that more items exist). | -| <a id="cirunnerjobexecutionstatus"></a>`jobExecutionStatus` **{warning-solid}** | [`CiRunnerJobExecutionStatus`](#cirunnerjobexecutionstatus) | **Introduced** in 15.7. This feature is in Alpha. It can be changed or removed at any time. Job execution status of the runner. | +| <a id="cirunnerjobexecutionstatus"></a>`jobExecutionStatus` **{warning-solid}** | [`CiRunnerJobExecutionStatus`](#cirunnerjobexecutionstatus) | **Introduced** in 15.7. This feature is an Experiment. It can be changed or removed at any time. Job execution status of the runner. | | <a id="cirunnerlocked"></a>`locked` | [`Boolean`](#boolean) | Indicates the runner is locked. | -| <a id="cirunnermachines"></a>`machines` **{warning-solid}** | [`CiRunnerMachineConnection`](#cirunnermachineconnection) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Machines associated with the runner configuration. | | <a id="cirunnermaintenancenote"></a>`maintenanceNote` | [`String`](#string) | Runner's maintenance notes. | | <a id="cirunnermaintenancenotehtml"></a>`maintenanceNoteHtml` | [`String`](#string) | GitLab Flavored Markdown rendering of `maintenance_note`. | +| <a id="cirunnermanagers"></a>`managers` **{warning-solid}** | [`CiRunnerManagerConnection`](#cirunnermanagerconnection) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Machines associated with the runner configuration. | | <a id="cirunnermaximumtimeout"></a>`maximumTimeout` | [`Int`](#int) | Maximum timeout (in seconds) for jobs processed by the runner. | | <a id="cirunnerownerproject"></a>`ownerProject` | [`Project`](#project) | Project that owns the runner. For project runners only. | | <a id="cirunnerpaused"></a>`paused` | [`Boolean!`](#boolean) | Indicates the runner is paused and not available to run jobs. | @@ -11908,7 +12176,7 @@ CI/CD variables for a project. | <a id="cirunnershortsha"></a>`shortSha` | [`String`](#string) | First eight characters of the runner's token used to authenticate new job requests. Used as the runner's unique ID. | | <a id="cirunnertaglist"></a>`tagList` | [`[String!]`](#string) | Tags associated with the runner. | | <a id="cirunnertokenexpiresat"></a>`tokenExpiresAt` | [`Time`](#time) | Runner token expiration time. | -| <a id="cirunnerupgradestatus"></a>`upgradeStatus` **{warning-solid}** | [`CiRunnerUpgradeStatus`](#cirunnerupgradestatus) | **Introduced** in 14.10. This feature is in Alpha. It can be changed or removed at any time. Availability of upgrades for the runner. | +| <a id="cirunnerupgradestatus"></a>`upgradeStatus` **{warning-solid}** | [`CiRunnerUpgradeStatus`](#cirunnerupgradestatus) | **Introduced** in 14.10. This feature is an Experiment. It can be changed or removed at any time. Availability of upgrades for the runner. | | <a id="cirunneruserpermissions"></a>`userPermissions` | [`RunnerPermissions!`](#runnerpermissions) | Permissions for the current user on the resource. | | <a id="cirunnerversion"></a>`version` | [`String`](#string) | Version of the runner. | @@ -11962,24 +12230,24 @@ Returns [`CiRunnerStatus!`](#cirunnerstatus). | ---- | ---- | ----------- | | <a id="cirunnerstatuslegacymode"></a>`legacyMode` **{warning-solid}** | [`String`](#string) | **Deprecated** in 15.0. Will be removed in 17.0. In GitLab 16.0 and later, the field will act as if `legacyMode` is null. | -### `CiRunnerMachine` +### `CiRunnerManager` #### Fields | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="cirunnermachinearchitecturename"></a>`architectureName` | [`String`](#string) | Architecture provided by the runner machine. | -| <a id="cirunnermachinecontactedat"></a>`contactedAt` | [`Time`](#time) | Timestamp of last contact from the runner machine. | -| <a id="cirunnermachinecreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of creation of the runner machine. | -| <a id="cirunnermachineexecutorname"></a>`executorName` | [`String`](#string) | Executor last advertised by the runner. | -| <a id="cirunnermachineid"></a>`id` | [`CiRunnerMachineID!`](#cirunnermachineid) | ID of the runner machine. | -| <a id="cirunnermachineipaddress"></a>`ipAddress` | [`String`](#string) | IP address of the runner machine. | -| <a id="cirunnermachineplatformname"></a>`platformName` | [`String`](#string) | Platform provided by the runner machine. | -| <a id="cirunnermachinerevision"></a>`revision` | [`String`](#string) | Revision of the runner. | -| <a id="cirunnermachinerunner"></a>`runner` | [`CiRunner`](#cirunner) | Runner configuration for the runner machine. | -| <a id="cirunnermachinestatus"></a>`status` | [`CiRunnerStatus!`](#cirunnerstatus) | Status of the runner machine. | -| <a id="cirunnermachinesystemid"></a>`systemId` | [`String!`](#string) | System ID associated with the runner machine. | -| <a id="cirunnermachineversion"></a>`version` | [`String`](#string) | Version of the runner. | +| <a id="cirunnermanagerarchitecturename"></a>`architectureName` | [`String`](#string) | Architecture provided by the runner manager. | +| <a id="cirunnermanagercontactedat"></a>`contactedAt` | [`Time`](#time) | Timestamp of last contact from the runner manager. | +| <a id="cirunnermanagercreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of creation of the runner manager. | +| <a id="cirunnermanagerexecutorname"></a>`executorName` | [`String`](#string) | Executor last advertised by the runner. | +| <a id="cirunnermanagerid"></a>`id` | [`CiRunnerManagerID!`](#cirunnermanagerid) | ID of the runner manager. | +| <a id="cirunnermanageripaddress"></a>`ipAddress` | [`String`](#string) | IP address of the runner manager. | +| <a id="cirunnermanagerplatformname"></a>`platformName` | [`String`](#string) | Platform provided by the runner manager. | +| <a id="cirunnermanagerrevision"></a>`revision` | [`String`](#string) | Revision of the runner. | +| <a id="cirunnermanagerrunner"></a>`runner` | [`CiRunner`](#cirunner) | Runner configuration for the runner manager. | +| <a id="cirunnermanagerstatus"></a>`status` | [`CiRunnerStatus!`](#cirunnerstatus) | Status of the runner manager. | +| <a id="cirunnermanagersystemid"></a>`systemId` | [`String!`](#string) | System ID associated with the runner manager. | +| <a id="cirunnermanagerversion"></a>`version` | [`String`](#string) | Version of the runner. | ### `CiSecureFileRegistry` @@ -12194,6 +12462,15 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="commitpipelinesupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Pipelines updated before this date. | | <a id="commitpipelinesusername"></a>`username` | [`String`](#string) | Filter pipelines by the user that triggered the pipeline. | +### `CommitParentNames` + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="commitparentnamesnames"></a>`names` | [`[String!]`](#string) | Names of the commit parent (branch or tag). | +| <a id="commitparentnamestotalcount"></a>`totalCount` | [`Int`](#int) | Total of parent branches or tags. | + ### `ComplianceFramework` Represents a ComplianceFramework associated with a Project. @@ -12809,12 +13086,14 @@ The deployment of an environment. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="deploymentapprovalsummary"></a>`approvalSummary` | [`DeploymentApprovalSummary`](#deploymentapprovalsummary) | Approval summary of the deployment.This field can only be resolved for one deployment in any single request. | +| <a id="deploymentapprovals"></a>`approvals` | [`[DeploymentApproval!]`](#deploymentapproval) | Current approvals of the deployment. | | <a id="deploymentcommit"></a>`commit` | [`Commit`](#commit) | Commit details of the deployment. | | <a id="deploymentcreatedat"></a>`createdAt` | [`Time`](#time) | When the deployment record was created. | | <a id="deploymentfinishedat"></a>`finishedAt` | [`Time`](#time) | When the deployment finished. | | <a id="deploymentid"></a>`id` | [`ID`](#id) | Global ID of the deployment. | | <a id="deploymentiid"></a>`iid` | [`ID`](#id) | Project-level internal ID of the deployment. | | <a id="deploymentjob"></a>`job` | [`CiJob`](#cijob) | Pipeline job of the deployment. | +| <a id="deploymentpendingapprovalcount"></a>`pendingApprovalCount` | [`Int`](#int) | Number of pending unified approvals on the deployment. | | <a id="deploymentref"></a>`ref` | [`String`](#string) | Git-Ref that the deployment ran on. | | <a id="deploymentsha"></a>`sha` | [`String`](#string) | Git-SHA that the deployment ran on. | | <a id="deploymentstatus"></a>`status` | [`DeploymentStatus`](#deploymentstatus) | Status of the deployment. | @@ -13330,7 +13609,7 @@ Returns [`[DoraMetric!]`](#dorametric). | <a id="egressnodeartifactsegress"></a>`artifactsEgress` | [`BigInt!`](#bigint) | Artifacts egress for that project in that period of time. | | <a id="egressnodedate"></a>`date` | [`String!`](#string) | First day of the node range. There is one node per month. | | <a id="egressnodepackagesegress"></a>`packagesEgress` | [`BigInt!`](#bigint) | Packages egress for that project in that period of time. | -| <a id="egressnoderegistryegress"></a>`registryEgress` | [`BigInt!`](#bigint) | Registery egress for that project in that period of time. | +| <a id="egressnoderegistryegress"></a>`registryEgress` | [`BigInt!`](#bigint) | Registry egress for that project in that period of time. | | <a id="egressnoderepositoryegress"></a>`repositoryEgress` | [`BigInt!`](#bigint) | Repository egress for that project in that period of time. | | <a id="egressnodetotalegress"></a>`totalEgress` | [`BigInt!`](#bigint) | Total egress for that project in that period of time. | @@ -13519,7 +13798,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="epicancestorsmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Filter epics by milestone title, computed from epic's issues. | | <a id="epicancestorsmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. | | <a id="epicancestorsnot"></a>`not` | [`NegatedEpicFilterInput`](#negatedepicfilterinput) | Negated epic arguments. | -| <a id="epicancestorsor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | +| <a id="epicancestorsor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | | <a id="epicancestorssearch"></a>`search` | [`String`](#string) | Search query for title or description. | | <a id="epicancestorssort"></a>`sort` | [`EpicSort`](#epicsort) | List epics by sort order. | | <a id="epicancestorsstartdate"></a>`startDate` **{warning-solid}** | [`Time`](#time) | **Deprecated** in 13.5. Use timeframe.start. | @@ -13558,7 +13837,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="epicchildrenmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Filter epics by milestone title, computed from epic's issues. | | <a id="epicchildrenmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. | | <a id="epicchildrennot"></a>`not` | [`NegatedEpicFilterInput`](#negatedepicfilterinput) | Negated epic arguments. | -| <a id="epicchildrenor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | +| <a id="epicchildrenor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | | <a id="epicchildrensearch"></a>`search` | [`String`](#string) | Search query for title or description. | | <a id="epicchildrensort"></a>`sort` | [`EpicSort`](#epicsort) | List epics by sort order. | | <a id="epicchildrenstartdate"></a>`startDate` **{warning-solid}** | [`Time`](#time) | **Deprecated** in 13.5. Use timeframe.start. | @@ -13855,7 +14134,7 @@ Represents epic board list metadata. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="epiclistmetadataepicscount"></a>`epicsCount` | [`Int`](#int) | Count of epics in the list. | -| <a id="epiclistmetadatatotalweight"></a>`totalWeight` **{warning-solid}** | [`Int`](#int) | **Introduced** in 14.7. This feature is in Alpha. It can be changed or removed at any time. Total weight of all issues in the list. | +| <a id="epiclistmetadatatotalweight"></a>`totalWeight` **{warning-solid}** | [`Int`](#int) | **Introduced** in 14.7. This feature is an Experiment. It can be changed or removed at any time. Total weight of all issues in the list. | ### `EpicPermissions` @@ -14049,7 +14328,7 @@ Find Dependency Proxy Blob registries on this Geo node. WARNING: **Introduced** in 15.6. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`DependencyProxyBlobRegistryConnection`](#dependencyproxyblobregistryconnection). @@ -14218,6 +14497,25 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="geonodepipelineartifactregistriesreplicationstate"></a>`replicationState` | [`ReplicationStateEnum`](#replicationstateenum) | Filters registries by their replication state. | | <a id="geonodepipelineartifactregistriesverificationstate"></a>`verificationState` | [`VerificationStateEnum`](#verificationstateenum) | Filters registries by their verification state. | +##### `GeoNode.projectWikiRepositoryRegistries` + +Find Project Wiki Repository registries on this Geo node. Ignored if `geo_project_wiki_repository_replication` feature flag is disabled. + +Returns [`ProjectWikiRepositoryRegistryConnection`](#projectwikirepositoryregistryconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="geonodeprojectwikirepositoryregistriesids"></a>`ids` | [`[ID!]`](#id) | Filters registries by their ID. | +| <a id="geonodeprojectwikirepositoryregistrieskeyword"></a>`keyword` | [`String`](#string) | Filters registries by their attributes using a keyword. | +| <a id="geonodeprojectwikirepositoryregistriesreplicationstate"></a>`replicationState` | [`ReplicationStateEnum`](#replicationstateenum) | Filters registries by their replication state. | +| <a id="geonodeprojectwikirepositoryregistriesverificationstate"></a>`verificationState` | [`VerificationStateEnum`](#verificationstateenum) | Filters registries by their verification state. | + ##### `GeoNode.snippetRepositoryRegistries` Find snippet repository registries on this Geo node. @@ -14309,7 +14607,6 @@ GPG signature for a signed commit. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="groupachievements"></a>`achievements` **{warning-solid}** | [`AchievementConnection`](#achievementconnection) | **Introduced** in 15.8. This feature is in Alpha. It can be changed or removed at any time. Achievements for the namespace. Returns `null` if the `achievements` feature flag is disabled. | | <a id="groupactualrepositorysizelimit"></a>`actualRepositorySizeLimit` | [`Float`](#float) | Size limit for repositories in the namespace in bytes. | | <a id="groupadditionalpurchasedstoragesize"></a>`additionalPurchasedStorageSize` | [`Float`](#float) | Additional storage purchased for the root namespace in bytes. | | <a id="groupallowstalerunnerpruning"></a>`allowStaleRunnerPruning` | [`Boolean!`](#boolean) | Indicates whether to regularly prune stale group runners. Defaults to false. | @@ -14318,7 +14615,7 @@ GPG signature for a signed commit. | <a id="groupcontainerrepositoriescount"></a>`containerRepositoriesCount` | [`Int!`](#int) | Number of container repositories in the group. | | <a id="groupcontainslockedprojects"></a>`containsLockedProjects` | [`Boolean!`](#boolean) | Includes at least one project where the repository size exceeds the limit. | | <a id="groupcrossprojectpipelineavailable"></a>`crossProjectPipelineAvailable` | [`Boolean!`](#boolean) | Indicates if the cross_project_pipeline feature is available for the namespace. | -| <a id="groupcustomemoji"></a>`customEmoji` **{warning-solid}** | [`CustomEmojiConnection`](#customemojiconnection) | **Introduced** in 13.6. This feature is in Alpha. It can be changed or removed at any time. Custom emoji within this namespace. | +| <a id="groupcustomemoji"></a>`customEmoji` **{warning-solid}** | [`CustomEmojiConnection`](#customemojiconnection) | **Introduced** in 13.6. This feature is an Experiment. It can be changed or removed at any time. Custom emoji within this namespace. | | <a id="groupdependencyproxyblobcount"></a>`dependencyProxyBlobCount` | [`Int!`](#int) | Number of dependency proxy blobs cached in the group. | | <a id="groupdependencyproxyblobs"></a>`dependencyProxyBlobs` | [`DependencyProxyBlobConnection`](#dependencyproxyblobconnection) | Dependency Proxy blobs. (see [Connections](#connections)) | | <a id="groupdependencyproxyimagecount"></a>`dependencyProxyImageCount` | [`Int!`](#int) | Number of dependency proxy images cached in the group. | @@ -14335,7 +14632,7 @@ GPG signature for a signed commit. | <a id="groupepicboards"></a>`epicBoards` | [`EpicBoardConnection`](#epicboardconnection) | Find epic boards. (see [Connections](#connections)) | | <a id="groupepicsenabled"></a>`epicsEnabled` | [`Boolean`](#boolean) | Indicates if Epics are enabled for namespace. | | <a id="groupexternalauditeventdestinations"></a>`externalAuditEventDestinations` | [`ExternalAuditEventDestinationConnection`](#externalauditeventdestinationconnection) | External locations that receive audit events belonging to the group. (see [Connections](#connections)) | -| <a id="groupflowmetrics"></a>`flowMetrics` **{warning-solid}** | [`GroupValueStreamAnalyticsFlowMetrics`](#groupvaluestreamanalyticsflowmetrics) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Flow metrics for value stream analytics. | +| <a id="groupflowmetrics"></a>`flowMetrics` **{warning-solid}** | [`GroupValueStreamAnalyticsFlowMetrics`](#groupvaluestreamanalyticsflowmetrics) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Flow metrics for value stream analytics. | | <a id="groupfullname"></a>`fullName` | [`String!`](#string) | Full name of the namespace. | | <a id="groupfullpath"></a>`fullPath` | [`ID!`](#id) | Full path of the namespace. | | <a id="groupid"></a>`id` | [`ID!`](#id) | ID of the namespace. | @@ -14358,7 +14655,7 @@ GPG signature for a signed commit. | <a id="groupstoragesizelimit"></a>`storageSizeLimit` | [`Float`](#float) | Total storage limit of the root namespace in bytes. | | <a id="groupsubgroupcreationlevel"></a>`subgroupCreationLevel` | [`String`](#string) | Permission level required to create subgroups within the group. | | <a id="grouptemporarystorageincreaseendson"></a>`temporaryStorageIncreaseEndsOn` | [`Time`](#time) | Date until the temporary storage increase is active. | -| <a id="grouptimelogcategories"></a>`timelogCategories` **{warning-solid}** | [`TimeTrackingTimelogCategoryConnection`](#timetrackingtimelogcategoryconnection) | **Introduced** in 15.3. This feature is in Alpha. It can be changed or removed at any time. Timelog categories for the namespace. | +| <a id="grouptimelogcategories"></a>`timelogCategories` **{warning-solid}** | [`TimeTrackingTimelogCategoryConnection`](#timetrackingtimelogcategoryconnection) | **Introduced** in 15.3. This feature is an Experiment. It can be changed or removed at any time. Timelog categories for the namespace. | | <a id="grouptotalrepositorysize"></a>`totalRepositorySize` | [`Float`](#float) | Total repository size of all projects in the root namespace in bytes. | | <a id="grouptotalrepositorysizeexcess"></a>`totalRepositorySizeExcess` | [`Float`](#float) | Total excess repository size of all projects in the root namespace in bytes. | | <a id="grouptwofactorgraceperiod"></a>`twoFactorGracePeriod` | [`Int`](#int) | Time before two-factor authentication is enforced. | @@ -14369,6 +14666,26 @@ GPG signature for a signed commit. #### Fields with arguments +##### `Group.achievements` + +Achievements for the namespace. Returns `null` if the `achievements` feature flag is disabled. + +WARNING: +**Introduced** in 15.8. +This feature is an Experiment. It can be changed or removed at any time. + +Returns [`AchievementConnection`](#achievementconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="groupachievementsids"></a>`ids` | [`[AchievementsAchievementID!]`](#achievementsachievementid) | Filter achievements by IDs. | + ##### `Group.billableMembersCount` Number of billable users in the group. @@ -14549,7 +14866,7 @@ Returns [`GroupDataTransfer`](#groupdatatransfer). | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="groupdatatransferfrom"></a>`from` | [`Date`](#date) | Retain egress data for 1 year. Current month will increase dynamically as egress occurs. | +| <a id="groupdatatransferfrom"></a>`from` | [`Date`](#date) | Retain egress data for one year. Data for the current month will increase dynamically as egress occurs. | | <a id="groupdatatransferto"></a>`to` | [`Date`](#date) | End date for the data. | ##### `Group.descendantGroups` @@ -14595,7 +14912,7 @@ Returns [`Epic`](#epic). | <a id="groupepicmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Filter epics by milestone title, computed from epic's issues. | | <a id="groupepicmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. | | <a id="groupepicnot"></a>`not` | [`NegatedEpicFilterInput`](#negatedepicfilterinput) | Negated epic arguments. | -| <a id="groupepicor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | +| <a id="groupepicor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | | <a id="groupepicsearch"></a>`search` | [`String`](#string) | Search query for title or description. | | <a id="groupepicsort"></a>`sort` | [`EpicSort`](#epicsort) | List epics by sort order. | | <a id="groupepicstartdate"></a>`startDate` **{warning-solid}** | [`Time`](#time) | **Deprecated** in 13.5. Use timeframe.start. | @@ -14646,7 +14963,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupepicsmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Filter epics by milestone title, computed from epic's issues. | | <a id="groupepicsmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. | | <a id="groupepicsnot"></a>`not` | [`NegatedEpicFilterInput`](#negatedepicfilterinput) | Negated epic arguments. | -| <a id="groupepicsor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | +| <a id="groupepicsor"></a>`or` **{warning-solid}** | [`UnionedEpicFilterInput`](#unionedepicfilterinput) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. List of arguments with inclusive OR. Ignored unless `or_issuable_queries` flag is enabled. | | <a id="groupepicssearch"></a>`search` | [`String`](#string) | Search query for title or description. | | <a id="groupepicssort"></a>`sort` | [`EpicSort`](#epicsort) | List epics by sort order. | | <a id="groupepicsstartdate"></a>`startDate` **{warning-solid}** | [`Time`](#time) | **Deprecated** in 13.5. Use timeframe.start. | @@ -14850,6 +15167,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="groupmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="groupmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="groupmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="groupmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | @@ -15262,11 +15580,11 @@ Returns [`ValueStreamAnalyticsMetric`](#valuestreamanalyticsmetric). | ---- | ---- | ----------- | | <a id="groupvaluestreamanalyticsflowmetricscycletimeassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | | <a id="groupvaluestreamanalyticsflowmetricscycletimeauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | -| <a id="groupvaluestreamanalyticsflowmetricscycletimefrom"></a>`from` | [`Time!`](#time) | Issues created after the date. | +| <a id="groupvaluestreamanalyticsflowmetricscycletimefrom"></a>`from` | [`Time!`](#time) | After the date. | | <a id="groupvaluestreamanalyticsflowmetricscycletimelabelnames"></a>`labelNames` | [`[String!]`](#string) | Labels applied to the issue. | | <a id="groupvaluestreamanalyticsflowmetricscycletimemilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Milestone applied to the issue. | | <a id="groupvaluestreamanalyticsflowmetricscycletimeprojectids"></a>`projectIds` | [`[ID!]`](#id) | Project IDs within the group hierarchy. | -| <a id="groupvaluestreamanalyticsflowmetricscycletimeto"></a>`to` | [`Time!`](#time) | Issues created before the date. | +| <a id="groupvaluestreamanalyticsflowmetricscycletimeto"></a>`to` | [`Time!`](#time) | Before the date. | ##### `GroupValueStreamAnalyticsFlowMetrics.deploymentCount` @@ -15278,9 +15596,9 @@ Returns [`ValueStreamAnalyticsMetric`](#valuestreamanalyticsmetric). | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="groupvaluestreamanalyticsflowmetricsdeploymentcountfrom"></a>`from` | [`Time!`](#time) | Deployments finished after the date. | +| <a id="groupvaluestreamanalyticsflowmetricsdeploymentcountfrom"></a>`from` | [`Time!`](#time) | After the date. | | <a id="groupvaluestreamanalyticsflowmetricsdeploymentcountprojectids"></a>`projectIds` | [`[ID!]`](#id) | Project IDs within the group hierarchy. | -| <a id="groupvaluestreamanalyticsflowmetricsdeploymentcountto"></a>`to` | [`Time!`](#time) | Deployments finished before the date. | +| <a id="groupvaluestreamanalyticsflowmetricsdeploymentcountto"></a>`to` | [`Time!`](#time) | Before the date. | ##### `GroupValueStreamAnalyticsFlowMetrics.issueCount` @@ -15294,11 +15612,11 @@ Returns [`ValueStreamAnalyticsMetric`](#valuestreamanalyticsmetric). | ---- | ---- | ----------- | | <a id="groupvaluestreamanalyticsflowmetricsissuecountassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | | <a id="groupvaluestreamanalyticsflowmetricsissuecountauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | -| <a id="groupvaluestreamanalyticsflowmetricsissuecountfrom"></a>`from` | [`Time!`](#time) | Issues created after the date. | +| <a id="groupvaluestreamanalyticsflowmetricsissuecountfrom"></a>`from` | [`Time!`](#time) | After the date. | | <a id="groupvaluestreamanalyticsflowmetricsissuecountlabelnames"></a>`labelNames` | [`[String!]`](#string) | Labels applied to the issue. | | <a id="groupvaluestreamanalyticsflowmetricsissuecountmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Milestone applied to the issue. | | <a id="groupvaluestreamanalyticsflowmetricsissuecountprojectids"></a>`projectIds` | [`[ID!]`](#id) | Project IDs within the group hierarchy. | -| <a id="groupvaluestreamanalyticsflowmetricsissuecountto"></a>`to` | [`Time!`](#time) | Issues created before the date. | +| <a id="groupvaluestreamanalyticsflowmetricsissuecountto"></a>`to` | [`Time!`](#time) | Before the date. | ##### `GroupValueStreamAnalyticsFlowMetrics.leadTime` @@ -15312,11 +15630,11 @@ Returns [`ValueStreamAnalyticsMetric`](#valuestreamanalyticsmetric). | ---- | ---- | ----------- | | <a id="groupvaluestreamanalyticsflowmetricsleadtimeassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | | <a id="groupvaluestreamanalyticsflowmetricsleadtimeauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | -| <a id="groupvaluestreamanalyticsflowmetricsleadtimefrom"></a>`from` | [`Time!`](#time) | Issues created after the date. | +| <a id="groupvaluestreamanalyticsflowmetricsleadtimefrom"></a>`from` | [`Time!`](#time) | After the date. | | <a id="groupvaluestreamanalyticsflowmetricsleadtimelabelnames"></a>`labelNames` | [`[String!]`](#string) | Labels applied to the issue. | | <a id="groupvaluestreamanalyticsflowmetricsleadtimemilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Milestone applied to the issue. | | <a id="groupvaluestreamanalyticsflowmetricsleadtimeprojectids"></a>`projectIds` | [`[ID!]`](#id) | Project IDs within the group hierarchy. | -| <a id="groupvaluestreamanalyticsflowmetricsleadtimeto"></a>`to` | [`Time!`](#time) | Issues created before the date. | +| <a id="groupvaluestreamanalyticsflowmetricsleadtimeto"></a>`to` | [`Time!`](#time) | Before the date. | ### `GroupWikiRepositoryRegistry` @@ -15970,6 +16288,7 @@ Defines which user roles, users, or groups can merge into a protected branch. | <a id="mergerequestautomergeenabled"></a>`autoMergeEnabled` | [`Boolean!`](#boolean) | Indicates if auto merge is enabled for the merge request. | | <a id="mergerequestautomergestrategy"></a>`autoMergeStrategy` | [`String`](#string) | Selected auto merge strategy. | | <a id="mergerequestavailableautomergestrategies"></a>`availableAutoMergeStrategies` | [`[String!]`](#string) | Array of available auto merge strategies. | +| <a id="mergerequestawardemoji"></a>`awardEmoji` | [`AwardEmojiConnection`](#awardemojiconnection) | List of award emojis associated with the merge request. (see [Connections](#connections)) | | <a id="mergerequestcommenters"></a>`commenters` | [`UserCoreConnection!`](#usercoreconnection) | All commenters on this noteable. (see [Connections](#connections)) | | <a id="mergerequestcommitcount"></a>`commitCount` | [`Int`](#int) | Number of commits in the merge request. | | <a id="mergerequestcommits"></a>`commits` | [`CommitConnection`](#commitconnection) | Merge request commits. (see [Connections](#connections)) | @@ -16032,7 +16351,7 @@ Defines which user roles, users, or groups can merge into a protected branch. | <a id="mergerequestsquashonmerge"></a>`squashOnMerge` | [`Boolean!`](#boolean) | Indicates if the merge request will be squashed when merged. | | <a id="mergerequeststate"></a>`state` | [`MergeRequestState!`](#mergerequeststate) | State of the merge request. | | <a id="mergerequestsubscribed"></a>`subscribed` | [`Boolean!`](#boolean) | Indicates if the currently logged in user is subscribed to this merge request. | -| <a id="mergerequestsuggestedreviewers"></a>`suggestedReviewers` **{warning-solid}** | [`SuggestedReviewersType`](#suggestedreviewerstype) | **Introduced** in 15.4. This feature is in Alpha. It can be changed or removed at any time. Suggested reviewers for merge request. Returns `null` if `suggested_reviewers` feature flag is disabled. This flag is disabled by default and only available on GitLab.com because the feature is experimental and is subject to change without notice. | +| <a id="mergerequestsuggestedreviewers"></a>`suggestedReviewers` **{warning-solid}** | [`SuggestedReviewersType`](#suggestedreviewerstype) | **Introduced** in 15.4. This feature is an Experiment. It can be changed or removed at any time. Suggested reviewers for merge request. Returns `null` if `suggested_reviewers` feature flag is disabled. This flag is disabled by default and only available on GitLab.com because the feature is experimental and is subject to change without notice. | | <a id="mergerequesttargetbranch"></a>`targetBranch` | [`String!`](#string) | Target branch of the merge request. | | <a id="mergerequesttargetbranchexists"></a>`targetBranchExists` | [`Boolean!`](#boolean) | Indicates if the target branch of the merge request exists. | | <a id="mergerequesttargetproject"></a>`targetProject` | [`Project!`](#project) | Target project of the merge request. | @@ -16158,7 +16477,7 @@ A user assigned to a merge request. | <a id="mergerequestassigneesavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="mergerequestassigneestate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="mergerequestassigneestatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | -| <a id="mergerequestassigneeuserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | +| <a id="mergerequestassigneeuserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | | <a id="mergerequestassigneeuserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | | <a id="mergerequestassigneeusername"></a>`username` | [`String!`](#string) | Username of the user. Unique within this instance of GitLab. | | <a id="mergerequestassigneewebpath"></a>`webPath` | [`String!`](#string) | Web path of the user. | @@ -16180,6 +16499,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestassigneeassignedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestassigneeassignedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="mergerequestassigneeassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestassigneeassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -16214,6 +16534,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestassigneeauthoredmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestassigneeauthoredmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="mergerequestassigneeauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestassigneeauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -16265,6 +16586,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestassigneereviewrequestedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestassigneereviewrequestedmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="mergerequestassigneereviewrequestedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="mergerequestassigneereviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | @@ -16405,7 +16727,7 @@ The author of the merge request. | <a id="mergerequestauthorsavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="mergerequestauthorstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="mergerequestauthorstatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | -| <a id="mergerequestauthoruserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | +| <a id="mergerequestauthoruserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | | <a id="mergerequestauthoruserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | | <a id="mergerequestauthorusername"></a>`username` | [`String!`](#string) | Username of the user. Unique within this instance of GitLab. | | <a id="mergerequestauthorwebpath"></a>`webPath` | [`String!`](#string) | Web path of the user. | @@ -16427,6 +16749,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestauthorassignedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestauthorassignedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="mergerequestauthorassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestauthorassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -16461,6 +16784,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestauthorauthoredmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestauthorauthoredmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="mergerequestauthorauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestauthorauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -16512,6 +16836,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestauthorreviewrequestedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestauthorreviewrequestedmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="mergerequestauthorreviewrequestedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="mergerequestauthorreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | @@ -16671,7 +16996,7 @@ A user participating in a merge request. | <a id="mergerequestparticipantsavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="mergerequestparticipantstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="mergerequestparticipantstatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | -| <a id="mergerequestparticipantuserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | +| <a id="mergerequestparticipantuserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | | <a id="mergerequestparticipantuserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | | <a id="mergerequestparticipantusername"></a>`username` | [`String!`](#string) | Username of the user. Unique within this instance of GitLab. | | <a id="mergerequestparticipantwebpath"></a>`webPath` | [`String!`](#string) | Web path of the user. | @@ -16693,6 +17018,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestparticipantassignedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestparticipantassignedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="mergerequestparticipantassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestparticipantassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -16727,6 +17053,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestparticipantauthoredmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestparticipantauthoredmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="mergerequestparticipantauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestparticipantauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -16778,6 +17105,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestparticipantreviewrequestedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestparticipantreviewrequestedmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="mergerequestparticipantreviewrequestedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="mergerequestparticipantreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | @@ -16937,7 +17265,7 @@ A user assigned to a merge request as a reviewer. | <a id="mergerequestreviewersavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="mergerequestreviewerstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="mergerequestreviewerstatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | -| <a id="mergerequestrevieweruserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | +| <a id="mergerequestrevieweruserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | | <a id="mergerequestrevieweruserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | | <a id="mergerequestreviewerusername"></a>`username` | [`String!`](#string) | Username of the user. Unique within this instance of GitLab. | | <a id="mergerequestreviewerwebpath"></a>`webPath` | [`String!`](#string) | Web path of the user. | @@ -16959,6 +17287,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestreviewerassignedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestreviewerassignedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="mergerequestreviewerassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestreviewerassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -16993,6 +17322,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestreviewerauthoredmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestreviewerauthoredmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="mergerequestreviewerauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestreviewerauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -17044,6 +17374,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mergerequestreviewerreviewrequestedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="mergerequestreviewerreviewrequestedmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="mergerequestreviewerreviewrequestedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="mergerequestreviewerreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | @@ -17275,7 +17606,6 @@ Contains statistics about a milestone. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="namespaceachievements"></a>`achievements` **{warning-solid}** | [`AchievementConnection`](#achievementconnection) | **Introduced** in 15.8. This feature is in Alpha. It can be changed or removed at any time. Achievements for the namespace. Returns `null` if the `achievements` feature flag is disabled. | | <a id="namespaceactualrepositorysizelimit"></a>`actualRepositorySizeLimit` | [`Float`](#float) | Size limit for repositories in the namespace in bytes. | | <a id="namespaceadditionalpurchasedstoragesize"></a>`additionalPurchasedStorageSize` | [`Float`](#float) | Additional storage purchased for the root namespace in bytes. | | <a id="namespacecontainslockedprojects"></a>`containsLockedProjects` | [`Boolean!`](#boolean) | Includes at least one project where the repository size exceeds the limit. | @@ -17296,13 +17626,33 @@ Contains statistics about a milestone. | <a id="namespacesharedrunnerssetting"></a>`sharedRunnersSetting` | [`SharedRunnersSetting`](#sharedrunnerssetting) | Shared runners availability for the namespace and its descendants. | | <a id="namespacestoragesizelimit"></a>`storageSizeLimit` | [`Float`](#float) | Total storage limit of the root namespace in bytes. | | <a id="namespacetemporarystorageincreaseendson"></a>`temporaryStorageIncreaseEndsOn` | [`Time`](#time) | Date until the temporary storage increase is active. | -| <a id="namespacetimelogcategories"></a>`timelogCategories` **{warning-solid}** | [`TimeTrackingTimelogCategoryConnection`](#timetrackingtimelogcategoryconnection) | **Introduced** in 15.3. This feature is in Alpha. It can be changed or removed at any time. Timelog categories for the namespace. | +| <a id="namespacetimelogcategories"></a>`timelogCategories` **{warning-solid}** | [`TimeTrackingTimelogCategoryConnection`](#timetrackingtimelogcategoryconnection) | **Introduced** in 15.3. This feature is an Experiment. It can be changed or removed at any time. Timelog categories for the namespace. | | <a id="namespacetotalrepositorysize"></a>`totalRepositorySize` | [`Float`](#float) | Total repository size of all projects in the root namespace in bytes. | | <a id="namespacetotalrepositorysizeexcess"></a>`totalRepositorySizeExcess` | [`Float`](#float) | Total excess repository size of all projects in the root namespace in bytes. | | <a id="namespacevisibility"></a>`visibility` | [`String`](#string) | Visibility of the namespace. | #### Fields with arguments +##### `Namespace.achievements` + +Achievements for the namespace. Returns `null` if the `achievements` feature flag is disabled. + +WARNING: +**Introduced** in 15.8. +This feature is an Experiment. It can be changed or removed at any time. + +Returns [`AchievementConnection`](#achievementconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="namespaceachievementsids"></a>`ids` | [`[AchievementsAchievementID!]`](#achievementsachievementid) | Filter achievements by IDs. | + ##### `Namespace.complianceFrameworks` Compliance frameworks available to projects in this namespace. @@ -18212,11 +18562,11 @@ Represents a product analytics dashboard visualization. | <a id="projectcreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of the project creation. | | <a id="projectdastscannerprofiles"></a>`dastScannerProfiles` | [`DastScannerProfileConnection`](#dastscannerprofileconnection) | DAST scanner profiles associated with the project. (see [Connections](#connections)) | | <a id="projectdastsiteprofiles"></a>`dastSiteProfiles` | [`DastSiteProfileConnection`](#dastsiteprofileconnection) | DAST Site Profiles associated with the project. (see [Connections](#connections)) | -| <a id="projectdependencies"></a>`dependencies` **{warning-solid}** | [`DependencyConnection`](#dependencyconnection) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. Software dependencies used by the project. | +| <a id="projectdependencies"></a>`dependencies` **{warning-solid}** | [`DependencyConnection`](#dependencyconnection) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. Software dependencies used by the project. | | <a id="projectdescription"></a>`description` | [`String`](#string) | Short description of the project. | | <a id="projectdescriptionhtml"></a>`descriptionHtml` | [`String`](#string) | GitLab Flavored Markdown rendering of `description`. | | <a id="projectdora"></a>`dora` | [`Dora`](#dora) | Project's DORA metrics. | -| <a id="projectflowmetrics"></a>`flowMetrics` **{warning-solid}** | [`ProjectValueStreamAnalyticsFlowMetrics`](#projectvaluestreamanalyticsflowmetrics) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Flow metrics for value stream analytics. | +| <a id="projectflowmetrics"></a>`flowMetrics` **{warning-solid}** | [`ProjectValueStreamAnalyticsFlowMetrics`](#projectvaluestreamanalyticsflowmetrics) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Flow metrics for value stream analytics. | | <a id="projectforkscount"></a>`forksCount` | [`Int!`](#int) | Number of times the project has been forked. | | <a id="projectfullpath"></a>`fullPath` | [`ID!`](#id) | Full path of the project. | | <a id="projectgrafanaintegration"></a>`grafanaIntegration` | [`GrafanaIntegration`](#grafanaintegration) | Grafana integration details for the project. | @@ -18225,10 +18575,11 @@ Represents a product analytics dashboard visualization. | <a id="projectid"></a>`id` | [`ID!`](#id) | ID of the project. | | <a id="projectimportstatus"></a>`importStatus` | [`String`](#string) | Status of import background job of the project. | | <a id="projectincidentmanagementtimelineeventtags"></a>`incidentManagementTimelineEventTags` | [`[TimelineEventTagType!]`](#timelineeventtagtype) | Timeline event tags for the project. | +| <a id="projectiscatalogresource"></a>`isCatalogResource` **{warning-solid}** | [`Boolean`](#boolean) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. Indicates if a project is a catalog resource. | | <a id="projectissuesenabled"></a>`issuesEnabled` | [`Boolean`](#boolean) | Indicates if Issues are enabled for the current user. | | <a id="projectjiraimportstatus"></a>`jiraImportStatus` | [`String`](#string) | Status of Jira import background job of the project. | | <a id="projectjiraimports"></a>`jiraImports` | [`JiraImportConnection`](#jiraimportconnection) | Jira imports into the project. (see [Connections](#connections)) | -| <a id="projectjitsukey"></a>`jitsuKey` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.7. This feature is in Alpha. It can be changed or removed at any time. Jitsu key assigned to the project. | +| <a id="projectjitsukey"></a>`jitsuKey` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.7. This feature is an Experiment. It can be changed or removed at any time. Jitsu key assigned to the project. | | <a id="projectjobsenabled"></a>`jobsEnabled` | [`Boolean`](#boolean) | Indicates if CI/CD pipeline jobs are enabled for the current user. | | <a id="projectlanguages"></a>`languages` | [`[RepositoryLanguage!]`](#repositorylanguage) | Programming languages used in the project. | | <a id="projectlastactivityat"></a>`lastActivityAt` | [`Time`](#time) | Timestamp of the project last activity. | @@ -18249,7 +18600,7 @@ Represents a product analytics dashboard visualization. | <a id="projectpathlocks"></a>`pathLocks` | [`PathLockConnection`](#pathlockconnection) | The project's path locks. (see [Connections](#connections)) | | <a id="projectpipelineanalytics"></a>`pipelineAnalytics` | [`PipelineAnalytics`](#pipelineanalytics) | Pipeline analytics. | | <a id="projectprintingmergerequestlinkenabled"></a>`printingMergeRequestLinkEnabled` | [`Boolean`](#boolean) | Indicates if a link to create or view a merge request should display after a push to Git repositories of the project from the command line. | -| <a id="projectproductanalyticsstate"></a>`productAnalyticsState` **{warning-solid}** | [`ProductAnalyticsState`](#productanalyticsstate) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Current state of the product analytics stack for this project.Can only be called for one project in a single request. | +| <a id="projectproductanalyticsstate"></a>`productAnalyticsState` **{warning-solid}** | [`ProductAnalyticsState`](#productanalyticsstate) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Current state of the product analytics stack for this project.Can only be called for one project in a single request. | | <a id="projectpublicjobs"></a>`publicJobs` | [`Boolean`](#boolean) | Indicates if there is public access to pipelines and job details of the project, including output logs and artifacts. | | <a id="projectpushrules"></a>`pushRules` | [`PushRules`](#pushrules) | Project's push rules settings. | | <a id="projectrecentissueboards"></a>`recentIssueBoards` | [`BoardConnection`](#boardconnection) | List of recently visited boards of the project. Maximum size is 4. (see [Connections](#connections)) | @@ -18274,7 +18625,7 @@ Represents a product analytics dashboard visualization. | <a id="projectsuggestioncommitmessage"></a>`suggestionCommitMessage` | [`String`](#string) | Commit message used to apply merge request suggestions. | | <a id="projecttaglist"></a>`tagList` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.12. Use `topics`. | | <a id="projectterraformstates"></a>`terraformStates` | [`TerraformStateConnection`](#terraformstateconnection) | Terraform states associated with the project. (see [Connections](#connections)) | -| <a id="projecttimelogcategories"></a>`timelogCategories` **{warning-solid}** | [`TimeTrackingTimelogCategoryConnection`](#timetrackingtimelogcategoryconnection) | **Introduced** in 15.3. This feature is in Alpha. It can be changed or removed at any time. Timelog categories for the project. | +| <a id="projecttimelogcategories"></a>`timelogCategories` **{warning-solid}** | [`TimeTrackingTimelogCategoryConnection`](#timetrackingtimelogcategoryconnection) | **Introduced** in 15.3. This feature is an Experiment. It can be changed or removed at any time. Timelog categories for the project. | | <a id="projecttopics"></a>`topics` | [`[String!]`](#string) | List of project topics. | | <a id="projectuserpermissions"></a>`userPermissions` | [`ProjectPermissions!`](#projectpermissions) | Permissions for the current user on the resource. | | <a id="projectvisibility"></a>`visibility` | [`String`](#string) | Visibility of the project. | @@ -18408,13 +18759,26 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="projectboardsid"></a>`id` | [`BoardID`](#boardid) | Find a board by its ID. | +##### `Project.branchesTippingAtCommit` + +Get branch names tipping at a given commit. + +Returns [`CommitParentNames`](#commitparentnames). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectbranchestippingatcommitcommitsha"></a>`commitSha` | [`String!`](#string) | Project commit SHA identifier. For example, `287774414568010855642518513f085491644061`. | +| <a id="projectbranchestippingatcommitlimit"></a>`limit` | [`Int`](#int) | Number of branch names to return. | + ##### `Project.ciConfigVariables` CI/CD config variable. WARNING: **Introduced** in 15.3. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`[CiConfigVariable!]`](#ciconfigvariable). @@ -18422,7 +18786,7 @@ Returns [`[CiConfigVariable!]`](#ciconfigvariable). | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="projectciconfigvariablessha"></a>`sha` | [`String!`](#string) | Sha. | +| <a id="projectciconfigvariablesref"></a>`ref` | [`String!`](#string) | Ref. | ##### `Project.ciTemplate` @@ -18566,7 +18930,7 @@ Returns [`ProjectDataTransfer`](#projectdatatransfer). | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="projectdatatransferfrom"></a>`from` | [`Date`](#date) | Retain egress data for 1 year. Current month will increase dynamically as egress occurs. | +| <a id="projectdatatransferfrom"></a>`from` | [`Date`](#date) | Retain egress data for one year. Data for the current month will increase dynamically as egress occurs. | | <a id="projectdatatransferto"></a>`to` | [`Date`](#date) | End date for the data. | ##### `Project.deployment` @@ -18621,7 +18985,7 @@ Details of the fork project compared to its upstream project. WARNING: **Introduced** in 15.7. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`ForkDetails`](#forkdetails). @@ -19007,6 +19371,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="projectmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="projectmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="projectmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="projectmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | @@ -19183,7 +19548,7 @@ Product Analytics dashboards of the project. WARNING: **Introduced** in 15.6. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`ProductAnalyticsDashboardConnection`](#productanalyticsdashboardconnection). @@ -19418,6 +19783,19 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="projectsnippetsids"></a>`ids` | [`[SnippetID!]`](#snippetid) | Array of global snippet IDs. For example, `gid://gitlab/ProjectSnippet/1`. | | <a id="projectsnippetsvisibility"></a>`visibility` | [`VisibilityScopesEnum`](#visibilityscopesenum) | Visibility of the snippet. | +##### `Project.tagsTippingAtCommit` + +Get tag names tipping at a given commit. + +Returns [`CommitParentNames`](#commitparentnames). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projecttagstippingatcommitcommitsha"></a>`commitSha` | [`String!`](#string) | Project commit SHA identifier. For example, `287774414568010855642518513f085491644061`. | +| <a id="projecttagstippingatcommitlimit"></a>`limit` | [`Int`](#int) | Number of branch names to return. | + ##### `Project.terraformState` Find a single Terraform state by name. @@ -19459,7 +19837,7 @@ Visible forks of the project. WARNING: **Introduced** in 15.10. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`ProjectConnection`](#projectconnection). @@ -19560,7 +19938,7 @@ Work items of the project. WARNING: **Introduced** in 15.1. -This feature is in Alpha. It can be changed or removed at any time. +This feature is an Experiment. It can be changed or removed at any time. Returns [`WorkItemConnection`](#workitemconnection). @@ -19572,7 +19950,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="projectworkitemsauthorusername"></a>`authorUsername` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. Filter work items by author username. | +| <a id="projectworkitemsauthorusername"></a>`authorUsername` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. Filter work items by author username. | | <a id="projectworkitemsiid"></a>`iid` | [`String`](#string) | IID of the work item. For example, "1". | | <a id="projectworkitemsiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of work items. For example, `["1", "2"]`. | | <a id="projectworkitemsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | @@ -19749,10 +20127,10 @@ Returns [`ValueStreamAnalyticsMetric`](#valuestreamanalyticsmetric). | ---- | ---- | ----------- | | <a id="projectvaluestreamanalyticsflowmetricscycletimeassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | | <a id="projectvaluestreamanalyticsflowmetricscycletimeauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | -| <a id="projectvaluestreamanalyticsflowmetricscycletimefrom"></a>`from` | [`Time!`](#time) | Issues created after the date. | +| <a id="projectvaluestreamanalyticsflowmetricscycletimefrom"></a>`from` | [`Time!`](#time) | After the date. | | <a id="projectvaluestreamanalyticsflowmetricscycletimelabelnames"></a>`labelNames` | [`[String!]`](#string) | Labels applied to the issue. | | <a id="projectvaluestreamanalyticsflowmetricscycletimemilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Milestone applied to the issue. | -| <a id="projectvaluestreamanalyticsflowmetricscycletimeto"></a>`to` | [`Time!`](#time) | Issues created before the date. | +| <a id="projectvaluestreamanalyticsflowmetricscycletimeto"></a>`to` | [`Time!`](#time) | Before the date. | ##### `ProjectValueStreamAnalyticsFlowMetrics.deploymentCount` @@ -19764,8 +20142,8 @@ Returns [`ValueStreamAnalyticsMetric`](#valuestreamanalyticsmetric). | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="projectvaluestreamanalyticsflowmetricsdeploymentcountfrom"></a>`from` | [`Time!`](#time) | Deployments finished after the date. | -| <a id="projectvaluestreamanalyticsflowmetricsdeploymentcountto"></a>`to` | [`Time!`](#time) | Deployments finished before the date. | +| <a id="projectvaluestreamanalyticsflowmetricsdeploymentcountfrom"></a>`from` | [`Time!`](#time) | After the date. | +| <a id="projectvaluestreamanalyticsflowmetricsdeploymentcountto"></a>`to` | [`Time!`](#time) | Before the date. | ##### `ProjectValueStreamAnalyticsFlowMetrics.issueCount` @@ -19779,10 +20157,10 @@ Returns [`ValueStreamAnalyticsMetric`](#valuestreamanalyticsmetric). | ---- | ---- | ----------- | | <a id="projectvaluestreamanalyticsflowmetricsissuecountassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | | <a id="projectvaluestreamanalyticsflowmetricsissuecountauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | -| <a id="projectvaluestreamanalyticsflowmetricsissuecountfrom"></a>`from` | [`Time!`](#time) | Issues created after the date. | +| <a id="projectvaluestreamanalyticsflowmetricsissuecountfrom"></a>`from` | [`Time!`](#time) | After the date. | | <a id="projectvaluestreamanalyticsflowmetricsissuecountlabelnames"></a>`labelNames` | [`[String!]`](#string) | Labels applied to the issue. | | <a id="projectvaluestreamanalyticsflowmetricsissuecountmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Milestone applied to the issue. | -| <a id="projectvaluestreamanalyticsflowmetricsissuecountto"></a>`to` | [`Time!`](#time) | Issues created before the date. | +| <a id="projectvaluestreamanalyticsflowmetricsissuecountto"></a>`to` | [`Time!`](#time) | Before the date. | ##### `ProjectValueStreamAnalyticsFlowMetrics.leadTime` @@ -19796,10 +20174,29 @@ Returns [`ValueStreamAnalyticsMetric`](#valuestreamanalyticsmetric). | ---- | ---- | ----------- | | <a id="projectvaluestreamanalyticsflowmetricsleadtimeassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | | <a id="projectvaluestreamanalyticsflowmetricsleadtimeauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | -| <a id="projectvaluestreamanalyticsflowmetricsleadtimefrom"></a>`from` | [`Time!`](#time) | Issues created after the date. | +| <a id="projectvaluestreamanalyticsflowmetricsleadtimefrom"></a>`from` | [`Time!`](#time) | After the date. | | <a id="projectvaluestreamanalyticsflowmetricsleadtimelabelnames"></a>`labelNames` | [`[String!]`](#string) | Labels applied to the issue. | | <a id="projectvaluestreamanalyticsflowmetricsleadtimemilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Milestone applied to the issue. | -| <a id="projectvaluestreamanalyticsflowmetricsleadtimeto"></a>`to` | [`Time!`](#time) | Issues created before the date. | +| <a id="projectvaluestreamanalyticsflowmetricsleadtimeto"></a>`to` | [`Time!`](#time) | Before the date. | + +### `ProjectWikiRepositoryRegistry` + +Represents the Geo replication and verification state of a project_wiki_repository. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectwikirepositoryregistrycreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp when the ProjectWikiRepositoryRegistry was created. | +| <a id="projectwikirepositoryregistryid"></a>`id` | [`ID!`](#id) | ID of the ProjectWikiRepositoryRegistry. | +| <a id="projectwikirepositoryregistrylastsyncfailure"></a>`lastSyncFailure` | [`String`](#string) | Error message during sync of the ProjectWikiRepositoryRegistry. | +| <a id="projectwikirepositoryregistrylastsyncedat"></a>`lastSyncedAt` | [`Time`](#time) | Timestamp of the most recent successful sync of the ProjectWikiRepositoryRegistry. | +| <a id="projectwikirepositoryregistryprojectwikirepositoryid"></a>`projectWikiRepositoryId` | [`ID!`](#id) | ID of the Project Wiki Repository. | +| <a id="projectwikirepositoryregistryretryat"></a>`retryAt` | [`Time`](#time) | Timestamp after which the ProjectWikiRepositoryRegistry is resynced. | +| <a id="projectwikirepositoryregistryretrycount"></a>`retryCount` | [`Int`](#int) | Number of consecutive failed sync attempts of the ProjectWikiRepositoryRegistry. | +| <a id="projectwikirepositoryregistrystate"></a>`state` | [`RegistryState`](#registrystate) | Sync state of the ProjectWikiRepositoryRegistry. | +| <a id="projectwikirepositoryregistryverificationretryat"></a>`verificationRetryAt` | [`Time`](#time) | Timestamp after which the ProjectWikiRepositoryRegistry is reverified. | +| <a id="projectwikirepositoryregistryverifiedat"></a>`verifiedAt` | [`Time`](#time) | Timestamp of the most recent successful verification of the ProjectWikiRepositoryRegistry. | ### `PrometheusAlert` @@ -20058,6 +20455,18 @@ Returns [`[String!]`](#string). | <a id="repositorybranchnamesoffset"></a>`offset` | [`Int!`](#int) | Number of branch names to skip. | | <a id="repositorybranchnamessearchpattern"></a>`searchPattern` | [`String!`](#string) | Pattern to search for branch names by. | +##### `Repository.codeOwnersPath` + +Path to CODEOWNERS file in a ref. + +Returns [`String`](#string). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="repositorycodeownerspathref"></a>`ref` | [`String`](#string) | Name of the ref. | + ##### `Repository.paginatedTree` Paginated tree of the repository. @@ -20376,6 +20785,7 @@ Represents the scan result policy. | <a id="scanresultpolicyenabled"></a>`enabled` | [`Boolean!`](#boolean) | Indicates whether this policy is enabled. | | <a id="scanresultpolicygroupapprovers"></a>`groupApprovers` | [`[Group!]`](#group) | Approvers of the group type. | | <a id="scanresultpolicyname"></a>`name` | [`String!`](#string) | Name of the policy. | +| <a id="scanresultpolicyroleapprovers"></a>`roleApprovers` | [`[MemberAccessLevelName!]`](#memberaccesslevelname) | Approvers of the role type. Users belonging to these role(s) alone will be approvers. | | <a id="scanresultpolicysource"></a>`source` | [`SecurityPolicySource!`](#securitypolicysource) | Source of the policy. Its fields depend on the source type. | | <a id="scanresultpolicyupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp of when the policy YAML was last updated. | | <a id="scanresultpolicyuserapprovers"></a>`userApprovers` | [`[UserCore!]`](#usercore) | Approvers of the user type. | @@ -21079,6 +21489,7 @@ Describes an incident management timeline event. | <a id="timelogissue"></a>`issue` | [`Issue`](#issue) | Issue that logged time was added to. | | <a id="timelogmergerequest"></a>`mergeRequest` | [`MergeRequest`](#mergerequest) | Merge request that logged time was added to. | | <a id="timelognote"></a>`note` | [`Note`](#note) | Note where the quick action was executed to add the logged time. | +| <a id="timelogproject"></a>`project` | [`Project!`](#project) | Target project of the timelog merge request or issue. | | <a id="timelogspentat"></a>`spentAt` | [`Time`](#time) | Timestamp of when the time tracked was spent at. | | <a id="timelogsummary"></a>`summary` | [`String`](#string) | Summary of how the time was spent. | | <a id="timelogtimespent"></a>`timeSpent` | [`Int!`](#int) | Time spent displayed in seconds. | @@ -21251,7 +21662,7 @@ Core represention of a GitLab user. | <a id="usercoresavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="usercorestate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="usercorestatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | -| <a id="usercoreuserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | +| <a id="usercoreuserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | | <a id="usercoreuserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | | <a id="usercoreusername"></a>`username` | [`String!`](#string) | Username of the user. Unique within this instance of GitLab. | | <a id="usercorewebpath"></a>`webPath` | [`String!`](#string) | Web path of the user. | @@ -21273,6 +21684,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="usercoreassignedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="usercoreassignedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="usercoreassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="usercoreassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -21307,6 +21719,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="usercoreauthoredmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="usercoreauthoredmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="usercoreauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="usercoreauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -21358,6 +21771,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="usercorereviewrequestedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="usercorereviewrequestedmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="usercorereviewrequestedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="usercorereviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | @@ -22174,7 +22588,7 @@ Represents vulnerability letter grades with associated projects. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="workitemauthor"></a>`author` **{warning-solid}** | [`UserCore`](#usercore) | **Introduced** in 15.9. This feature is in Alpha. It can be changed or removed at any time. User that created the work item. | +| <a id="workitemauthor"></a>`author` **{warning-solid}** | [`UserCore`](#usercore) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. User that created the work item. | | <a id="workitemclosedat"></a>`closedAt` | [`Time`](#time) | Timestamp of when the work item was closed. | | <a id="workitemconfidential"></a>`confidential` | [`Boolean!`](#boolean) | Indicates the work item is confidential. | | <a id="workitemcreatedat"></a>`createdAt` | [`Time!`](#time) | Timestamp of when the work item was created. | @@ -22183,7 +22597,8 @@ Represents vulnerability letter grades with associated projects. | <a id="workitemid"></a>`id` | [`WorkItemID!`](#workitemid) | Global ID of the work item. | | <a id="workitemiid"></a>`iid` | [`ID!`](#id) | Internal ID of the work item. | | <a id="workitemlockversion"></a>`lockVersion` | [`Int!`](#int) | Lock version of the work item. Incremented each time the work item is updated. | -| <a id="workitemproject"></a>`project` **{warning-solid}** | [`Project!`](#project) | **Introduced** in 15.3. This feature is in Alpha. It can be changed or removed at any time. Project the work item belongs to. | +| <a id="workitemnamespace"></a>`namespace` **{warning-solid}** | [`Namespace`](#namespace) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Namespace the work item belongs to. | +| <a id="workitemproject"></a>`project` **{warning-solid}** | [`Project`](#project) | **Introduced** in 15.3. This feature is an Experiment. It can be changed or removed at any time. Project the work item belongs to. | | <a id="workitemstate"></a>`state` | [`WorkItemState!`](#workitemstate) | State of the work item. | | <a id="workitemtitle"></a>`title` | [`String!`](#string) | Title of the work item. | | <a id="workitemtitlehtml"></a>`titleHtml` | [`String`](#string) | GitLab Flavored Markdown rendering of `title`. | @@ -22201,6 +22616,7 @@ Check permissions for the current user on a work item. | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="workitempermissionsadminparentlink"></a>`adminParentLink` | [`Boolean!`](#boolean) | Indicates the user can perform `admin_parent_link` on this resource. | | <a id="workitempermissionsadminworkitem"></a>`adminWorkItem` | [`Boolean!`](#boolean) | Indicates the user can perform `admin_work_item` on this resource. | | <a id="workitempermissionsdeleteworkitem"></a>`deleteWorkItem` | [`Boolean!`](#boolean) | Indicates the user can perform `delete_work_item` on this resource. | | <a id="workitempermissionsreadworkitem"></a>`readWorkItem` | [`Boolean!`](#boolean) | Indicates the user can perform `read_work_item` on this resource. | @@ -22229,6 +22645,47 @@ Represents an assignees widget. | <a id="workitemwidgetassigneescaninvitemembers"></a>`canInviteMembers` | [`Boolean!`](#boolean) | Indicates whether the current user can invite members to the work item's project. | | <a id="workitemwidgetassigneestype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. | +### `WorkItemWidgetAwardEmoji` + +Represents the award emoji widget. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="workitemwidgetawardemojiawardemoji"></a>`awardEmoji` | [`AwardEmojiConnection`](#awardemojiconnection) | Award emoji on the work item. (see [Connections](#connections)) | +| <a id="workitemwidgetawardemojidownvotes"></a>`downvotes` | [`Int!`](#int) | Number of downvotes the work item has received. | +| <a id="workitemwidgetawardemojitype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. | +| <a id="workitemwidgetawardemojiupvotes"></a>`upvotes` | [`Int!`](#int) | Number of upvotes the work item has received. | + +### `WorkItemWidgetCurrentUserTodos` + +Represents a todos widget. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="workitemwidgetcurrentusertodostype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. | + +#### Fields with arguments + +##### `WorkItemWidgetCurrentUserTodos.currentUserTodos` + +To-do items for the current user. + +Returns [`TodoConnection!`](#todoconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="workitemwidgetcurrentusertodoscurrentusertodosstate"></a>`state` | [`TodoStateEnum`](#todostateenum) | State of the to-do items. | + ### `WorkItemWidgetDescription` Represents a description widget. @@ -22634,6 +23091,7 @@ Available fields to be exported as CSV. | <a id="availableexportfieldsauthor"></a>`AUTHOR` | Author name. | | <a id="availableexportfieldsauthor_username"></a>`AUTHOR_USERNAME` | Author username. | | <a id="availableexportfieldscreated_at"></a>`CREATED_AT` | Date of creation. | +| <a id="availableexportfieldsdescription"></a>`DESCRIPTION` | Description. | | <a id="availableexportfieldsid"></a>`ID` | Unique identifier. | | <a id="availableexportfieldstitle"></a>`TITLE` | Title. | | <a id="availableexportfieldstype"></a>`TYPE` | Type of the work item. | @@ -22654,6 +23112,7 @@ Include type. | Value | Description | | ----- | ----------- | +| <a id="ciconfigincludetypecomponent"></a>`component` | Component include. | | <a id="ciconfigincludetypefile"></a>`file` | Project file include. | | <a id="ciconfigincludetypelocal"></a>`local` | Local include. | | <a id="ciconfigincludetyperemote"></a>`remote` | Remote include. | @@ -22720,8 +23179,8 @@ Direction of access. | Value | Description | | ----- | ----------- | -| <a id="cirunnerjobexecutionstatusidle"></a>`IDLE` **{warning-solid}** | **Introduced** in 15.7. This feature is in Alpha. It can be changed or removed at any time. Runner is idle. | -| <a id="cirunnerjobexecutionstatusrunning"></a>`RUNNING` **{warning-solid}** | **Introduced** in 15.7. This feature is in Alpha. It can be changed or removed at any time. Runner is executing jobs. | +| <a id="cirunnerjobexecutionstatusidle"></a>`IDLE` **{warning-solid}** | **Introduced** in 15.7. This feature is an Experiment. It can be changed or removed at any time. Runner is idle. | +| <a id="cirunnerjobexecutionstatusrunning"></a>`RUNNING` **{warning-solid}** | **Introduced** in 15.7. This feature is an Experiment. It can be changed or removed at any time. Runner is executing jobs. | ### `CiRunnerMembershipFilter` @@ -22729,7 +23188,7 @@ Values for filtering runners in namespaces. The previous type name `RunnerMember | Value | Description | | ----- | ----------- | -| <a id="cirunnermembershipfilterall_available"></a>`ALL_AVAILABLE` **{warning-solid}** | **Introduced** in 15.5. This feature is in Alpha. It can be changed or removed at any time. Include all runners. This list includes runners for all projects in the group and subgroups, as well as for the parent groups and instance. | +| <a id="cirunnermembershipfilterall_available"></a>`ALL_AVAILABLE` **{warning-solid}** | **Introduced** in 15.5. This feature is an Experiment. It can be changed or removed at any time. Include all runners. This list includes runners for all projects in the group and subgroups, as well as for the parent groups and instance. | | <a id="cirunnermembershipfilterdescendants"></a>`DESCENDANTS` | Include runners that have either a direct or inherited relationship. These runners can be specific to a project or a group. | | <a id="cirunnermembershipfilterdirect"></a>`DIRECT` | Include runners that have a direct relationship. | @@ -23393,6 +23852,7 @@ Issuable resource link type enum. | Value | Description | | ----- | ----------- | | <a id="issuableresourcelinktypegeneral"></a>`general` | General link type. | +| <a id="issuableresourcelinktypepagerduty"></a>`pagerduty` | Pagerduty link type. | | <a id="issuableresourcelinktypeslack"></a>`slack` | Slack link type. | | <a id="issuableresourcelinktypezoom"></a>`zoom` | Zoom link type. | @@ -23528,10 +23988,10 @@ Issue type. | ----- | ----------- | | <a id="issuetypeincident"></a>`INCIDENT` | Incident issue type. | | <a id="issuetypeissue"></a>`ISSUE` | Issue issue type. | -| <a id="issuetypekey_result"></a>`KEY_RESULT` **{warning-solid}** | **Introduced** in 15.7. This feature is in Alpha. It can be changed or removed at any time. Key Result issue type. Available only when feature flag `okrs_mvc` is enabled. | -| <a id="issuetypeobjective"></a>`OBJECTIVE` **{warning-solid}** | **Introduced** in 15.6. This feature is in Alpha. It can be changed or removed at any time. Objective issue type. Available only when feature flag `okrs_mvc` is enabled. | +| <a id="issuetypekey_result"></a>`KEY_RESULT` **{warning-solid}** | **Introduced** in 15.7. This feature is an Experiment. It can be changed or removed at any time. Key Result issue type. Available only when feature flag `okrs_mvc` is enabled. | +| <a id="issuetypeobjective"></a>`OBJECTIVE` **{warning-solid}** | **Introduced** in 15.6. This feature is an Experiment. It can be changed or removed at any time. Objective issue type. Available only when feature flag `okrs_mvc` is enabled. | | <a id="issuetyperequirement"></a>`REQUIREMENT` | Requirement issue type. | -| <a id="issuetypetask"></a>`TASK` **{warning-solid}** | **Introduced** in 15.2. This feature is in Alpha. It can be changed or removed at any time. Task issue type. | +| <a id="issuetypetask"></a>`TASK` **{warning-solid}** | **Introduced** in 15.2. This feature is an Experiment. It can be changed or removed at any time. Task issue type. | | <a id="issuetypetest_case"></a>`TEST_CASE` | Test Case issue type. | ### `IterationSearchableField` @@ -23649,6 +24109,18 @@ Access level of a group or project member. | <a id="memberaccesslevelowner"></a>`OWNER` | Owner access. | | <a id="memberaccesslevelreporter"></a>`REPORTER` | Reporter access. | +### `MemberAccessLevelName` + +Name of access levels of a group or project member. + +| Value | Description | +| ----- | ----------- | +| <a id="memberaccesslevelnamedeveloper"></a>`DEVELOPER` | Developer access. | +| <a id="memberaccesslevelnameguest"></a>`GUEST` | Guest access. | +| <a id="memberaccesslevelnamemaintainer"></a>`MAINTAINER` | Maintainer access. | +| <a id="memberaccesslevelnameowner"></a>`OWNER` | Owner access. | +| <a id="memberaccesslevelnamereporter"></a>`REPORTER` | Reporter access. | + ### `MemberSort` Values for sorting members. @@ -24053,6 +24525,15 @@ State of a Geo registry. | <a id="registrystatestarted"></a>`STARTED` | Registry currently syncing. | | <a id="registrystatesynced"></a>`SYNCED` | Registry that is synced. | +### `RelativePositionType` + +The position to which the object should be moved. + +| Value | Description | +| ----- | ----------- | +| <a id="relativepositiontypeafter"></a>`AFTER` | Object is moved after an adjacent object. | +| <a id="relativepositiontypebefore"></a>`BEFORE` | Object is moved before an adjacent object. | + ### `ReleaseAssetLinkType` Type of the link: `other`, `runbook`, `image`, `package`. @@ -24149,6 +24630,7 @@ The status of the security scan. | Value | Description | | ----- | ----------- | | <a id="securityreporttypeenumapi_fuzzing"></a>`API_FUZZING` | API FUZZING scan report. | +| <a id="securityreporttypeenumbreach_and_attack_simulation"></a>`BREACH_AND_ATTACK_SIMULATION` | BREACH AND ATTACK SIMULATION scan report. | | <a id="securityreporttypeenumcluster_image_scanning"></a>`CLUSTER_IMAGE_SCANNING` | CLUSTER IMAGE SCANNING scan report. | | <a id="securityreporttypeenumcontainer_scanning"></a>`CONTAINER_SCANNING` | CONTAINER SCANNING scan report. | | <a id="securityreporttypeenumcoverage_fuzzing"></a>`COVERAGE_FUZZING` | COVERAGE FUZZING scan report. | @@ -24165,6 +24647,7 @@ The type of the security scanner. | Value | Description | | ----- | ----------- | | <a id="securityscannertypeapi_fuzzing"></a>`API_FUZZING` | API Fuzzing scanner. | +| <a id="securityscannertypebreach_and_attack_simulation"></a>`BREACH_AND_ATTACK_SIMULATION` | Breach And Attack Simulation scanner. | | <a id="securityscannertypecluster_image_scanning"></a>`CLUSTER_IMAGE_SCANNING` | Cluster Image Scanning scanner. | | <a id="securityscannertypecontainer_scanning"></a>`CONTAINER_SCANNING` | Container Scanning scanner. | | <a id="securityscannertypecoverage_fuzzing"></a>`COVERAGE_FUZZING` | Coverage Fuzzing scanner. | @@ -24633,6 +25116,15 @@ Weight ID wildcard values. | <a id="weightwildcardidany"></a>`ANY` | Weight is assigned. | | <a id="weightwildcardidnone"></a>`NONE` | No weight is assigned. | +### `WorkItemAwardEmojiUpdateAction` + +Values for work item award emoji update enum. + +| Value | Description | +| ----- | ----------- | +| <a id="workitemawardemojiupdateactionadd"></a>`ADD` | Adds the emoji. | +| <a id="workitemawardemojiupdateactionremove"></a>`REMOVE` | Removes the emoji. | + ### `WorkItemSort` Values for sorting work items. @@ -24668,6 +25160,15 @@ Values for work item state events. | <a id="workitemstateeventclose"></a>`CLOSE` | Closes the work item. | | <a id="workitemstateeventreopen"></a>`REOPEN` | Reopens the work item. | +### `WorkItemTodoUpdateAction` + +Values for work item to-do update enum. + +| Value | Description | +| ----- | ----------- | +| <a id="workitemtodoupdateactionadd"></a>`ADD` | Adds the to-do. | +| <a id="workitemtodoupdateactionmark_as_done"></a>`MARK_AS_DONE` | Marks the to-do as done. | + ### `WorkItemWidgetType` Type of a work item widget. @@ -24675,6 +25176,8 @@ Type of a work item widget. | Value | Description | | ----- | ----------- | | <a id="workitemwidgettypeassignees"></a>`ASSIGNEES` | Assignees widget. | +| <a id="workitemwidgettypeaward_emoji"></a>`AWARD_EMOJI` | Award Emoji widget. | +| <a id="workitemwidgettypecurrent_user_todos"></a>`CURRENT_USER_TODOS` | Current User Todos widget. | | <a id="workitemwidgettypedescription"></a>`DESCRIPTION` | Description widget. | | <a id="workitemwidgettypehealth_status"></a>`HEALTH_STATUS` | Health Status widget. | | <a id="workitemwidgettypehierarchy"></a>`HIERARCHY` | Hierarchy widget. | @@ -24713,6 +25216,12 @@ A `AchievementsUserAchievementID` is a global ID. It is encoded as a string. An example `AchievementsUserAchievementID` is: `"gid://gitlab/Achievements::UserAchievement/1"`. +### `AiModelID` + +A `AiModelID` is a global ID. It is encoded as a string. + +An example `AiModelID` is: `"gid://gitlab/Ai::Model/1"`. + ### `AlertManagementAlertID` A `AlertManagementAlertID` is a global ID. It is encoded as a string. @@ -24811,11 +25320,11 @@ A `CiRunnerID` is a global ID. It is encoded as a string. An example `CiRunnerID` is: `"gid://gitlab/Ci::Runner/1"`. -### `CiRunnerMachineID` +### `CiRunnerManagerID` -A `CiRunnerMachineID` is a global ID. It is encoded as a string. +A `CiRunnerManagerID` is a global ID. It is encoded as a string. -An example `CiRunnerMachineID` is: `"gid://gitlab/Ci::RunnerMachine/1"`. +An example `CiRunnerManagerID` is: `"gid://gitlab/Ci::RunnerManager/1"`. ### `ClustersAgentID` @@ -25551,6 +26060,7 @@ Implementations: - [`EpicIssue`](#epicissue) - [`Issue`](#issue) - [`MergeRequest`](#mergerequest) +- [`WorkItemWidgetCurrentUserTodos`](#workitemwidgetcurrentusertodos) ##### Fields with arguments @@ -25821,7 +26331,7 @@ Implementations: | <a id="usersavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="userstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="userstatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | -| <a id="useruserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is in Alpha. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | +| <a id="useruserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Achievements for the user. Only returns for namespaces where the `achievements` feature flag is enabled. | | <a id="useruserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | | <a id="userusername"></a>`username` | [`String!`](#string) | Username of the user. Unique within this instance of GitLab. | | <a id="userwebpath"></a>`webPath` | [`String!`](#string) | Web path of the user. | @@ -25843,6 +26353,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="userassignedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="userassignedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="userassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="userassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -25877,6 +26388,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="userauthoredmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="userauthoredmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="userauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="userauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | @@ -25928,6 +26440,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="userreviewrequestedmergerequestsapproved"></a>`approved` | [`Boolean`](#boolean) | Limit results to approved merge requests. | | <a id="userreviewrequestedmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | | <a id="userreviewrequestedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | | <a id="userreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | @@ -26043,6 +26556,8 @@ four standard [pagination arguments](#connection-pagination-arguments): Implementations: - [`WorkItemWidgetAssignees`](#workitemwidgetassignees) +- [`WorkItemWidgetAwardEmoji`](#workitemwidgetawardemoji) +- [`WorkItemWidgetCurrentUserTodos`](#workitemwidgetcurrentusertodos) - [`WorkItemWidgetDescription`](#workitemwidgetdescription) - [`WorkItemWidgetHealthStatus`](#workitemwidgethealthstatus) - [`WorkItemWidgetHierarchy`](#workitemwidgethierarchy) @@ -26072,6 +26587,40 @@ be used as arguments). Only general use input types are listed here. For mutation input types, see the associated mutation type above. +### `AiExplainCodeInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="aiexplaincodeinputmessages"></a>`messages` | [`[AiExplainCodeMessageInput!]!`](#aiexplaincodemessageinput) | Code messages that is passed to be explained by AI. | +| <a id="aiexplaincodeinputresourceid"></a>`resourceId` | [`AiModelID!`](#aimodelid) | GID of the resource to mutate. | + +### `AiExplainCodeMessageInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="aiexplaincodemessageinputcontent"></a>`content` | [`String!`](#string) | Content of the message. | +| <a id="aiexplaincodemessageinputrole"></a>`role` | [`String!`](#string) | Role of the message (system, user, assistant). | + +### `AiExplainVulnerabilityInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="aiexplainvulnerabilityinputresourceid"></a>`resourceId` | [`AiModelID!`](#aimodelid) | GID of the resource to mutate. | + +### `AiSummarizeCommentsInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="aisummarizecommentsinputresourceid"></a>`resourceId` | [`AiModelID!`](#aimodelid) | GID of the resource to mutate. | + ### `AlertManagementPayloadAlertFieldInput` Field that are available while modifying the custom mapping attributes for an HTTP integration. @@ -26171,6 +26720,7 @@ Attributes for defining a CI/CD variable. | <a id="complianceviolationinputmergedafter"></a>`mergedAfter` | [`Date`](#date) | Merge requests merged after this date (inclusive). | | <a id="complianceviolationinputmergedbefore"></a>`mergedBefore` | [`Date`](#date) | Merge requests merged before this date (inclusive). | | <a id="complianceviolationinputprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter compliance violations by project. | +| <a id="complianceviolationinputtargetbranch"></a>`targetBranch` | [`String`](#string) | Filter compliance violations by target branch. | ### `DastProfileCadenceInput` @@ -26644,7 +27194,9 @@ A time-frame defined as a closed inclusive range of two dates. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="workitemupdatedtaskinputassigneeswidget"></a>`assigneesWidget` | [`WorkItemWidgetAssigneesInput`](#workitemwidgetassigneesinput) | Input for assignees widget. | +| <a id="workitemupdatedtaskinputawardemojiwidget"></a>`awardEmojiWidget` | [`WorkItemWidgetAwardEmojiUpdateInput`](#workitemwidgetawardemojiupdateinput) | Input for award emoji widget. | | <a id="workitemupdatedtaskinputconfidential"></a>`confidential` | [`Boolean`](#boolean) | Sets the work item confidentiality. | +| <a id="workitemupdatedtaskinputcurrentusertodoswidget"></a>`currentUserTodosWidget` | [`WorkItemWidgetCurrentUserTodosInput`](#workitemwidgetcurrentusertodosinput) | Input for to-dos widget. | | <a id="workitemupdatedtaskinputdescriptionwidget"></a>`descriptionWidget` | [`WorkItemWidgetDescriptionInput`](#workitemwidgetdescriptioninput) | Input for description widget. | | <a id="workitemupdatedtaskinputhierarchywidget"></a>`hierarchyWidget` | [`WorkItemWidgetHierarchyUpdateInput`](#workitemwidgethierarchyupdateinput) | Input for hierarchy widget. | | <a id="workitemupdatedtaskinputid"></a>`id` | [`WorkItemID!`](#workitemid) | Global ID of the work item. | @@ -26663,6 +27215,24 @@ A time-frame defined as a closed inclusive range of two dates. | ---- | ---- | ----------- | | <a id="workitemwidgetassigneesinputassigneeids"></a>`assigneeIds` | [`[UserID!]!`](#userid) | Global IDs of assignees. | +### `WorkItemWidgetAwardEmojiUpdateInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="workitemwidgetawardemojiupdateinputaction"></a>`action` | [`WorkItemAwardEmojiUpdateAction!`](#workitemawardemojiupdateaction) | Action for the update. | +| <a id="workitemwidgetawardemojiupdateinputname"></a>`name` | [`String!`](#string) | Emoji name. | + +### `WorkItemWidgetCurrentUserTodosInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="workitemwidgetcurrentusertodosinputaction"></a>`action` | [`WorkItemTodoUpdateAction!`](#workitemtodoupdateaction) | Action for the update. | +| <a id="workitemwidgetcurrentusertodosinputtodoid"></a>`todoId` | [`TodoID`](#todoid) | Global ID of the to-do. If not present, all to-dos of the work item will be updated. | + ### `WorkItemWidgetDescriptionInput` #### Arguments @@ -26693,8 +27263,10 @@ A time-frame defined as a closed inclusive range of two dates. | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="workitemwidgethierarchyupdateinputadjacentworkitemid"></a>`adjacentWorkItemId` | [`WorkItemID`](#workitemid) | ID of the work item to be switched with. | | <a id="workitemwidgethierarchyupdateinputchildrenids"></a>`childrenIds` | [`[WorkItemID!]`](#workitemid) | Global IDs of children work items. | | <a id="workitemwidgethierarchyupdateinputparentid"></a>`parentId` | [`WorkItemID`](#workitemid) | Global ID of the parent work item. Use `null` to remove the association. | +| <a id="workitemwidgethierarchyupdateinputrelativeposition"></a>`relativePosition` | [`RelativePositionType`](#relativepositiontype) | Type of switch. Valid values are `BEFORE` or `AFTER`. | ### `WorkItemWidgetIterationInput` diff --git a/doc/api/group_clusters.md b/doc/api/group_clusters.md index a180c7a1f92..a4369ecad5f 100644 --- a/doc/api/group_clusters.md +++ b/doc/api/group_clusters.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/group_import_export.md b/doc/api/group_import_export.md index 5dd0e4e3d52..a9ec3e8547c 100644 --- a/doc/api/group_import_export.md +++ b/doc/api/group_import_export.md @@ -6,8 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Group import and export API **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20353) in GitLab 12.8. - Use the group import and export API to export a group structure and import it to a new location. When you use the group import and export API with the [project import and export API](project_import_export.md), you can preserve connections with group-level relationships, such as connections between project issues and group epics. @@ -29,6 +27,11 @@ If you import groups into a parent group, the subgroups inherit by default a sim To preserve the member list and their respective permissions on imported groups, review the users in these groups. Make sure these users exist before importing the desired groups. +## Prerequisites + +For information on prerequisites for group import and export API, see prerequisites for +[migrating groups by uploading an export file](../user/group/import/index.md#preparation). + ## Schedule new export Start a new group export. diff --git a/doc/api/group_protected_branches.md b/doc/api/group_protected_branches.md index db648f6870e..db42ca98f0e 100644 --- a/doc/api/group_protected_branches.md +++ b/doc/api/group_protected_branches.md @@ -167,7 +167,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ | `allowed_to_merge` | array | no | Array of access levels allowed to merge, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. | | `allowed_to_push` | array | no | Array of access levels allowed to push, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. | | `allowed_to_unprotect` | array | no | Array of access levels allowed to unprotect, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. | -| `code_owner_approval_required` | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/code_owners.md). Default: `false`. | +| `code_owner_approval_required` | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/codeowners/index.md). Default: `false`. | | `merge_access_level` | integer | no | Access levels allowed to merge. Defaults: `40`, Maintainer role. | | `push_access_level` | integer | no | Access levels allowed to push. Defaults: `40`, Maintainer role. | | `unprotect_access_level` | integer | no | Access levels allowed to unprotect. Defaults: `40`, Maintainer role. | @@ -382,7 +382,7 @@ curl --request PATCH --header "PRIVATE-TOKEN: <your_access_token>" \ | `allowed_to_push` | array | no | Array of push access levels, with each described by a hash. | | `allowed_to_merge` | array | no | Array of merge access levels, with each described by a hash. | | `allowed_to_unprotect` | array | no | Array of unprotect access levels, with each described by a hash. | -| `code_owner_approval_required` | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/code_owners.md). Default: `false`. | +| `code_owner_approval_required` | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/codeowners/index.md). Default: `false`. | Elements in the `allowed_to_push`, `allowed_to_merge` and `allowed_to_unprotect` arrays should: diff --git a/doc/api/group_protected_environments.md b/doc/api/group_protected_environments.md index bddfd774d43..67b2b818ea9 100644 --- a/doc/api/group_protected_environments.md +++ b/doc/api/group_protected_environments.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/api/group_relations_export.md b/doc/api/group_relations_export.md index 065ae03259a..7a8000eafd1 100644 --- a/doc/api/group_relations_export.md +++ b/doc/api/group_relations_export.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w With the Group Relations Export API, you can partially export group structure. This API is similar to [group export](group_import_export.md), but it exports each top-level relation (for example, milestones/boards/labels) as a separate file -instead of one archive. The group relations export API is primarily used in [group migration](../user/group/index.md). +instead of one archive. The group relations export API is primarily used in [group migration](../user/group/import/index.md). ## Schedule new export diff --git a/doc/api/group_releases.md b/doc/api/group_releases.md index 978218041f2..a735f36eb82 100644 --- a/doc/api/group_releases.md +++ b/doc/api/group_releases.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/group_repository_storage_moves.md b/doc/api/group_repository_storage_moves.md index 8d685c75f60..95d261e79a9 100644 --- a/doc/api/group_repository_storage_moves.md +++ b/doc/api/group_repository_storage_moves.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/api/group_wikis.md b/doc/api/group_wikis.md index 6fb74ea00b7..c03224373de 100644 --- a/doc/api/group_wikis.md +++ b/doc/api/group_wikis.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" type: reference, api --- diff --git a/doc/api/groups.md b/doc/api/groups.md index 8c0e94ece28..879b10f4b7a 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -27,7 +27,7 @@ Parameters: | `search` | string | no | Return the list of authorized groups matching the search criteria | | `order_by` | string | no | Order groups by `name`, `path`, `id`, or `similarity` (if searching, [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332889) in GitLab 14.1). Default is `name` | | `sort` | string | no | Order groups in `asc` or `desc` order. Default is `asc` | -| `statistics` | boolean | no | Include group statistics (administrators only) | +| `statistics` | boolean | no | Include group statistics (administrators only).<br>*Note:* The REST API response does not provide the full `RootStorageStatistics` data that is shown in the UI. To match the data in the UI, use GraphQL instead of REST. For more information, see the [Group GraphQL reference](../api/graphql/reference/index.md#group).| | `with_custom_attributes` | boolean | no | Include [custom attributes](custom_attributes.md) in response (administrators only) | | `owned` | boolean | no | Limit to groups explicitly owned by the current user | | `min_access_level` | integer | no | Limit to groups where current user has at least this [role (`access_level`)](members.md#roles) | @@ -521,11 +521,6 @@ To get the details of all projects within a group, use either the [list a group' curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/4" ``` -NOTE: -There is [a known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345200) that can -prevent `runners_token` from being returned when the call has the `with_projects=false` -parameter. - This endpoint returns: - All projects and shared projects in GitLab 12.5 and earlier. diff --git a/doc/api/import.md b/doc/api/import.md index 87bbb56869d..4aeb5277b4a 100644 --- a/doc/api/import.md +++ b/doc/api/import.md @@ -10,16 +10,16 @@ Use the Import API to import repositories from GitHub or Bitbucket Server. Related APIs include: -- [Group migration by direct transfer API](bulk_imports.md) -- [Group import and export API](group_import_export.md) -- [Project import and export API](project_import_export.md) +- [Group migration by direct transfer API](bulk_imports.md). +- [Group import and export API](group_import_export.md). +- [Project import and export API](project_import_export.md). ## Prerequisites For information on prerequisites for using the Import API, see: -- [Prerequisites for GitHub importer](../user/project/import/github.md#prerequisites) -- [Prerequisites for Bitbucket Server importer](../user/project/import/bitbucket_server.md#import-your-bitbucket-repositories) +- [Prerequisites for GitHub importer](../user/project/import/github.md#prerequisites). +- [Prerequisites for Bitbucket Server importer](../user/project/import/bitbucket_server.md#import-your-bitbucket-repositories). ## Import repository from GitHub @@ -141,11 +141,7 @@ Returns the following status codes: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371099) in GitLab 15.8 [with a flag](../administration/feature_flags.md) named `github_import_gists`. Disabled by default. Enabled on GitLab.com. > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/386579) in GitLab 15.10. - -FLAG: -On self-managed GitLab, this feature is available by default. To hide the feature, -ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `github_import_gists`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/386579) in GitLab 15.11. Feature flag `github_import_gists` removed. You can use the GitLab API to import personal GitHub gists (with up to 10 files) into personal GitLab snippets. GitHub gists with more than 10 files are skipped. You should manually migrate these GitHub gists. diff --git a/doc/api/index.md b/doc/api/index.md index 110534ceced..92301a1c3e0 100644 --- a/doc/api/index.md +++ b/doc/api/index.md @@ -10,8 +10,9 @@ Automate and interact with GitLab, and integrate with external applications. - [Integrations](../integration/index.md) - [Webhooks](../user/project/integrations/webhooks.md) -- [Visual Studio Code extension](../user/project/repository/vscode.md) - [REST API](rest/index.md) - [GraphQL API](graphql/index.md) -- [Lint `.gitlab-ci.yml`](lint.md) -- [GitLab as an OAuth2 provider](oauth2.md) +- [OAuth 2.0 identity provider API](oauth2.md) +- [GitLab CLI (glab)](../integration/glab/index.md) +- [Visual Studio Code extension](../user/project/repository/vscode.md) +- [Code Suggestions](../user/project/repository/code_suggestions.md) diff --git a/doc/api/instance_clusters.md b/doc/api/instance_clusters.md index 51296c2e96e..751e3203990 100644 --- a/doc/api/instance_clusters.md +++ b/doc/api/instance_clusters.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/integrations.md b/doc/api/integrations.md index e25753b892e..c69daa70b53 100644 --- a/doc/api/integrations.md +++ b/doc/api/integrations.md @@ -944,6 +944,8 @@ Parameters: | `username` | string | yes | The username of the user created to be used with GitLab/Jira. | | `password` | string | yes | The password of the user created to be used with GitLab/Jira. | | `active` | boolean | no | Activates or deactivates the integration. Defaults to false (deactivated). | +| `jira_issue_prefix` | string | no | Prefix to match Jira issue keys. | +| `jira_issue_regex` | string | no | Regular expression to match Jira issue keys. | | `jira_issue_transition_automatic` | boolean | no | Enable [automatic issue transitions](../integration/jira/issues.md#automatic-issue-transitions). Takes precedence over `jira_issue_transition_id` if enabled. Defaults to `false` | | `jira_issue_transition_id` | string | no | The ID of one or more transitions for [custom issue transitions](../integration/jira/issues.md#custom-issue-transitions). Ignored if `jira_issue_transition_automatic` is enabled. Defaults to a blank string, which disables custom transitions. | | `commit_events` | boolean | false | Enable notifications for commit events | diff --git a/doc/api/job_artifacts.md b/doc/api/job_artifacts.md index 715b0614cc4..f121246b169 100644 --- a/doc/api/job_artifacts.md +++ b/doc/api/job_artifacts.md @@ -295,7 +295,7 @@ If the artifacts were deleted successfully, a response with status `204 No Conte Delete artifacts of a project that can be deleted. -By default, [artifacts from the most recent successful pipeline of each ref are kept](../ci/pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). +By default, [artifacts from the most recent successful pipeline of each ref are kept](../ci/jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). ```plaintext DELETE /projects/:id/artifacts diff --git a/doc/api/jobs.md b/doc/api/jobs.md index 2d65ea82edd..19d28420069 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -331,9 +331,9 @@ In earlier GitLab versions, jobs are sorted by ID in ascending order (oldest fir In GitLab 13.9 and later, this endpoint can include retried jobs in the response with `include_retried` set to `true`. -## List pipeline bridges +## List pipeline trigger jobs -Get a list of bridge jobs for a pipeline. +Get a list of trigger jobs for a pipeline. ```plaintext GET /projects/:id/pipelines/:pipeline_id/bridges diff --git a/doc/api/keys.md b/doc/api/keys.md index 90144310238..337758e6c33 100644 --- a/doc/api/keys.md +++ b/doc/api/keys.md @@ -11,15 +11,15 @@ If using a SHA256 fingerprint in an API call, you should URL-encode the fingerpr ## Get SSH key with user by ID of an SSH key -Get SSH key with user by ID of an SSH key. Note only administrators can lookup SSH key with user by ID of an SSH key. +Get SSH key with user by ID of an SSH key. Only available to administrators. ```plaintext GET /keys/:id ``` -| Attribute | Type | Required | Description | -|:----------|:--------|:---------|:---------------------| -| `id` | integer | yes | The ID of an SSH key | +| Attribute | Type | Required | Description | +|:----------|:--------|:---------|:----------------------| +| `id` | integer | yes | The ID of an SSH key. | Example request: @@ -78,9 +78,9 @@ You can search for a user that owns a specific SSH key. Note only administrators GET /keys ``` -| Attribute | Type | Required | Description | -|:--------------|:-------|:---------|:------------------------------| -| `fingerprint` | string | yes | The fingerprint of an SSH key | +| Attribute | Type | Required | Description | +|:--------------|:-------|:---------|:-------------------------------| +| `fingerprint` | string | yes | The fingerprint of an SSH key. | Example request: diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 3e5982faee8..4fabf524702 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -11,6 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - `merged_by` was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/350534) in GitLab 14.7. > - `merge_user` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/349031) as an eventual replacement for `merged_by` in GitLab 14.7. > - `merge_status` was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/3169#note_1162532204) in favor of `detailed_merge_status` in GitLab 15.6. +> - `with_merge_status_recheck` [changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115948) in GitLab 15.11 [with a flag](../administration/feature_flags.md) named `restrict_merge_status_recheck` to be ignored for requests from users insufficient permissions. Disabled by default. Every API call to merge requests must be authenticated. @@ -45,6 +46,7 @@ Supported attributes: | ------------------------------- | -------------- | -------- | ----------- | | `approved_by_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have been approved by all the users with the given `id`. Maximum of 5. `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | | `approver_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have specified all the users with the given `id` as individual approvers. `None` returns merge requests without approvers. `Any` returns merge requests with an approver. | +| `approved` | string | **{dotted-circle}** No | Filters merge requests by their `approved` status. `yes` returns only approved merge requests. `no` returns only non-approved merge requests. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3159) in GitLab 15.11. | | `assignee_id` | integer | **{dotted-circle}** No | Returns merge requests assigned to the given user `id`. `None` returns unassigned merge requests. `Any` returns merge requests with an assignee. | | `author_id` | integer | **{dotted-circle}** No | Returns merge requests created by the given user `id`. Mutually exclusive with `author_username`. Combine with `scope=all` or `scope=assigned_to_me`. | | `author_username` | string | **{dotted-circle}** No | Returns merge requests created by the given `username`. Mutually exclusive with `author_id`. | @@ -71,7 +73,7 @@ Supported attributes: | `updated_before` | datetime | **{dotted-circle}** No | Returns merge requests updated on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | | `view` | string | **{dotted-circle}** No | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request. | | `with_labels_details` | boolean | **{dotted-circle}** No | If `true`, response returns more details for each label in labels field: `:name`, `:color`, `:description`, `:description_html`, `:text_color`. Default is `false`. | -| `with_merge_status_recheck` | boolean | **{dotted-circle}** No | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. | +| `with_merge_status_recheck` | boolean | **{dotted-circle}** No | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. In GitLab 15.11 and later, enable the `restrict_merge_status_recheck` feature [flag](../administration/feature_flags.md) for this attribute to be ignored when requested by users without at least the Developer role. | | `wip` | string | **{dotted-circle}** No | Filter merge requests against their `wip` status. `yes` to return *only* draft merge requests, `no` to return *non-draft* merge requests. | ```json @@ -248,6 +250,7 @@ Supported attributes: | `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | | `approved_by_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have been approved by all the users with the given `id`, with a maximum of 5. `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | | `approver_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have specified all the users with the given `id` as individual approvers. `None` returns merge requests without approvers. `Any` returns merge requests with an approver. | +| `approved` | string | **{dotted-circle}** No | Filters merge requests by their `approved` status. `yes` returns only approved merge requests. `no` returns only non-approved merge requests. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3159) in GitLab 15.11. | | `assignee_id` | integer | **{dotted-circle}** No | Returns merge requests assigned to the given user `id`. `None` returns unassigned merge requests. `Any` returns merge requests with an assignee. | | `author_id` | integer | **{dotted-circle}** No | Returns merge requests created by the given user `id`. Mutually exclusive with `author_username`. | | `author_username` | string | **{dotted-circle}** No | Returns merge requests created by the given `username`. Mutually exclusive with `author_id`. | @@ -273,7 +276,7 @@ Supported attributes: | `view` | string | **{dotted-circle}** No | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request. | | `wip` | string | **{dotted-circle}** No | Filter merge requests against their `wip` status. `yes` to return *only* draft merge requests, `no` to return *non-draft* merge requests. | | `with_labels_details` | boolean | **{dotted-circle}** No | If `true`, response returns more details for each label in labels field: `:name`, `:color`, `:description`, `:description_html`, `:text_color`. Default is `false`. | -| `with_merge_status_recheck` | boolean | **{dotted-circle}** No | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. | +| `with_merge_status_recheck` | boolean | **{dotted-circle}** No | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. In GitLab 15.11 and later, enable the `restrict_merge_status_recheck` feature [flag](../administration/feature_flags.md) for this attribute to be ignored when requested by users without at least the Developer role. | ```json [ @@ -438,6 +441,7 @@ Supported attributes: | `approved_by_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have been approved by all the users with the given `id`, with a maximum of 5. `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | | `approved_by_usernames` **(PREMIUM)** | string array | **{dotted-circle}** No | Returns merge requests which have been approved by all the users with the given `username`, with a maximum of 5. `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | | `approver_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have specified all the users with the given `id`s as individual approvers. `None` returns merge requests without approvers. `Any` returns merge requests with an approver. | +| `approved` | string | **{dotted-circle}** No | Filters merge requests by their `approved` status. `yes` returns only approved merge requests. `no` returns only non-approved merge requests. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3159) in GitLab 15.11. | | `assignee_id` | integer | **{dotted-circle}** No | Returns merge requests assigned to the given user `id`. `None` returns unassigned merge requests. `Any` returns merge requests with an assignee. | | `author_id` | integer | **{dotted-circle}** No | Returns merge requests created by the given user `id`. Mutually exclusive with `author_username`. | | `author_username` | string | **{dotted-circle}** No | Returns merge requests created by the given `username`. Mutually exclusive with `author_id`. | @@ -461,7 +465,7 @@ Supported attributes: | `updated_before` | datetime | **{dotted-circle}** No | Returns merge requests updated on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | | `view` | string | **{dotted-circle}** No | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request. | | `with_labels_details` | boolean | **{dotted-circle}** No | If `true`, response returns more details for each label in labels field: `:name`, `:color`, `:description`, `:description_html`, `:text_color`. Default is `false`. | -| `with_merge_status_recheck` | boolean | **{dotted-circle}** No | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. | +| `with_merge_status_recheck` | boolean | **{dotted-circle}** No | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. In GitLab 15.11 and later, enable the `restrict_merge_status_recheck` feature [flag](../administration/feature_flags.md) for this attribute to be ignored when requested by users without at least the Developer role. | ```json [ @@ -631,13 +635,13 @@ Supported attributes: | `assignees` | array | Assignees of the merge request. | | `author` | object | User who created this merge request. | | `blocking_discussions_resolved` | boolean | Indicates if all discussions are resolved only if all are required before merge request can be merged. | -| `changes_count` | string | Number of changes made on the merge request. | +| `changes_count` | string | Number of changes made on the merge request. Empty when the merge request is created, and populates asynchronously. See [Empty API Fields for new merge requests](#empty-api-fields-for-new-merge-requests).| | `closed_at` | datetime | Timestamp of when the merge request was closed. | | `closed_by` | object | User who closed this merge request. | | `created_at` | datetime | Timestamp of when the merge request was created. | | `description` | string | Description of the merge request. Contains Markdown rendered as HTML for caching. | | `detailed_merge_status` | string | Detailed merge status of the merge request. Read [merge status](#merge-status) for a list of potential values. | -| `diff_refs` | object | References of the base SHA, the head SHA, and the start SHA for this merge request. Corresponds to the latest diff version of the merge request. Empty when the merge request is created, and populates asynchronously. See [Empty `diff_refs` for new merge requests](#empty-diff_refs-for-new-merge-requests). | +| `diff_refs` | object | References of the base SHA, the head SHA, and the start SHA for this merge request. Corresponds to the latest diff version of the merge request. Empty when the merge request is created, and populates asynchronously. See [Empty API fields for new merge requests](#empty-api-fields-for-new-merge-requests). | | `discussion_locked` | boolean | Indicates if comments on the merge request are locked to members only. | | `downvotes` | integer | Number of downvotes for the merge request. | | `draft` | boolean | Indicates if the merge request is a draft. | @@ -842,7 +846,8 @@ Use `detailed_merge_status` instead of `merge_status` to account for all potenti - The `detailed_merge_status` field can contain one of the following values related to the merge request: - `blocked_status`: Blocked by another merge request. - `broken_status`: Can't merge into the target branch due to a potential conflict. - - `checking`: Mergeability checks are still in progress. + - `checking`: Git is testing if a valid merge is possible. + - `unchecked`: Git has not yet tested if a valid merge is possible. - `ci_must_pass`: A CI/CD pipeline must succeed before merge. - `ci_still_running`: A CI/CD pipeline is still running. - `discussions_not_resolved`: All discussions must be resolved before merge. @@ -852,7 +857,6 @@ Use `detailed_merge_status` instead of `merge_status` to account for all potenti - `not_approved`: Approval is required before merge. - `not_open`: The merge request must be open before merge. - `policies_denied`: The merge request contains denied policies. - - `unchecked`: The merge status has not been checked. ## Get single merge request participants @@ -2919,11 +2923,11 @@ To track which state was set, who did it, and when it happened, check out ## Troubleshooting -### Empty `diff_refs` for new merge requests +### Empty API fields for new merge requests -When a merge request is created, the `diff_refs` field is initially empty. This field -is populated asynchronously after the merge request is created. For more -information, see the issue -[`diff_refs` empty after merge request is created](https://gitlab.com/gitlab-org/gitlab/-/issues/386562), +When a merge request is created, the `diff_refs` and `changes_count` fields are +initially empty. These fields are populated asynchronously after the +merge request is created. For more information, see the issue +[Some merge request API fields (`diff_refs`, `changes_count`) empty after MR is created](https://gitlab.com/gitlab-org/gitlab/-/issues/386562), and the [related discussion](https://forum.gitlab.com/t/diff-refs-empty-after-mr-is-created/78975) in the GitLab forums. diff --git a/doc/api/metadata.md b/doc/api/metadata.md index 92f7177482a..9a02ca2a5e4 100644 --- a/doc/api/metadata.md +++ b/doc/api/metadata.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/notes.md b/doc/api/notes.md index 305bdd294c5..48d343267a1 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -422,7 +422,7 @@ Parameters: | `merge_request_iid` | integer | yes | The IID of a project merge request | | `body` | string | yes | The content of a note. Limited to 1,000,000 characters. | | `created_at` | string | no | Date time string, ISO 8601 formatted. Example: `2016-03-11T03:45:40Z` (requires administrator or project/group owner rights) | -| `merge_request_diff_sha`| string | no | Required for the `/merge` [quick action](../user/project/quick_actions.md). The SHA of the head commit, which ensures the merge request wasn't updated after the API request was sent. | +| `merge_request_diff_head_sha`| string | no | Required for the `/merge` [quick action](../user/project/quick_actions.md). The SHA of the head commit, which ensures the merge request wasn't updated after the API request was sent. | ### Modify existing merge request note diff --git a/doc/api/packages/composer.md b/doc/api/packages/composer.md index e75dbfeb92f..857f87a751e 100644 --- a/doc/api/packages/composer.md +++ b/doc/api/packages/composer.md @@ -18,7 +18,7 @@ package registry, see the [Composer package registry documentation](../../user/p NOTE: These endpoints do not adhere to the standard API authentication methods. See the [Composer package registry documentation](../../user/packages/composer_repository/index.md) -for details on which headers and token types are supported. +for details on which headers and token types are supported. Undocumented authentication methods might be removed in the future. ## Base repository request diff --git a/doc/api/packages/conan.md b/doc/api/packages/conan.md index c37296ad664..0dc6f68ed4c 100644 --- a/doc/api/packages/conan.md +++ b/doc/api/packages/conan.md @@ -17,7 +17,7 @@ package registry, see the [Conan package registry documentation](../../user/pack NOTE: These endpoints do not adhere to the standard API authentication methods. -See each route for details on how credentials are expected to be passed. +See each route for details on how credentials are expected to be passed. Undocumented authentication methods might be removed in the future. NOTE: The Conan registry is not FIPS compliant and is disabled when [FIPS mode](../../development/fips_compliance.md) is enabled. diff --git a/doc/api/packages/debian.md b/doc/api/packages/debian.md index 87b583de5e6..267ebb59c19 100644 --- a/doc/api/packages/debian.md +++ b/doc/api/packages/debian.md @@ -24,7 +24,7 @@ package registry, see the [Debian registry documentation](../../user/packages/de NOTE: These endpoints do not adhere to the standard API authentication methods. See the [Debian registry documentation](../../user/packages/debian_repository/index.md) -for details on which headers and token types are supported. +for details on which headers and token types are supported. Undocumented authentication methods might be removed in the future. ## Enable the Debian API @@ -46,7 +46,8 @@ See [Authenticate to the Debian Package Repositories](../../user/packages/debian ## Upload a package file -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62028) in GitLab 14.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62028) in GitLab 14.0. +> - Upload with explicit distribution and component [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101838) in GitLab 15.9. Upload a Debian package file: @@ -54,18 +55,29 @@ Upload a Debian package file: PUT projects/:id/packages/debian/:file_name ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | string | yes | The ID or full path of the project. | -| `file_name` | string | yes | The name of the Debian package file. | +| Attribute | Type | Required | Description | +| -------------- | ------ | -------- | ----------- | +| `id` | string | yes | The ID or full path of the project. | +| `file_name` | string | yes | The name of the Debian package file. | +| `distribution` | string | no | The distribution codename or suite. Used with `component` for upload with explicit distribution and component. | +| `component` | string | no | The package file component. Used with `distribution` for upload with explicit distribution and component. | ```shell curl --request PUT \ - --user <username>:<personal_access_token> \ + --user "<username>:<personal_access_token>" \ --upload-file path/to/mypkg.deb \ "https://gitlab.example.com/api/v4/projects/1/packages/debian/mypkg.deb" ``` +Upload with explicit distribution and component: + +```shell +curl --request PUT \ + --user "<username>:<personal_access_token>" \ + --upload-file /path/to/myother.deb \ + "https://gitlab.example.com/api/v4/projects/1/packages/debian/myother.deb?distribution=sid&component=main" +``` + ## Download a package > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64923) in GitLab 14.2. diff --git a/doc/api/packages/debian_group_distributions.md b/doc/api/packages/debian_group_distributions.md index 23bc85bf0a0..0c7f4cdfeb8 100644 --- a/doc/api/packages/debian_group_distributions.md +++ b/doc/api/packages/debian_group_distributions.md @@ -45,7 +45,7 @@ GET /groups/:id/-/debian_distributions | `suite` | string | no | Filter with specific `suite`. | ```shell -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/debian_distributions" +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/-/debian_distributions" ``` Example response: @@ -86,7 +86,7 @@ GET /groups/:id/-/debian_distributions/:codename | `codename` | integer | yes | The `codename` of a distribution. | ```shell -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/debian_distributions/unstable" +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/-/debian_distributions/unstable" ``` Example response: @@ -125,7 +125,7 @@ GET /groups/:id/-/debian_distributions/:codename/key.asc | `codename` | integer | yes | The `codename` of a distribution. | ```shell -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/debian_distributions/unstable/key.asc" +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/-/debian_distributions/unstable/key.asc" ``` Example response: @@ -170,7 +170,7 @@ POST /groups/:id/-/debian_distributions | `architectures` | architectures | no | The new Debian distribution's list of architectures. | ```shell -curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/debian_distributions?codename=sid" +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/-/debian_distributions?codename=sid" ``` Example response: @@ -217,7 +217,7 @@ PUT /groups/:id/-/debian_distributions/:codename | `architectures` | architectures | no | The Debian distribution's new list of architectures. | ```shell -curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/debian_distributions/unstable?suite=new-suite&valid_time_duration_seconds=604800" +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/-/debian_distributions/unstable?suite=new-suite&valid_time_duration_seconds=604800" ``` Example response: @@ -256,5 +256,5 @@ DELETE /groups/:id/-/debian_distributions/:codename | `codename` | integer | yes | The codename of the Debian distribution. | ```shell -curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/debian_distributions/unstable" +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/-/debian_distributions/unstable" ``` diff --git a/doc/api/packages/go_proxy.md b/doc/api/packages/go_proxy.md index 08a7138a82a..1cfc0f0b296 100644 --- a/doc/api/packages/go_proxy.md +++ b/doc/api/packages/go_proxy.md @@ -20,7 +20,7 @@ For instructions on how to work with the Go Proxy, see the [Go Proxy package doc NOTE: These endpoints do not adhere to the standard API authentication methods. See the [Go Proxy package documentation](../../user/packages/go_proxy/index.md) -for details on which headers and token types are supported. +for details on which headers and token types are supported. Undocumented authentication methods might be removed in the future. ## List diff --git a/doc/api/packages/helm.md b/doc/api/packages/helm.md index 0e9a72c2389..d69f524c47c 100644 --- a/doc/api/packages/helm.md +++ b/doc/api/packages/helm.md @@ -19,7 +19,7 @@ Package Registry, see the [Helm registry documentation](../../user/packages/helm NOTE: These endpoints do not adhere to the standard API authentication methods. See the [Helm registry documentation](../../user/packages/helm_repository/index.md) -for details on which headers and token types are supported. +for details on which headers and token types are supported. Undocumented authentication methods might be removed in the future. ## Download a chart index diff --git a/doc/api/packages/maven.md b/doc/api/packages/maven.md index 4086b68b750..733f4735ba2 100644 --- a/doc/api/packages/maven.md +++ b/doc/api/packages/maven.md @@ -18,7 +18,7 @@ package registry, see the [Maven package registry documentation](../../user/pack NOTE: These endpoints do not adhere to the standard API authentication methods. See [Maven package registry documentation](../../user/packages/maven_repository/index.md) -for details on which headers and token types are supported. +for details on which headers and token types are supported. Undocumented authentication methods might be removed in the future. ## Download a package file at the instance-level diff --git a/doc/api/packages/npm.md b/doc/api/packages/npm.md index 7e8732d9553..bf48fbc8f65 100644 --- a/doc/api/packages/npm.md +++ b/doc/api/packages/npm.md @@ -18,7 +18,7 @@ package registry, see the [npm package registry documentation](../../user/packag NOTE: These endpoints do not adhere to the standard API authentication methods. See the [npm package registry documentation](../../user/packages/npm_registry/index.md) -for details on which headers and token types are supported. +for details on which headers and token types are supported. Undocumented authentication methods might be removed in the future. ## Download a package diff --git a/doc/api/packages/nuget.md b/doc/api/packages/nuget.md index ebcb51be447..a0761b56645 100644 --- a/doc/api/packages/nuget.md +++ b/doc/api/packages/nuget.md @@ -18,7 +18,7 @@ package registry, see the [NuGet package registry documentation](../../user/pack NOTE: These endpoints do not adhere to the standard API authentication methods. See the [NuGet package registry documentation](../../user/packages/nuget_repository/index.md) -for details on which headers and token types are supported. +for details on which headers and token types are supported. Undocumented authentication methods might be removed in the future. ## Package index diff --git a/doc/api/packages/pypi.md b/doc/api/packages/pypi.md index e9546f50e36..4e7c59adf3a 100644 --- a/doc/api/packages/pypi.md +++ b/doc/api/packages/pypi.md @@ -18,7 +18,7 @@ package registry, see the [PyPI package registry documentation](../../user/packa NOTE: These endpoints do not adhere to the standard API authentication methods. See the [PyPI package registry documentation](../../user/packages/pypi_repository/index.md) -for details on which headers and token types are supported. +for details on which headers and token types are supported. Undocumented authentication methods might be removed in the future. NOTE: [Twine 3.4.2](https://twine.readthedocs.io/en/stable/changelog.html?highlight=FIPS#id28) or greater diff --git a/doc/api/packages/rubygems.md b/doc/api/packages/rubygems.md index c2d05f24085..17338161a7b 100644 --- a/doc/api/packages/rubygems.md +++ b/doc/api/packages/rubygems.md @@ -19,7 +19,7 @@ package registry, see the [Ruby gems registry documentation](../../user/packages NOTE: These endpoints do not adhere to the standard API authentication methods. See the [Ruby gems registry documentation](../../user/packages/rubygems_registry/index.md) -for details on which headers and token types are supported. +for details on which headers and token types are supported. Undocumented authentication methods might be removed in the future. ## Enable the Ruby gems API diff --git a/doc/api/packages/terraform-modules.md b/doc/api/packages/terraform-modules.md index 6b3d6b477b2..ff6ac24495e 100644 --- a/doc/api/packages/terraform-modules.md +++ b/doc/api/packages/terraform-modules.md @@ -4,16 +4,16 @@ group: Package Registry info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Terraform Registry API **(FREE)** +# Terraform Module Registry API **(FREE)** -This is the API documentation for [Terraform Modules](../../user/packages/terraform_module_registry/index.md). +This is the API documentation for the [Terraform Module Registry](../../user/packages/terraform_module_registry/index.md). WARNING: This API is used by the [Terraform CLI](https://www.terraform.io/) -and is generally not meant for manual consumption. +and is generally not meant for manual consumption. Undocumented authentication methods might be removed in the future. For instructions on how to upload and install Terraform modules from the GitLab -infrastructure registry, see the [Terraform modules registry documentation](../../user/packages/terraform_module_registry/index.md). +Terraform Module Registry, see the [Terraform Module Registry documentation](../../user/packages/terraform_module_registry/index.md). ## List available versions for a specific module @@ -35,7 +35,7 @@ curl --header "Authorization: Bearer <personal_access_token>" "https://gitlab.ex Example response: -```shell +```json { "modules": [ { @@ -93,9 +93,9 @@ curl --header "Authorization: Bearer <personal_access_token>" "https://gitlab.ex Example response: -```shell +```json { - "name": "hellow-world/local", + "name": "hello-world/local", "provider": "local", "providers": [ "local" @@ -132,9 +132,9 @@ curl --header "Authorization: Bearer <personal_access_token>" "https://gitlab.ex Example response: -```shell +```json { - "name": "hellow-world/local", + "name": "hello-world/local", "provider": "local", "providers": [ "local" @@ -171,7 +171,7 @@ curl --header "Authorization: Bearer <personal_access_token>" "https://gitlab.ex Example response: -```shell +```plaintext HTTP/1.1 204 No Content Content-Length: 0 X-Terraform-Get: /api/v4/packages/terraform/modules/v1/group/hello-world/local/1.0.0/file?token=&archive=tgz @@ -200,7 +200,7 @@ curl --header "Authorization: Bearer <personal_access_token>" "https://gitlab.ex Example response: -```shell +```plaintext HTTP/1.1 204 No Content Content-Length: 0 X-Terraform-Get: /api/v4/packages/terraform/modules/v1/group/hello-world/local/1.0.0/file?token=&archive=tgz diff --git a/doc/api/pages.md b/doc/api/pages.md index ed63090b9be..2821f5d510c 100644 --- a/doc/api/pages.md +++ b/doc/api/pages.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/pages_domains.md b/doc/api/pages_domains.md index e83811d0415..7d52c803c88 100644 --- a/doc/api/pages_domains.md +++ b/doc/api/pages_domains.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index 7049d3c3927..fed0e553a9e 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -15,7 +15,14 @@ Read more on [pagination](rest/index.md#pagination). ## List project pipelines -> `iid` in response [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342223) in GitLab 14.6. +> - `iid` in response [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342223) in GitLab 14.6. +> - `name` in request and response [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115310) in GitLab 15.11 [with a flag](../administration/feature_flags.md) named `pipeline_name_in_api`. Disabled by default. + +FLAG: +On self-managed GitLab, by default the `name` field is not available. +To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) +named `pipeline_name_in_api`. This feature is not ready for production use. +On GitLab.com, this feature is not available. List pipelines in a project. Child pipelines are not included in the results, but you can [get child pipeline](pipelines.md#get-a-single-pipeline) individually. @@ -36,6 +43,7 @@ GET /projects/:id/pipelines | `username`| string | no | The username of the user who triggered pipelines | | `updated_after` | datetime | no | Return pipelines updated after the specified date. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | | `updated_before` | datetime | no | Return pipelines updated before the specified date. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `name` | string | no | Return pipelines with the specified name. Introduced in GitLab 15.11, not available by default. | | `order_by`| string | no | Order pipelines by `id`, `status`, `ref`, `updated_at` or `user_id` (default: `id`) | | `sort` | string | no | Sort pipelines in `asc` or `desc` order (default: `desc`) | @@ -55,6 +63,7 @@ Example of response "source": "push", "ref": "new-pipeline", "sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "name": "Build pipeline", "web_url": "https://example.com/foo/bar/pipelines/47", "created_at": "2016-08-11T11:28:34.085Z", "updated_at": "2016-08-11T11:32:35.169Z" @@ -67,6 +76,7 @@ Example of response "source": "web", "ref": "new-pipeline", "sha": "eb94b618fb5865b26e80fdd8ae531b7a63ad851a", + "name": "Build pipeline", "web_url": "https://example.com/foo/bar/pipelines/48", "created_at": "2016-08-12T10:06:04.561Z", "updated_at": "2016-08-12T10:09:56.223Z" @@ -76,7 +86,14 @@ Example of response ## Get a single pipeline -> `iid` in response [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342223) in GitLab 14.6. +> - `iid` in response [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342223) in GitLab 14.6. +> - `name` in response [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115310) in GitLab 15.11 [with a flag](../administration/feature_flags.md) named `pipeline_name_in_api`. Disabled by default. + +FLAG: +On self-managed GitLab, by default the `name` field is not available. +To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) +named `pipeline_name_in_api`. This feature is not ready for production use. +On GitLab.com, this feature is not available. Get one pipeline from a project. @@ -103,6 +120,7 @@ Example of response "id": 46, "iid": 11, "project_id": 1, + "name": "Build pipeline", "status": "success", "ref": "main", "sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", @@ -271,6 +289,14 @@ Sample response: ## Get the latest pipeline +> `name` in response [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115310) in GitLab 15.11 [with a flag](../administration/feature_flags.md) named `pipeline_name_in_api`. Disabled by default. + +FLAG: +On self-managed GitLab, by default the `name` field is not available. +To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) +named `pipeline_name_in_api`. This feature is not ready for production use. +On GitLab.com, this feature is not available. + Get the latest pipeline for a specific ref in a project. ```plaintext @@ -292,6 +318,7 @@ Example of response "id": 287, "iid": 144, "project_id": 21, + "name": "Build pipeline", "sha": "50f0acb76a40e34a4ff304f7347dcc6587da8a14", "ref": "main", "status": "success", diff --git a/doc/api/product_analytics.md b/doc/api/product_analytics.md index 8eda24d1c65..c37fe223778 100644 --- a/doc/api/product_analytics.md +++ b/doc/api/product_analytics.md @@ -8,6 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Introduced in GitLab 15.4 [with a flag](../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default. > - `cube_api_proxy` removed and replaced with `product_analytics_internal_preview` in GitLab 15.10. +> - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `cube_api_proxy`. diff --git a/doc/api/project_clusters.md b/doc/api/project_clusters.md index f6d572f9a7d..65efcaf13b9 100644 --- a/doc/api/project_clusters.md +++ b/doc/api/project_clusters.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/project_import_export.md b/doc/api/project_import_export.md index 758c91dd3e7..98f154c17d7 100644 --- a/doc/api/project_import_export.md +++ b/doc/api/project_import_export.md @@ -8,14 +8,16 @@ info: To determine the technical writer assigned to the Stage/Group associated w Use the project import and export API to import and export projects using file transfers. -For more information, see: - -- [Migrating projects using file exports](../user/project/settings/import_export.md) -- [Project import and export Rake tasks](../administration/raketasks/project_import_export.md) - Before using the project import and export API, you might want to use the [group import and export API](group_import_export.md). +## Prerequisites + +For information on prerequisites for project import and export API, see: + +- Prerequisites for [project export](../user/project/settings/import_export.md#export-a-project-and-its-data). +- Prerequisites for [project import](../user/project/settings/import_export.md#import-a-project-and-its-data). + ## Schedule an export Start a new export. @@ -44,14 +46,15 @@ POST /projects/:id/export | Attribute | Type | Required | Description | | --------- | -------------- | -------- | ---------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `description` | string | no | Overrides the project description | -| `upload` | hash | no | Hash that contains the information to upload the exported project to a web server | -| `upload[url]` | string | yes | The URL to upload the project | -| `upload[http_method]` | string | no | The HTTP method to upload the exported project. Only `PUT` and `POST` methods allowed. Default is `PUT` | +| `id` | integer or string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. +| `upload[url]` | string | yes | The URL to upload the project. +| `description` | string | no | Overrides the project description. +| `upload` | hash | no | Hash that contains the information to upload the exported project to a web server. +| `upload[http_method]` | string | no | The HTTP method to upload the exported project. Only `PUT` and `POST` methods allowed. Default is `PUT`. ```shell -curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/export" \ +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/1/export" \ --data "upload[http_method]=PUT" \ --data-urlencode "upload[url]=https://example-bucket.s3.eu-west-3.amazonaws.com/backup?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIMBJHN2O62W8IELQ%2F20180312%2Feu-west-3%2Fs3%2Faws4_request&X-Amz-Date=20180312T110328Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=8413facb20ff33a49a147a0b4abcff4c8487cc33ee1f7e450c46e8f695569dbd" ``` @@ -72,10 +75,11 @@ GET /projects/:id/export | Attribute | Type | Required | Description | | --------- | -------------- | -------- | ---------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer or string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. ```shell -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/export" +curl --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/1/export" ``` Status can be one of: @@ -118,9 +122,9 @@ Download the finished export. GET /projects/:id/export/download ``` -| Attribute | Type | Required | Description | -| --------- | -------------- | -------- | ---------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | +| Attribute | Type | Required | Description | +| --------- | ----------------- | -------- | ---------------------------------------- | +| `id` | integer or string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" --remote-header-name \ @@ -138,15 +142,14 @@ ls *export.tar.gz POST /projects/import ``` -| Attribute | Type | Required | Description | -| --------- | -------------- | -------- | ---------------------------------------- | -<!-- markdownlint-disable-next-line gitlab.SentenceSpacing --> -| `namespace` | integer/string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace.<br/><br/>Requires at least the Maintainer role on the destination group to import to. Using the Developer role for this purpose was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. | -| `name` | string | no | The name of the project to be imported. Defaults to the path of the project if not provided | -| `file` | string | yes | The file to be uploaded | -| `path` | string | yes | Name and path for new project | -| `overwrite` | boolean | no | If there is a project with the same path the import overwrites it. Default to false | -| `override_params` | Hash | no | Supports all fields defined in the [Project API](projects.md) | +| Attribute | Type | Required | Description | +| ----------- | -------------- | -------- | ---------------------------------------- | +| `file` | string | yes | The file to be uploaded. +| `path` | string | yes | Name and path for new project. +| `name` | string | no | The name of the project to be imported. Defaults to the path of the project if not provided. +| `namespace` | integer or string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace.<br/><br/> Requires at least the Maintainer role on the destination group to import to. Using the Developer role for this purpose was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and is scheduled for removal in GitLab 16.0. +| `override_params` | Hash | no | Supports all fields defined in the [Project API](projects.md). +| `overwrite` | boolean | no | If there is a project with the same path the import overwrites it. Defaults to `false`. The override parameters passed take precedence over all values defined inside the export file. @@ -195,39 +198,29 @@ requests.post(url, headers=headers, data=data, files=files) ``` NOTE: -The maximum import file size can be set by the Administrator, default is `0` (unlimited).. +The maximum import file size can be set by the Administrator. It defaults to `0` (unlimited). As an administrator, you can modify the maximum import file size. To do so, use the `max_import_size` option in the [Application settings API](settings.md#change-application-settings) or the [Admin Area](../user/admin_area/settings/account_and_limit_settings.md). Default [modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to 0 in GitLab 13.8. -## Import a file from a remote object storage - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/282503) in GitLab 13.12 in [Beta](../policy/alpha-beta-support.md#beta-features). - -This endpoint is behind a feature flag that is enabled by default. +## Import a file from a remote object storage (Beta) -To enable this endpoint: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/282503) in GitLab 13.12 in [Beta](../policy/alpha-beta-support.md#beta) [with a flag](../administration/feature_flags.md) named `import_project_from_remote_file`. Enabled by default. -```ruby -Feature.enable(:import_project_from_remote_file) -``` - -To disable this endpoint: - -```ruby -Feature.disable(:import_project_from_remote_file) -``` +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `import_project_from_remote_file`. +On GitLab.com, this feature is available. ```plaintext POST /projects/remote-import ``` -| Attribute | Type | Required | Description | -| ----------------- | -------------- | -------- | ---------------------------------------- | -| `namespace` | integer/string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace. | -| `name` | string | no | The name of the project to import. If not provided, defaults to the path of the project. | -| `url` | string | yes | URL for the file to import. | -| `path` | string | yes | Name and path for the new project. | -| `overwrite` | boolean | no | Whether to overwrite a project with the same path when importing. Defaults to `false`. | -| `override_params` | Hash | no | Supports all fields defined in the [Project API](projects.md). | +| Attribute | Type | Required | Description | +| ----------------- | ----------------- | -------- | ---------------------------------------- | +| `path` | string | yes | Name and path for the new project. +| `url` | string | yes | URL for the file to import. +| `name` | string | no | The name of the project to import. If not provided, defaults to the path of the project. +| `namespace` | integer or string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace. +| `overwrite` | boolean | no | Whether to overwrite a project with the same path when importing. Defaults to `false`. +| `override_params` | Hash | no | Supports all fields defined in the [Project API](projects.md). The passed override parameters take precedence over all values defined in the export file. @@ -255,16 +248,14 @@ curl --request POST \ } ``` -The `Content-Length` header must return a valid number. The maximum file size is 10 gigabytes. +The `Content-Length` header must return a valid number. The maximum file size is 10 GB. The `Content-Type` header must be `application/gzip`. ## Import a file from AWS S3 > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348874) in GitLab 14.9 in [Beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta), [with a flag](../administration/feature_flags.md) named `import_project_from_remote_file_s3`. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/348874) in GitLab 14.10. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `import_project_from_remote_file_s3`. On GitLab.com, this feature is available. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/350571) in GitLab 15.11. Feature flag `import_project_from_remote_file_s3` removed. ```plaintext POST /projects/remote-import-s3 @@ -272,14 +263,14 @@ POST /projects/remote-import-s3 | Attribute | Type | Required | Description | | ------------------- | -------------- | -------- | ---------------------------------------- | -| `namespace` | integer/string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace. | -| `name` | string | no | The name of the project to import. If not provided, defaults to the path of the project. | -| `path` | string | yes | The full path of the new project. | -| `region` | string | yes | [AWS S3 region name where the file is stored.](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html#Regions) | -| `bucket_name` | string | yes | [AWS S3 bucket name where the file is stored.](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html) | -| `file_key` | string | yes | [AWS S3 file key to identify the file.](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingObjects.html) | -| `access_key_id` | string | yes | [AWS S3 access key ID.](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys). | -| `secret_access_key` | string | yes | [AWS S3 secret access key.](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) | +| `access_key_id` | string | yes | [AWS S3 access key ID](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys). +| `bucket_name` | string | yes | [AWS S3 bucket name](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html) where the file is stored. +| `file_key` | string | yes | [AWS S3 file key](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingObjects.html) to identify the file. +| `path` | string | yes | The full path of the new project. +| `region` | string | yes | [AWS S3 region name](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html#Regions) where the file is stored. +| `secret_access_key` | string | yes | [AWS S3 secret access key](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys). +| `name` | string | no | The name of the project to import. If not provided, defaults to the path of the project. +| `namespace` | integer or string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace. The passed override parameters take precedence over all values defined in the export file. @@ -346,10 +337,11 @@ GET /projects/:id/import | Attribute | Type | Required | Description | | --------- | -------------- | -------- | ---------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer or string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. ```shell -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/import" +curl --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/1/import" ``` Status can be one of: @@ -362,8 +354,10 @@ Status can be one of: If the status is `failed`, it includes the import error message under `import_error`. If the status is `failed`, `started` or `finished`, the `failed_relations` array might -be populated with any occurrences of relations that failed to import either due to -unrecoverable errors or because retries were exhausted (a typical example are query timeouts.) +be populated with any occurrences of relations that failed to import due to either: + +- Unrecoverable errors. +- Retries were exhausted. A typical example: query timeouts. NOTE: An element's `id` field in `failed_relations` references the failure record, not the relation. @@ -445,3 +439,8 @@ GitHub and how many were already imported: } } ``` + +## Related topics + +- [Migrating projects using file exports](../user/project/settings/import_export.md). +- [Project import and export Rake tasks](../administration/raketasks/project_import_export.md). diff --git a/doc/api/projects.md b/doc/api/projects.md index 3cd7cdd811d..35400b0d540 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -67,6 +67,8 @@ GET /projects | `with_issues_enabled` | boolean | **{dotted-circle}** No | Limit by enabled issues feature. | | `with_merge_requests_enabled` | boolean | **{dotted-circle}** No | Limit by enabled merge requests feature. | | `with_programming_language` | string | **{dotted-circle}** No | Limit by projects which use the given programming language. | +| `updated_before` | datetime | **{dotted-circle}** No | Limit results to projects last updated before the specified time. Format: ISO 8601 (`YYYY-MM-DDTHH:MM:SSZ`). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393979) in GitLab 15.10. For this filter to work, you must also provide `updated_at` as the `order_by` attribute. | +| `updated_after` | datetime | **{dotted-circle}** No | Limit results to projects last updated after the specified time. Format: ISO 8601 (`YYYY-MM-DDTHH:MM:SSZ`). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393979) in GitLab 15.10. For this filter to work, you must also provide `updated_at` as the `order_by` attribute. | This endpoint supports [keyset pagination](rest/index.md#keyset-based-pagination) for selected `order_by` options. @@ -341,6 +343,8 @@ GET /users/:user_id/projects | `with_issues_enabled` | boolean | **{dotted-circle}** No | Limit by enabled issues feature. | | `with_merge_requests_enabled` | boolean | **{dotted-circle}** No | Limit by enabled merge requests feature. | | `with_programming_language` | string | **{dotted-circle}** No | Limit by projects which use the given programming language. | +| `updated_before` | datetime | **{dotted-circle}** No | Limit results to projects last updated before the specified time. Format: ISO 8601 (`YYYY-MM-DDTHH:MM:SSZ`). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393979) in GitLab 15.10. | +| `updated_after` | datetime | **{dotted-circle}** No | Limit results to projects last updated after the specified time. Format: ISO 8601 (`YYYY-MM-DDTHH:MM:SSZ`). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393979) in GitLab 15.10. | ```json [ @@ -607,6 +611,8 @@ GET /users/:user_id/starred_projects | `with_custom_attributes` | boolean | **{dotted-circle}** No | Include [custom attributes](custom_attributes.md) in response. _(administrator only)_ | | `with_issues_enabled` | boolean | **{dotted-circle}** No | Limit by enabled issues feature. | | `with_merge_requests_enabled` | boolean | **{dotted-circle}** No | Limit by enabled merge requests feature. | +| `updated_before` | datetime | **{dotted-circle}** No | Limit results to projects last updated before the specified time. Format: ISO 8601 (`YYYY-MM-DDTHH:MM:SSZ`). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393979) in GitLab 15.10. | +| `updated_after` | datetime | **{dotted-circle}** No | Limit results to projects last updated after the specified time. Format: ISO 8601 (`YYYY-MM-DDTHH:MM:SSZ`). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393979) in GitLab 15.10. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/users/5/starred_projects" @@ -1568,6 +1574,8 @@ GET /projects/:id/forks | `with_custom_attributes` | boolean | **{dotted-circle}** No | Include [custom attributes](custom_attributes.md) in response. _(administrators only)_ | | `with_issues_enabled` | boolean | **{dotted-circle}** No | Limit by enabled issues feature. | | `with_merge_requests_enabled` | boolean | **{dotted-circle}** No | Limit by enabled merge requests feature. | +| `updated_before` | datetime | **{dotted-circle}** No | Limit results to projects last updated before the specified time. Format: ISO 8601 (`YYYY-MM-DDTHH:MM:SSZ`). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393979) in GitLab 15.10. | +| `updated_after` | datetime | **{dotted-circle}** No | Limit results to projects last updated after the specified time. Format: ISO 8601 (`YYYY-MM-DDTHH:MM:SSZ`). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393979) in GitLab 15.10. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/forks" @@ -2215,6 +2223,9 @@ This endpoint: projects within a group to be deleted after a delayed period. When enabled, actual deletion happens after the number of days specified in the [default deletion delay](../user/admin_area/settings/visibility_and_access_controls.md#deletion-protection). +- From [GitLab 15.11](https://gitlab.com/gitlab-org/gitlab/-/issues/396500) on + [Premium or higher](https://about.gitlab.com/pricing/) tiers, deletes a project immediately if the project is already + marked for deletion, and the `permanently_remove` and `full_path` parameters are passed. WARNING: The default behavior of [Delayed Project deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) @@ -2225,9 +2236,11 @@ in GitLab 13.2, as discussed in [Enable delayed project deletion](../user/group/ DELETE /projects/:id ``` -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| Attribute | Type | Required | Description | +|------------------------------------|-------------------|------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `permanently_remove` **(PREMIUM)** | boolean/string | no | Immediately deletes a project if it is marked for deletion. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/396500) in GitLab 15.11 | +| `full_path` **(PREMIUM)** | string | no | Full path of project to use with `permanently_remove`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/396500) in GitLab 15.11. To find the project path, use `path_with_namespace` from [get single project](projects.md#get-single-project) | ## Restore project marked for deletion **(PREMIUM)** diff --git a/doc/api/protected_branches.md b/doc/api/protected_branches.md index 6c3bd63bbad..9bd220f0406 100644 --- a/doc/api/protected_branches.md +++ b/doc/api/protected_branches.md @@ -214,7 +214,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitla | `allowed_to_merge` **(PREMIUM)** | array | no | Array of access levels allowed to merge, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. | `allowed_to_push` **(PREMIUM)** | array | no | Array of access levels allowed to push, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. | `allowed_to_unprotect` **(PREMIUM)** | array | no | Array of access levels allowed to unprotect, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. The access level `No access` is not available for this field. | -| `code_owner_approval_required` **(PREMIUM)** | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/code_owners.md). (defaults: false) +| `code_owner_approval_required` **(PREMIUM)** | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/codeowners/index.md). (defaults: false) | `merge_access_level` | integer | no | Access levels allowed to merge. (defaults: `40`, Maintainer role) | `push_access_level` | integer | no | Access levels allowed to push. (defaults: `40`, Maintainer role) | `unprotect_access_level` | integer | no | Access levels allowed to unprotect. (defaults: `40`, Maintainer role) @@ -445,7 +445,7 @@ curl --request PATCH --header "PRIVATE-TOKEN: <your_access_token>" "https://gitl | `allowed_to_push` **(PREMIUM)** | array | no | Array of push access levels, with each described by a hash. | `allowed_to_merge` **(PREMIUM)** | array | no | Array of merge access levels, with each described by a hash. | `allowed_to_unprotect` **(PREMIUM)** | array | no | Array of unprotect access levels, with each described by a hash. The access level `No access` is not available for this field. -| `code_owner_approval_required` **(PREMIUM)** | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/code_owners.md). Defaults to `false`. | +| `code_owner_approval_required` **(PREMIUM)** | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/codeowners/index.md). Defaults to `false`. | Elements in the `allowed_to_push`, `allowed_to_merge` and `allowed_to_unprotect` arrays should be one of `user_id`, `group_id` or `access_level`, and take the form `{user_id: integer}`, `{group_id: integer}` or diff --git a/doc/api/protected_environments.md b/doc/api/protected_environments.md index 877add32fed..023046a4821 100644 --- a/doc/api/protected_environments.md +++ b/doc/api/protected_environments.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/api/releases/index.md b/doc/api/releases/index.md index 0d8e335b620..8956a71e403 100644 --- a/doc/api/releases/index.md +++ b/doc/api/releases/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/releases/links.md b/doc/api/releases/links.md index 2294f161759..7c2cac90a3d 100644 --- a/doc/api/releases/links.md +++ b/doc/api/releases/links.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/repositories.md b/doc/api/repositories.md index a34fde54e90..d3bee8de2c5 100644 --- a/doc/api/repositories.md +++ b/doc/api/repositories.md @@ -164,7 +164,8 @@ Supported attributes: Example request: ```shell -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/<project_id>/repository/archive?sha=<commit_sha>&path=<path>" +curl --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.com/api/v4/projects/<project_id>/repository/archive?sha=<commit_sha>&path=<path>" ``` ## Compare branches, tags or commits @@ -278,10 +279,11 @@ GET /projects/:id/repository/merge_base | `id` | integer or string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | | `refs` | array | yes | The refs to find the common ancestor of. Accepts multiple refs. | -Example request: +Example request, with the refs truncated for readability: ```shell -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/repository/merge_base?refs[]=304d257dcb821665ab5110318fc58a007bd104ed&refs[]=0031876facac3f2b2702a0e53a26e89939a42209" +curl --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/5/repository/merge_base?refs[]=304d257d&refs[]=0031876f" ``` Example response: @@ -385,26 +387,30 @@ If the last tag is `v0.9.0` and the default branch is `main`, the range of commi included in this example is `v0.9.0..main`: ```shell -curl --request POST --header "PRIVATE-TOKEN: token" --data "version=1.0.0" "https://gitlab.com/api/v4/projects/42/repository/changelog" +curl --request POST --header "PRIVATE-TOKEN: token" \ + --data "version=1.0.0" "https://gitlab.com/api/v4/projects/42/repository/changelog" ``` To generate the data on a different branch, specify the `branch` parameter. This command generates data from the `foo` branch: ```shell -curl --request POST --header "PRIVATE-TOKEN: token" --data "version=1.0.0&branch=foo" "https://gitlab.com/api/v4/projects/42/repository/changelog" +curl --request POST --header "PRIVATE-TOKEN: token" \ + --data "version=1.0.0&branch=foo" "https://gitlab.com/api/v4/projects/42/repository/changelog" ``` To use a different trailer, use the `trailer` parameter: ```shell -curl --request POST --header "PRIVATE-TOKEN: token" --data "version=1.0.0&trailer=Type" "https://gitlab.com/api/v4/projects/42/repository/changelog" +curl --request POST --header "PRIVATE-TOKEN: token" \ + --data "version=1.0.0&trailer=Type" "https://gitlab.com/api/v4/projects/42/repository/changelog" ``` To store the results in a different file, use the `file` parameter: ```shell -curl --request POST --header "PRIVATE-TOKEN: token" --data "version=1.0.0&file=NEWS" "https://gitlab.com/api/v4/projects/42/repository/changelog" +curl --request POST --header "PRIVATE-TOKEN: token" \ + --data "version=1.0.0&file=NEWS" "https://gitlab.com/api/v4/projects/42/repository/changelog" ``` ## Generate changelog data @@ -426,25 +432,30 @@ Supported attributes: | Attribute | Type | Required | Description | | :-------- | :------- | :--------- | :---------- | | `version` | string | yes | The version to generate the changelog for. The format must follow [semantic versioning](https://semver.org/). | -| `config_file` | string | no | The path of changelog configuration file in the project's Git repository, defaults to `.gitlab/changelog_config.yml`. | -| `date` | datetime | no | The date and time of the release, ISO 8601 formatted. Example: `2016-03-11T03:45:40Z`. Defaults to the current time. | +| `config_file` | string | no | The path of changelog configuration file in the project's Git repository. Defaults to `.gitlab/changelog_config.yml`. | +| `date` | datetime | no | The date and time of the release. Uses ISO 8601 format. Example: `2016-03-11T03:45:40Z`. Defaults to the current time. | | `from` | string | no | The start of the range of commits (as a SHA) to use for generating the changelog. This commit itself isn't included in the list. | | `to` | string | no | The end of the range of commits (as a SHA) to use for the changelog. This commit _is_ included in the list. Defaults to the HEAD of the default project branch. | -| `trailer` | string | no | The Git trailer to use for including commits, defaults to `Changelog`. | +| `trailer` | string | no | The Git trailer to use for including commits. Defaults to `Changelog`. | ```shell -curl --header "PRIVATE-TOKEN: token" "https://gitlab.com/api/v4/projects/42/repository/changelog?version=1.0.0" +curl --header "PRIVATE-TOKEN: token" \ + "https://gitlab.com/api/v4/projects/42/repository/changelog?version=1.0.0" ``` -Example Response: +Example response, with line breaks added for readability: ```json { - "notes": "## 1.0.0 (2021-11-17)\n\n### feature (2 changes)\n\n- [Title 2](namespace13/project13@ad608eb642124f5b3944ac0ac772fecaf570a6bf) ([merge request](namespace13/project13!2))\n- [Title 1](namespace13/project13@3c6b80ff7034fa0d585314e1571cc780596ce3c8) ([merge request](namespace13/project13!1))\n" + "notes": "## 1.0.0 (2021-11-17)\n\n### feature (2 changes)\n\n- + [Title 2](namespace13/project13@ad608eb642124f5b3944ac0ac772fecaf570a6bf) + ([merge request](namespace13/project13!2))\n- + [Title 1](namespace13/project13@3c6b80ff7034fa0d585314e1571cc780596ce3c8) + ([merge request](namespace13/project13!1))\n" } ``` ## Related topics - User documentation for [changelogs](../user/project/changelogs.md) -- Developer documentation for [changelog entries](../development/changelog.md) in GitLab. +- Developer documentation for [changelog entries](../development/changelog.md) in GitLab diff --git a/doc/api/resource_groups.md b/doc/api/resource_groups.md index 552c2a3ecfb..23c982c9296 100644 --- a/doc/api/resource_groups.md +++ b/doc/api/resource_groups.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/rest/index.md b/doc/api/rest/index.md index 91a70c3f2a9..2298eeb20a5 100644 --- a/doc/api/rest/index.md +++ b/doc/api/rest/index.md @@ -12,7 +12,7 @@ developers who are more comfortable with traditional API architecture. ## Compatibility guidelines -The HTTP API is versioned with a single number, which is currently `4`. This number +The HTTP API is versioned with a single number, which is `4`. This number symbolizes the major version number, as described by [SemVer](https://semver.org/). Because of this, backward-incompatible changes require this version number to change. @@ -83,9 +83,9 @@ authentication isn't provided. When authentication is not required, the document for each endpoint specifies this. For example, the [`/projects/:id` endpoint](../projects.md#get-single-project) does not require authentication. -There are several ways you can authenticate with the GitLab API: +You can authenticate with the GitLab API in several ways: -- [OAuth2 tokens](#oauth2-tokens) +- [OAuth 2.0 tokens](#oauth-20-tokens) - [Personal access tokens](../../user/profile/personal_access_tokens.md) - [Project access tokens](../../user/project/settings/project_access_tokens.md) - [Group access tokens](../../user/group/settings/group_access_tokens.md) @@ -116,30 +116,30 @@ NOTE: Deploy tokens can't be used with the GitLab public API. For details, see [Deploy Tokens](../../user/project/deploy_tokens/index.md). -### OAuth2 tokens +### OAuth 2.0 tokens -You can use an [OAuth2 token](../oauth2.md) to authenticate with the API by passing +You can use an [OAuth 2.0 token](../oauth2.md) to authenticate with the API by passing it in either the `access_token` parameter or the `Authorization` header. -Example of using the OAuth2 token in a parameter: +Example of using the OAuth 2.0 token in a parameter: ```shell curl "https://gitlab.example.com/api/v4/projects?access_token=OAUTH-TOKEN" ``` -Example of using the OAuth2 token in a header: +Example of using the OAuth 2.0 token in a header: ```shell curl --header "Authorization: Bearer OAUTH-TOKEN" "https://gitlab.example.com/api/v4/projects" ``` -Read more about [GitLab as an OAuth2 provider](../oauth2.md). +Read more about [GitLab as an OAuth 2.0 provider](../oauth2.md). NOTE: -We recommend OAuth access tokens have an expiration. You can use the `refresh_token` parameter +You should give OAuth access tokens an expiration. You can use the `refresh_token` parameter to refresh tokens. Integrations may need to be updated to use refresh tokens prior to expiration, which is based on the [`expires_in`](https://datatracker.ietf.org/doc/html/rfc6749#appendix-A.14) -property in the token endpoint response. See [OAuth2 token](../oauth2.md) documentation +property in the token endpoint response. See [OAuth 2.0 token](../oauth2.md) documentation for examples requesting a new access token using a refresh token. A default refresh setting of two hours is tracked in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336598). @@ -299,52 +299,52 @@ insight into what went wrong. The following table gives an overview of how the API functions generally behave. -| Request type | Description | -|---------------|-------------| -| `GET` | Access one or more resources and return the result as JSON. | -| `POST` | Return `201 Created` if the resource is successfully created and return the newly created resource as JSON. | -| `GET` / `PUT` | Return `200 OK` if the resource is accessed or modified successfully. The (modified) result is returned as JSON. | +| Request type | Description | +|:--------------|:--------------------------------------------------------------------------------------------------------------------------------| +| `GET` | Access one or more resources and return the result as JSON. | +| `POST` | Return `201 Created` if the resource is successfully created and return the newly created resource as JSON. | +| `GET` / `PUT` | Return `200 OK` if the resource is accessed or modified successfully. The (modified) result is returned as JSON. | | `DELETE` | Returns `204 No Content` if the resource was deleted successfully or `202 Accepted` if the resource is scheduled to be deleted. | The following table shows the possible return codes for API requests. -| Return values | Description | -|--------------------------|-------------| -| `200 OK` | The `GET`, `PUT` or `DELETE` request was successful, and the resource itself is returned as JSON. | -| `201 Created` | The `POST` request was successful, and the resource is returned as JSON. | -| `202 Accepted` | The `GET`, `PUT` or `DELETE` request was successful, and the resource is scheduled for processing. | -| `204 No Content` | The server has successfully fulfilled the request, and there is no additional content to send in the response payload body. | -| `304 Not Modified` | The resource hasn't been modified since the last request. | -| `400 Bad Request` | A required attribute of the API request is missing. For example, the title of an issue is not given. | -| `401 Unauthorized` | The user isn't authenticated. A valid [user token](#authentication) is necessary. | -| `403 Forbidden` | The request isn't allowed. For example, the user isn't allowed to delete a project. | -| `404 Not Found` | A resource couldn't be accessed. For example, an ID for a resource couldn't be found, or the user isn't authorized to access the resource. | -| `405 Method Not Allowed` | The request isn't supported. | -| `409 Conflict` | A conflicting resource already exists. For example, creating a project with a name that already exists. | -| `412 Precondition Failed`| The request was denied. This can happen if the `If-Unmodified-Since` header is provided when trying to delete a resource, which was modified in between. | -| `422 Unprocessable` | The entity couldn't be processed. | -| `429 Too Many Requests` | The user exceeded the [application rate limits](../../administration/instance_limits.md#rate-limits). | -| `500 Server Error` | While handling the request, something went wrong on the server. | +| Return values | Description | +|:--------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------| +| `200 OK` | The `GET`, `PUT` or `DELETE` request was successful, and the resource itself is returned as JSON. | +| `201 Created` | The `POST` request was successful, and the resource is returned as JSON. | +| `202 Accepted` | The `GET`, `PUT` or `DELETE` request was successful, and the resource is scheduled for processing. | +| `204 No Content` | The server has successfully fulfilled the request, and there is no additional content to send in the response payload body. | +| `304 Not Modified` | The resource hasn't been modified since the last request. | +| `400 Bad Request` | A required attribute of the API request is missing. For example, the title of an issue is not given. | +| `401 Unauthorized` | The user isn't authenticated. A valid [user token](#authentication) is necessary. | +| `403 Forbidden` | The request isn't allowed. For example, the user isn't allowed to delete a project. | +| `404 Not Found` | A resource couldn't be accessed. For example, an ID for a resource couldn't be found, or the user isn't authorized to access the resource. | +| `405 Method Not Allowed` | The request isn't supported. | +| `409 Conflict` | A conflicting resource already exists. For example, creating a project with a name that already exists. | +| `412 Precondition Failed` | The request was denied. This can happen if the `If-Unmodified-Since` header is provided when trying to delete a resource, which was modified in between. | +| `422 Unprocessable` | The entity couldn't be processed. | +| `429 Too Many Requests` | The user exceeded the [application rate limits](../../administration/instance_limits.md#rate-limits). | +| `500 Server Error` | While handling the request, something went wrong on the server. | ## Pagination GitLab supports the following pagination methods: -- Offset-based pagination. This is the default method and is available on all endpoints. +- Offset-based pagination. The default method and available on all endpoints. - Keyset-based pagination. Added to selected endpoints but being [progressively rolled out](https://gitlab.com/groups/gitlab-org/-/epics/2039). -For large collections, for performance reasons we recommend keyset pagination -(when available) instead of offset pagination. +For large collections, you should use keyset pagination +(when available) instead of offset pagination, for performance reasons. ### Offset-based pagination Sometimes, the returned result spans many pages. When listing resources, you can pass the following parameters: -| Parameter | Description | -|------------|-------------| -| `page` | Page number (default: `1`). | +| Parameter | Description | +|:-----------|:--------------------------------------------------------------| +| `page` | Page number (default: `1`). | | `per_page` | Number of items to list per page (default: `20`, max: `100`). | In the following example, we list 50 [namespaces](../namespaces.md) per page: @@ -397,14 +397,14 @@ x-total-pages: 3 GitLab also returns the following additional pagination headers: -| Header | Description | -|-----------------|-------------| -| `x-next-page` | The index of the next page. | +| Header | Description | +|:----------------|:-----------------------------------------------| +| `x-next-page` | The index of the next page. | | `x-page` | The index of the current page (starting at 1). | -| `x-per-page` | The number of items per page. | -| `x-prev-page` | The index of the previous page. | -| `x-total` | The total number of items. | -| `x-total-pages` | The total number of pages. | +| `x-per-page` | The number of items per page. | +| `x-prev-page` | The index of the previous page. | +| `x-total` | The total number of items. | +| `x-total-pages` | The total number of pages. | For GitLab.com users, [some pagination headers may not be returned](../../user/gitlab_com/index.md#pagination-response-headers). @@ -476,20 +476,23 @@ When the end of the collection is reached and there are no additional records to retrieve, the `Link` header is absent and the resulting array is empty. -We recommend using only the given link to retrieve the next page instead of +You should use only the given link to retrieve the next page instead of building your own URL. Apart from the headers shown, we don't expose additional pagination headers. +#### Supported resources + Keyset-based pagination is supported only for selected resources and ordering options: -| Resource | Options | Availability | -|:----------------------------------------------------------------|:---------------------------------|:--------------------------------------------------------------------------------------------------------------| -| [Projects](../projects.md) | `order_by=id` only | Authenticated and unauthenticated users | -| [Groups](../groups.md) | `order_by=name`, `sort=asc` only | Unauthenticated users only | -| [Group audit events](../audit_events.md#group-audit-events) | `order_by=id`, `sort=desc` only | Authenticated users only ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333968) in GitLab 15.2) | -| [Project audit events](../audit_events.md#project-audit-events) | `order_by=id`, `sort=desc` only | Authenticated users only ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367528) in GitLab 15.10) | -| [Jobs](../jobs.md) | `order_by=id`, `sort=desc` only | Authenticated users only ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362172) in GitLab 15.9) | +| Resource | Options | Availability | +|:------------------------------------------------------------------|:---------------------------------|:--------------------------------------------------------------------------------------------------------------| +| [Group audit events](../audit_events.md#group-audit-events) | `order_by=id`, `sort=desc` only | Authenticated users only ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333968) in GitLab 15.2) | +| [Groups](../groups.md) | `order_by=name`, `sort=asc` only | Unauthenticated users only | +| [Instance audit events](../audit_events.md#instance-audit-events) | `order_by=id`, `sort=desc` only | Authenticated users only ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367528) in GitLab 15.11) | +| [Jobs](../jobs.md) | `order_by=id`, `sort=desc` only | Authenticated users only ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362172) in GitLab 15.9) | +| [Project audit events](../audit_events.md#project-audit-events) | `order_by=id`, `sort=desc` only | Authenticated users only ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367528) in GitLab 15.10) | +| [Projects](../projects.md) | `order_by=id` only | Authenticated and unauthenticated users | ### Pagination response headers @@ -643,6 +646,14 @@ For example, suppose a project with `id: 42` has an issue with `id: 46` and Not all resources with the `iid` field are fetched by `iid`. For guidance regarding which field to use, see the documentation for the specific resource. +## `null` vs `false` + +In API responses, some boolean fields can have `null` values. +A `null` boolean has no default value and is neither `true` nor `false`. +GitLab treats `null` values in boolean fields the same as `false`. + +In boolean arguments, you should only set `true` or `false` values (not `null`). + ## Data validation and error reporting When working with the API you may encounter validation errors, in which case diff --git a/doc/api/runners.md b/doc/api/runners.md index be69b762555..8d0be1c3aba 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -273,10 +273,10 @@ PUT /runners/:id | `id` | integer | yes | The ID of a runner | | `description` | string | no | The description of the runner | | `active` | boolean | no | Deprecated: Use `paused` instead. Flag indicating whether the runner is allowed to receive jobs | -| `paused` | boolean | no | Specifies whether the runner should ignore new jobs | +| `paused` | boolean | no | Specifies if the runner should ignore new jobs | | `tag_list` | array | no | The list of tags for the runner | -| `run_untagged` | boolean | no | Specifies whether the runner can execute untagged jobs | -| `locked` | boolean | no | Specifies whether the runner is locked | +| `run_untagged` | boolean | no | Specifies if the runner can execute untagged jobs | +| `locked` | boolean | no | Specifies if the runner is locked | | `access_level` | string | no | The access level of the runner; `not_protected` or `ref_protected` | | `maximum_timeout` | integer | no | Maximum timeout that limits the amount of time (in seconds) that runners can run jobs | @@ -656,20 +656,20 @@ Register a new runner for the instance. POST /runners ``` -| Attribute | Type | Required | Description | -|--------------------|--------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `token` | string | yes | [Registration token](#registration-and-authentication-tokens) | -| `description` | string | no | Runner's description | +| Attribute | Type | Required | Description | +|--------------------|--------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `token` | string | yes | [Registration token](#registration-and-authentication-tokens) | +| `description` | string | no | Description of the runner | | `info` | hash | no | Runner's metadata. You can include `name`, `version`, `revision`, `platform`, and `architecture`, but only `version`, `platform`, and `architecture` are displayed in the Admin Area of the UI | -| `active` | boolean | no | Deprecated: Use `paused` instead. Specifies whether the runner is allowed to receive new jobs | -| `paused` | boolean | no | Specifies whether the runner should ignore new jobs | -| `locked` | boolean | no | Specifies whether the runner should be locked for the current project | -| `run_untagged` | boolean | no | Specifies whether the runner should handle untagged jobs | -| `tag_list` | string array | no | A list of runner tags | -| `access_level` | string | no | The access level of the runner; `not_protected` or `ref_protected` | -| `maximum_timeout` | integer | no | Maximum timeout that limits the amount of time (in seconds) that runners can run jobs | -| `maintainer_note` | string | no | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/350730), see `maintenance_note` | -| `maintenance_note` | string | no | Free-form maintenance notes for the runner (1024 characters) | +| `active` | boolean | no | Deprecated: Use `paused` instead. Specifies if the runner is allowed to receive new jobs | +| `paused` | boolean | no | Specifies if the runner should ignore new jobs | +| `locked` | boolean | no | Specifies if the runner should be locked for the current project | +| `run_untagged` | boolean | no | Specifies if the runner should handle untagged jobs | +| `tag_list` | string array | no | A list of runner tags | +| `access_level` | string | no | The access level of the runner; `not_protected` or `ref_protected` | +| `maximum_timeout` | integer | no | Maximum timeout that limits the amount of time (in seconds) that runners can run jobs | +| `maintainer_note` | string | no | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/350730), see `maintenance_note` | +| `maintenance_note` | string | no | Free-form maintenance notes for the runner (1024 characters) | ```shell curl --request POST "https://gitlab.example.com/api/v4/runners" \ @@ -774,7 +774,7 @@ Example response: } ``` -## Reset instance's runner registration token +## Reset instance's runner registration token (deprecated) > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104691) in GitLab 15.7. @@ -793,7 +793,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/runners/reset_registration_token" ``` -## Reset project's runner registration token +## Reset project's runner registration token (deprecated) > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104691) in GitLab 15.7. @@ -812,7 +812,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/projects/9/runners/reset_registration_token" ``` -## Reset group's runner registration token +## Reset group's runner registration token (deprecated) > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104691) in GitLab 15.7. diff --git a/doc/api/settings.md b/doc/api/settings.md index c3c48fba339..36d2b74cb06 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -107,7 +107,8 @@ Example response: "external_pipeline_validation_service_token": null, "external_pipeline_validation_service_url": null, "jira_connect_application_key": null, - "jira_connect_proxy_url": null + "jira_connect_proxy_url": null, + "silent_mode_enabled": false } ``` @@ -115,11 +116,14 @@ Users on [GitLab Premium or Ultimate](https://about.gitlab.com/pricing/) may als the `group_owners_can_manage_default_branch_protection`, `file_template_project_id`, `delayed_project_deletion`, `delayed_group_deletion`, `deletion_adjourned_period`, `disable_personal_access_tokens`, or the `geo_node_allowed_ips` parameters: +From [GitLab 15.11](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113332), with the `always_perform_delayed_deletion` feature flag enabled, +the `delayed_project_deletion` and `delayed_group_deletion` attributes will not be exposed. These attributes will be removed in GitLab 16.0. + ```json { - "id" : 1, - "signup_enabled" : true, - "group_owners_can_manage_default_branch_protection" : true, + "id": 1, + "signup_enabled": true, + "group_owners_can_manage_default_branch_protection": true, "file_template_project_id": 1, "geo_node_allowed_ips": "0.0.0.0/0, ::/0", "delayed_project_deletion": false, @@ -228,7 +232,8 @@ Example response: "jira_connect_application_key": "123", "jira_connect_proxy_url": "http://gitlab.example.com", "user_defaults_to_private_profile": true, - "projects_api_rate_limit_unauthenticated": 400 + "projects_api_rate_limit_unauthenticated": 400, + "silent_mode_enabled": false } ``` @@ -244,6 +249,9 @@ these parameters: - `deletion_adjourned_period` - `disable_personal_access_tokens` +From [GitLab 15.11](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113332), with the `always_perform_delayed_deletion` feature flag enabled, +the `delayed_project_deletion` and `delayed_group_deletion` attributes will not be exposed. These attributes will be removed in GitLab 16.0. + Example responses: **(PREMIUM SELF)** ```json @@ -307,8 +315,8 @@ listed in the descriptions of the relevant settings. | `default_projects_limit` | integer | no | Project limit per user. Default is `100000`. | | `default_snippet_visibility` | string | no | What visibility level new snippets receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_syntax_highlighting_theme` | integer | no | Default syntax highlighting theme for new users and users who are not signed in. See [IDs of available themes](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/themes.rb#L16). -| `delayed_project_deletion` **(PREMIUM SELF)** | boolean | no | Enable delayed project deletion by default in new groups. Default is `false`. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), can only be enabled when `delayed_group_deletion` is true. | -| `delayed_group_deletion` **(PREMIUM SELF)** | boolean | no | Enable delayed group deletion. Default is `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352959) in GitLab 15.0. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), disables and locks the group-level setting for delayed protect deletion when set to `false`. | +| `delayed_project_deletion` **(PREMIUM SELF)** | boolean | no | Enable delayed project deletion by default in new groups. Default is `false`. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), can only be enabled when `delayed_group_deletion` is true. From [GitLab 15.11](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113332), with the `always_perform_delayed_deletion` feature flag enabled, this attribute has been removed. This attribute will be completely removed in GitLab 16.0. | +| `delayed_group_deletion` **(PREMIUM SELF)** | boolean | no | Enable delayed group deletion. Default is `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352959) in GitLab 15.0. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), disables and locks the group-level setting for delayed protect deletion when set to `false`. From [GitLab 15.11](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113332), with the `always_perform_delayed_deletion` feature flag enabled, this attribute has been removed. This attribute will be completely removed in GitLab 16.0. | | `deletion_adjourned_period` **(PREMIUM SELF)** | integer | no | The number of days to wait before deleting a project or group that is marked for deletion. Value must be between `1` and `90`. Defaults to `7`. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), a hook on `deletion_adjourned_period` sets the period to `1` on every update, and sets both `delayed_project_deletion` and `delayed_group_deletion` to `false` if the period is `0`. | | `diff_max_patch_bytes` | integer | no | Maximum [diff patch size](../user/admin_area/diff_limits.md), in bytes. | | `diff_max_files` | integer | no | Maximum [files in a diff](../user/admin_area/diff_limits.md). | @@ -469,6 +477,7 @@ listed in the descriptions of the relevant settings. | `sign_in_text` | string | no | Text on the login page. | | `signin_enabled` | string | no | (Deprecated: Use `password_authentication_enabled_for_web` instead) Flag indicating if password authentication is enabled for the web interface. | | `signup_enabled` | boolean | no | Enable registration. Default is `true`. | +| `silent_mode_enabled` | boolean | no | Enable [Silent mode](../administration/silent_mode/index.md). Default is `false`. | | `slack_app_enabled` **(PREMIUM)** | boolean | no | (**If enabled, requires:** `slack_app_id`, `slack_app_secret` and `slack_app_secret`) Enable Slack app. | | `slack_app_id` **(PREMIUM)** | string | required by: `slack_app_enabled` | The app ID of the Slack-app. | | `slack_app_secret` **(PREMIUM)** | string | required by: `slack_app_enabled` | The app secret of the Slack-app. | diff --git a/doc/api/status_checks.md b/doc/api/status_checks.md index 7299e529bda..7bff242dd3f 100644 --- a/doc/api/status_checks.md +++ b/doc/api/status_checks.md @@ -336,4 +336,4 @@ PUT /projects/:id/external_status_checks/:check_id ## Related topics -- [External status checks](../user/project/merge_requests/status_checks.md). +- [External status checks](../user/project/merge_requests/status_checks.md) diff --git a/doc/api/usage_data.md b/doc/api/usage_data.md index 33126521901..bf8924c1578 100644 --- a/doc/api/usage_data.md +++ b/doc/api/usage_data.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference, api --- @@ -35,7 +35,6 @@ Example response: product_section: enablement product_stage: enablement product_group: global_search - product_category: global_search value_type: number status: active time_frame: 28d diff --git a/doc/api/users.md b/doc/api/users.md index f877aa29b9c..c17dfd12c8b 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -1301,7 +1301,25 @@ Parameters: | `key` | string | yes | New GPG key | ```shell -curl --data "key=-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFV..." \ + +export KEY="$( gpg --armor --export <your_gpg_key_id>)" + +curl --data-urlencode "key=-----BEGIN PGP PUBLIC KEY BLOCK----- +> xsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj +> t1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O +> CfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa +> qKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO +> VaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57 +> vilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp +> IDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV +> CAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/ +> oO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5 +> crfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4 +> bjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn +> iE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp +> o4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI= +> =XQoy +> -----END PGP PUBLIC KEY BLOCK-----" \ --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/user/gpg_keys" ``` @@ -1311,7 +1329,7 @@ Example response: [ { "id": 1, - "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\r\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\r\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\r\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\r\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\r\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\r\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\r\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\r\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\r\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\r\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\r\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\r\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\r\n=XQoy\r\n-----END PGP PUBLIC KEY BLOCK-----", + "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\n=XQoy\n-----END PGP PUBLIC KEY BLOCK-----", "created_at": "2017-09-05T09:17:46.264Z" } ] @@ -1413,7 +1431,22 @@ Parameters: | `key_id` | integer | yes | ID of the GPG key | ```shell -curl --data "key=-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFV..." \ +curl --data-urlencode "key=-----BEGIN PGP PUBLIC KEY BLOCK----- +> xsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj +> t1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O +> CfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa +> qKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO +> VaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57 +> vilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp +> IDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV +> CAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/ +> oO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5 +> crfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4 +> bjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn +> iE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp +> o4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI= +> =XQoy +> -----END PGP PUBLIC KEY BLOCK-----" \ --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/users/2/gpg_keys" ``` @@ -1423,7 +1456,7 @@ Example response: [ { "id": 1, - "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\r\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\r\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\r\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\r\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\r\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\r\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\r\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\r\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\r\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\r\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\r\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\r\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\r\n=XQoy\r\n-----END PGP PUBLIC KEY BLOCK-----", + "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\n=XQoy\n-----END PGP PUBLIC KEY BLOCK-----", "created_at": "2017-09-05T09:17:46.264Z" } ] @@ -2163,3 +2196,43 @@ Returns: - `400 Bad request` if two factor authentication is not enabled for the specified user. - `403 Forbidden` if not authenticated as an administrator. - `404 User Not Found` if user cannot be found. + +## Create a CI runner **(FREE SELF)** + +It creates a new runner, linked to the current user. + +Requires administrator access or ownership of the target namespace or project. Token values are returned once. Make sure you save it because you can't access +it again. + +```plaintext +POST /user/runners +``` + +| Attribute | Type | Required | Description | +|--------------------|--------------|----------|---------------------------------------------------------------------------------------------------| +| `runner_type` | string | yes | Specifies the scope of the runner; `instance_type`, `group_type`, or `project_type`. | +| `group_id` | integer | no | The ID of the group that the runner is created in. Required if `runner_type` is `group_type`. | +| `project_id` | integer | no | The ID of the project that the runner is created in. Required if `runner_type` is `project_type`. | +| `description` | string | no | Description of the runner. | +| `paused` | boolean | no | Specifies if the runner should ignore new jobs. | +| `locked` | boolean | no | Specifies if the runner should be locked for the current project. | +| `run_untagged` | boolean | no | Specifies if the runner should handle untagged jobs. | +| `tag_list` | string array | no | A list of runner tags. | +| `access_level` | string | no | The access level of the runner; `not_protected` or `ref_protected`. | +| `maximum_timeout` | integer | no | Maximum timeout that limits the amount of time (in seconds) that runners can run jobs. | +| `maintenance_note` | string | no | Free-form maintenance notes for the runner (1024 characters). | + +```shell +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --data "runner_type=instance_type" \ + "https://gitlab.example.com/api/v4/user/runners" +``` + +Example response: + +```json +{ + "id": 9171, + "token": "glrt-kyahzxLaj4Dc1jQf4xjX", + "token_expires_at": null +} +``` diff --git a/doc/api/version.md b/doc/api/version.md index 0ceab92b9d6..d7a13d78c14 100644 --- a/doc/api/version.md +++ b/doc/api/version.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/visual_review_discussions.md b/doc/api/visual_review_discussions.md index 561dc1a26a5..496c732b337 100644 --- a/doc/api/visual_review_discussions.md +++ b/doc/api/visual_review_discussions.md @@ -4,13 +4,18 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Visual Review discussions API **(PREMIUM)** +<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> +# Visual Review discussions API (deprecated) **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18710) in GitLab 12.5. > - [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387751) in GitLab 15.8 +and is planned for removal in 17.0. This change is a breaking change. + Visual Review discussions are notes on merge requests sent as -feedback from [Visual Reviews](../ci/review_apps/index.md#visual-reviews). +feedback from [Visual Reviews](../ci/review_apps/index.md#visual-reviews-deprecated). ## Create new merge request thread @@ -45,3 +50,4 @@ Parameters: ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/11/visual_review_discussions?body=comment" ``` +<!--- end_remove --> diff --git a/doc/api/vulnerability_exports.md b/doc/api/vulnerability_exports.md index e1473ba68b6..28053eb20b6 100644 --- a/doc/api/vulnerability_exports.md +++ b/doc/api/vulnerability_exports.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/197494) in GitLab 12.10. [Updated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30397) in GitLab 13.0. WARNING: -This API is in an [Alpha](../policy/alpha-beta-support.md#alpha-features) stage and considered unstable. +This API is in an [Experiment](../policy/alpha-beta-support.md#experiment) and considered unstable. The response payload may be subject to change or breakage across GitLab releases. diff --git a/doc/api/wikis.md b/doc/api/wikis.md index 046901af56b..17317b7c594 100644 --- a/doc/api/wikis.md +++ b/doc/api/wikis.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/architecture/blueprints/_template.md b/doc/architecture/blueprints/_template.md index 64b79f996e0..e22cc2e6857 100644 --- a/doc/architecture/blueprints/_template.md +++ b/doc/architecture/blueprints/_template.md @@ -55,6 +55,9 @@ Blueprint statuses you can use: --> +<!-- Blueprints often contain forward-looking statements --> +<!-- vale gitlab.FutureTense = NO --> + # {+ Title of Blueprint +} <!-- diff --git a/doc/architecture/blueprints/cells/cells-feature-admin-area.md b/doc/architecture/blueprints/cells/cells-feature-admin-area.md index abcf4a5e263..31d5388d40b 100644 --- a/doc/architecture/blueprints/cells/cells-feature-admin-area.md +++ b/doc/architecture/blueprints/cells/cells-feature-admin-area.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Admin Area' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-agent-for-kubernetes.md b/doc/architecture/blueprints/cells/cells-feature-agent-for-kubernetes.md index 5f0c8c2823d..37347cf836d 100644 --- a/doc/architecture/blueprints/cells/cells-feature-agent-for-kubernetes.md +++ b/doc/architecture/blueprints/cells/cells-feature-agent-for-kubernetes.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Agent for Kubernetes' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-backups.md b/doc/architecture/blueprints/cells/cells-feature-backups.md index 80cac8764e6..d596bdd2078 100644 --- a/doc/architecture/blueprints/cells/cells-feature-backups.md +++ b/doc/architecture/blueprints/cells/cells-feature-backups.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Backups' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-ci-runners.md b/doc/architecture/blueprints/cells/cells-feature-ci-runners.md index 619c78d19ec..e352be17dd3 100644 --- a/doc/architecture/blueprints/cells/cells-feature-ci-runners.md +++ b/doc/architecture/blueprints/cells/cells-feature-ci-runners.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: CI Runners' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-container-registry.md b/doc/architecture/blueprints/cells/cells-feature-container-registry.md index b266de56205..a5761808941 100644 --- a/doc/architecture/blueprints/cells/cells-feature-container-registry.md +++ b/doc/architecture/blueprints/cells/cells-feature-container-registry.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Container Registry' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-contributions-forks.md b/doc/architecture/blueprints/cells/cells-feature-contributions-forks.md index cc6feb1c08a..3e498c24144 100644 --- a/doc/architecture/blueprints/cells/cells-feature-contributions-forks.md +++ b/doc/architecture/blueprints/cells/cells-feature-contributions-forks.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Contributions: Forks' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-dashboard.md b/doc/architecture/blueprints/cells/cells-feature-dashboard.md index 37ed13ad9f9..135f69c6ed3 100644 --- a/doc/architecture/blueprints/cells/cells-feature-dashboard.md +++ b/doc/architecture/blueprints/cells/cells-feature-dashboard.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Dashboard' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-data-migration.md b/doc/architecture/blueprints/cells/cells-feature-data-migration.md index e24cd0323f4..ef0865b4081 100644 --- a/doc/architecture/blueprints/cells/cells-feature-data-migration.md +++ b/doc/architecture/blueprints/cells/cells-feature-data-migration.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Data migration' --- +<!-- vale gitlab.FutureTense = NO --> + DISCLAIMER: This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for diff --git a/doc/architecture/blueprints/cells/cells-feature-database-sequences.md b/doc/architecture/blueprints/cells/cells-feature-database-sequences.md index 7966ac4bfdf..d94dc3be864 100644 --- a/doc/architecture/blueprints/cells/cells-feature-database-sequences.md +++ b/doc/architecture/blueprints/cells/cells-feature-database-sequences.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Database Sequences' --- +<!-- vale gitlab.FutureTense = NO --> + DISCLAIMER: This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for diff --git a/doc/architecture/blueprints/cells/cells-feature-git-access.md b/doc/architecture/blueprints/cells/cells-feature-git-access.md index 3582c99c7dd..70b3f136904 100644 --- a/doc/architecture/blueprints/cells/cells-feature-git-access.md +++ b/doc/architecture/blueprints/cells/cells-feature-git-access.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Git Access' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-gitlab-pages.md b/doc/architecture/blueprints/cells/cells-feature-gitlab-pages.md index 9d501918a21..7e4ab785095 100644 --- a/doc/architecture/blueprints/cells/cells-feature-gitlab-pages.md +++ b/doc/architecture/blueprints/cells/cells-feature-gitlab-pages.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: GitLab Pages' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-global-search.md b/doc/architecture/blueprints/cells/cells-feature-global-search.md index fdd56d7258d..c1e2b93bc2d 100644 --- a/doc/architecture/blueprints/cells/cells-feature-global-search.md +++ b/doc/architecture/blueprints/cells/cells-feature-global-search.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Global search' --- +<!-- vale gitlab.FutureTense = NO --> + DISCLAIMER: This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for diff --git a/doc/architecture/blueprints/cells/cells-feature-graphql.md b/doc/architecture/blueprints/cells/cells-feature-graphql.md index 8430d23c7f4..d936a1b81ba 100644 --- a/doc/architecture/blueprints/cells/cells-feature-graphql.md +++ b/doc/architecture/blueprints/cells/cells-feature-graphql.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: GraphQL' --- +<!-- vale gitlab.FutureTense = NO --> + DISCLAIMER: This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for diff --git a/doc/architecture/blueprints/cells/cells-feature-organizations.md b/doc/architecture/blueprints/cells/cells-feature-organizations.md index 9c2e33719ab..03178d9e6ce 100644 --- a/doc/architecture/blueprints/cells/cells-feature-organizations.md +++ b/doc/architecture/blueprints/cells/cells-feature-organizations.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Organizations' --- +<!-- vale gitlab.FutureTense = NO --> + DISCLAIMER: This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for diff --git a/doc/architecture/blueprints/cells/cells-feature-personal-namespaces.md b/doc/architecture/blueprints/cells/cells-feature-personal-namespaces.md index 467683ffadd..e8f5c250a8e 100644 --- a/doc/architecture/blueprints/cells/cells-feature-personal-namespaces.md +++ b/doc/architecture/blueprints/cells/cells-feature-personal-namespaces.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Personal Namespaces' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-router-endpoints-classification.md b/doc/architecture/blueprints/cells/cells-feature-router-endpoints-classification.md index c961857e4fa..7c2974ca258 100644 --- a/doc/architecture/blueprints/cells/cells-feature-router-endpoints-classification.md +++ b/doc/architecture/blueprints/cells/cells-feature-router-endpoints-classification.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Router Endpoints Classification' --- +<!-- vale gitlab.FutureTense = NO --> + DISCLAIMER: This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for diff --git a/doc/architecture/blueprints/cells/cells-feature-schema-changes.md b/doc/architecture/blueprints/cells/cells-feature-schema-changes.md index c43fc6f302e..d712b24a8a0 100644 --- a/doc/architecture/blueprints/cells/cells-feature-schema-changes.md +++ b/doc/architecture/blueprints/cells/cells-feature-schema-changes.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Schema changes' --- +<!-- vale gitlab.FutureTense = NO --> + DISCLAIMER: This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for diff --git a/doc/architecture/blueprints/cells/cells-feature-secrets.md b/doc/architecture/blueprints/cells/cells-feature-secrets.md index 44b65c27683..20260c89ccd 100644 --- a/doc/architecture/blueprints/cells/cells-feature-secrets.md +++ b/doc/architecture/blueprints/cells/cells-feature-secrets.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Secrets' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-snippets.md b/doc/architecture/blueprints/cells/cells-feature-snippets.md index a40b44c7b74..f5e72c0e3a0 100644 --- a/doc/architecture/blueprints/cells/cells-feature-snippets.md +++ b/doc/architecture/blueprints/cells/cells-feature-snippets.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Snippets' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-template.md b/doc/architecture/blueprints/cells/cells-feature-template.md index 6951421023d..3cece3dc99e 100644 --- a/doc/architecture/blueprints/cells/cells-feature-template.md +++ b/doc/architecture/blueprints/cells/cells-feature-template.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Problem A' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/cells-feature-uploads.md b/doc/architecture/blueprints/cells/cells-feature-uploads.md index 9d3500b6c36..fdac3a9977c 100644 --- a/doc/architecture/blueprints/cells/cells-feature-uploads.md +++ b/doc/architecture/blueprints/cells/cells-feature-uploads.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells: Uploads' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/cells/images/term-cell.png b/doc/architecture/blueprints/cells/images/term-cell.png Binary files differindex 40e4a3e29c3..799b2eccd95 100644 --- a/doc/architecture/blueprints/cells/images/term-cell.png +++ b/doc/architecture/blueprints/cells/images/term-cell.png diff --git a/doc/architecture/blueprints/cells/images/term-cluster.png b/doc/architecture/blueprints/cells/images/term-cluster.png Binary files differindex d54a3569a08..03c92850b64 100644 --- a/doc/architecture/blueprints/cells/images/term-cluster.png +++ b/doc/architecture/blueprints/cells/images/term-cluster.png diff --git a/doc/architecture/blueprints/cells/images/term-organization.png b/doc/architecture/blueprints/cells/images/term-organization.png Binary files differindex 172ff7e415a..20485587b8b 100644 --- a/doc/architecture/blueprints/cells/images/term-organization.png +++ b/doc/architecture/blueprints/cells/images/term-organization.png diff --git a/doc/architecture/blueprints/cells/index.md b/doc/architecture/blueprints/cells/index.md index 54244265b30..3b88e2f731c 100644 --- a/doc/architecture/blueprints/cells/index.md +++ b/doc/architecture/blueprints/cells/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::enablement" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Cells This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. @@ -73,7 +75,6 @@ Organizations work under the following assumptions: #### Organization properties - Top-level namespaces belong to organizations -- Users can be members of different organizations - Organizations are isolated from each other by default meaning that cross-namespace features will only work for namespaces that exist within a single organization - User namespaces must not belong to an organization @@ -81,9 +82,7 @@ Discouraged synonyms: Billable entities, customers ### Top-Level namespace -A top-level namespace is the logical object container in the code that represents all groups, subgroups and projects that belong to an organization. - -A top-level namespace is the root of nested collection namespaces and projects. The namespace and its related entities form a tree-like hierarchy: Namespaces are the nodes of the tree, projects are the leaves. +Top-level namespace is the name given to the top most group of all other groups. Groups and projects are nested underneath the top-level namespace. Example: @@ -92,7 +91,9 @@ Example: - `gitlab-org` is a `top-level namespace`; the root for all groups and projects of an organization - `gitlab` is a `project`; a project of the organization. -Top-level namespaces may [be replaced by organizations](https://gitlab.com/gitlab-org/gitlab/-/issues/368237#high-level-goals). This proposal only uses the term top-level namespaces as the organization definition is ongoing. +The top-level namespace has served as the defacto Organization entity. With the creation of Organization, top-level namespaces will be [nested underneath Organizations](https://gitlab.com/gitlab-org/gitlab/-/issues/394796). + +Over time there won't be a distinction between a top level namespace and a group. All features that make Top-level namespaces different from groups will move to Organization. Discouraged synonyms: Root-level namespace @@ -105,14 +106,15 @@ Discouraged synonyms: Root-level namespace ### Users -Users are available globally and not restricted to a single Cell. Users can be members of many different organizations with varying permissions. Inside organizations, users can create multiple top-level namespaces. User activity is not limited to a single organization but their contributions (for example TODOs) are only aggregated within an organization. This avoids the need for aggregating across cells. +Users are available globally and not restricted to a single Cell. Users belong to a single organization, but can participate in many organizations through group and project membership with varying permissions. Inside organizations, users can create multiple top-level namespaces. User activity is not limited to a single organization but their contributions (for example TODOs) are only aggregated within an organization. This avoids the need for aggregating across cells. #### User properties - Users are shared globally across all Cells - Users can create multiple top-level namespaces - Users can be a member of multiple top-level namespaces -- Users can be a member of multiple organizations +- Users belong to one organization. See [!395736](https://gitlab.com/gitlab-org/gitlab/-/issues/395736) +- Users can be members of groups and projects in different organizations - Users can administer organizations - User activity is aggregated in an organization - Every user has one personal namespace @@ -164,12 +166,17 @@ self-managed instances. A number of technical issues need to be resolved to implement Cells (in no particular order). This section will be expanded. -1. How are users of an organization routed to the correct Cell? -1. How do users authenticate? +1. How are Cells provisioned? - [Design discussion](https://gitlab.com/gitlab-org/gitlab/-/issues/396641) +1. What is a Cells topology? - [Design discussion](https://gitlab.com/gitlab-org/gitlab/-/issues/396641) +1. How are users of an organization routed to the correct Cell? - +1. How do users authenticate with Cells and Organizations? - [Design discussion](https://gitlab.com/gitlab-org/gitlab/-/issues/395736) 1. How are Cells rebalanced? -1. How are Cells provisioned? 1. How can Cells implement disaster recovery capabilities? +## Decision log + +- 2022-03-15: Google Cloud as the cloud service. [Reference](https://gitlab.com/gitlab-org/gitlab/-/issues/396641#note_1314932272) + ## Cross-section impact Cells is a fundamental architecture change that impacts other sections and stages. This section summarizes and links to other groups that may be impacted and highlights potential conflicts that need to be resolved. The Tenant Scale group is not responsible for achieving the goals of other groups but we want to ensure that dependencies are resolved. @@ -233,87 +240,6 @@ We can't ship the entire Cells architecture in one go - it is too large. Instead 1. Migrate existing organizations from `cell` to `cell` 1. Add additional Cell capabilities (DR, Regions) -### Iteration 0: Introduce organizations - -In the first iteration, we introduce the concept of an organization -as a way to group top-level namespaces together. Support for organizations **does not require any Cells work** but having them will make all subsequent iterations of Cells simpler. This is mainly because we can group top-level namespaces for a single organization onto a Cell. Within an organization all interactions work as normal but we eliminate any cross-organizational interactions except in well defined cases (e.g. forking). - -This means that we don't have a large number of cross-cell interactions. - -Introducing organizations allows GitLab to move towards a multi-tenant system that is similar to Discord's with a single user account but many different "servers" - our organizations - that allow users to switch context. This model harmonizes the UX across self-managed and our SaaS Platforms and is a good fit for Cells. - -Organizations solve the following problems: - -1. We can group top-level namespaces by organization. It is very similar to the initial concept of "instance groups". For example these two top-level namespaces would belong to the organization `GitLab`: - 1. `https://gitlab.com/gitlab-org/` - 1. `https://gitlab.com/gitlab-com/` -1. We can isolate organizations from each other. Top-level namespaces of the same organization can interact within organizations but are not allowed to interact with other namespaces in other organizations. This is useful for customers because it means an organization provides clear boundaries - similar to a self-managed instance. This means we don't have to aggregate user dashboards across everything and can locally scope them to organizations. -1. We don't need to define hierarchies inside an organization. It is a container that could be filled with whatever hierarchy / entity set makes sense (organization, top-level namespaces etc.) -1. Self-managed instances would set a default organization. -1. Organizations can control user-profiles in a central way. This could be achieved by having an organization specific user-profile. Such a profile makes it possible for the organization administrators to control the user role in a company, enforce user emails, or show a graphical indicator of a user being part of the organization. An example would be a "GitLab Employee stamp" on comments. - - - -#### Why would customers opt-in to Organizations? - -By introducing organizations and Cells we can improve the reliability, performance and availability of our SaaS Platforms. - -The first iteration of organizations would also have some benefits by providing more isolation. A simple example would be that `@` mentions could be scoped to an organization. - -Future iterations would create additional value but are beyond the scope of this blueprint. - -Organizations will likely be required in the future as well. - -#### Initial user experience - -1. We create a default `GitLab.com public` organization and assign all public top-level namespaces to it. This allows existing users to access all the data on GitLab.com, exactly as it does now. -1. Any user wanting to opt-in to the benefits of organizations will need to set a single default organization. Any attempts for these users to load a global page like `/dashboard` will end up redirecting to `/-/organizations/<DEFAULT_ORGANIZATION>/dashboard`. -1. New users that opted in to organizations will only ever see data that is related to a single organization. Upon login, data is shown for the default organization. It will be clear to the user how they can switch to a different organization. Users can still navigate to the `GitLab.com` organization but they won't see TODOs from their new organizations in any such views. Instead they'd need to navigate directly to `/organizations/my-company/-/dashboard`. - -### Migrating to Organizations - -Existing customers could also opt-in to migrate their existing top-level paid namespaces to become part of an organization. In most cases this will be a 1-to-1 mapping. But in some cases it may allow a customer to move multiple top-level namespaces into one organization (for example GitLab). - -Migrating to Organizations would be optional. We could even recruit a few beta testers early on to see if this works for them. GitLab itself could dogfood organizations and we'd surface a lot of issues restricting interactions with other namespaces. - -## Iteration 1 - Introduce Cell US 0 - -### GitLab.com as Cell US0 - -GitLab.com will be treated as the first cell `Cell US 0`. It will be unique and much larger compared to newly created cells. All existing top-level namespaces and organizations will remain on `Cell US 0` in the first iteration. - -### Users are globally available - -Users are globally available and the same for all cells. This means that user data needs to be handled separately, for example via decomposition, see [!95941](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95941). - -### Cell groundwork - -In this iteration, we'll lay all the groundwork to support a second Cell for new organizations. This will be transparent to customers. - -## Iteration 2 - Introduce Cell US 1 - -### Add new organizations to Cell US 1 - -After we are ready to support a second Cell, newly created organizations are located by default on `Cell US 1`. The user experience for organizations is already well established. - -### Migrate existing organizations from Cell US 0 to Cell US 1 - -We know that we'll have to move organizations from `Cell US 0` to other cells to reduce its size and ultimately retire the existing GitLab.com architecture. - -By introducing organizations early, we should be able to draw strong "boundaries" across organizations and support migrating existing organizations to a new Cell. - -This is likely going to be GitLab itself - if we can dogfood this, we are likely going to be successful with other organizations as well. - -## Iteration 3 - Introduce Regions - -We can now leverage the Cells architecture to introduce Regions. - -## Iteration 4 - Introduce cross-organizational interactions as needed - -Based on user research, we may want to change certain features to work across organizations. Examples include: - -- Specific features allow for cross-organization interactions, for example forking, search. - ## Technical Proposals The Cells architecture do have long lasting implications to data processing, location, scalability and the GitLab architecture. diff --git a/doc/architecture/blueprints/cells/pods-feature-admin-area.md b/doc/architecture/blueprints/cells/pods-feature-admin-area.md deleted file mode 100644 index 354a55cacac..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-admin-area.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-admin-area.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-admin-area.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-agent-for-kubernetes.md b/doc/architecture/blueprints/cells/pods-feature-agent-for-kubernetes.md deleted file mode 100644 index c6a7e138508..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-agent-for-kubernetes.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-agent-for-kubernetes.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-agent-for-kubernetes.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-backups.md b/doc/architecture/blueprints/cells/pods-feature-backups.md deleted file mode 100644 index 6927142f993..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-backups.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-backups.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-backups.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-ci-runners.md b/doc/architecture/blueprints/cells/pods-feature-ci-runners.md deleted file mode 100644 index fd9c0094b2f..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-ci-runners.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-ci-runners.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-ci-runners.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-container-registry.md b/doc/architecture/blueprints/cells/pods-feature-container-registry.md deleted file mode 100644 index f6a1aa997c3..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-container-registry.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-container-registry.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-container-registry.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-contributions-forks.md b/doc/architecture/blueprints/cells/pods-feature-contributions-forks.md deleted file mode 100644 index 441a0da11b7..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-contributions-forks.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-contributions-forks.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-contributions-forks.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-dashboard.md b/doc/architecture/blueprints/cells/pods-feature-dashboard.md deleted file mode 100644 index 9404f3b647b..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-dashboard.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-dashboard.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-dashboard.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-data-migration.md b/doc/architecture/blueprints/cells/pods-feature-data-migration.md deleted file mode 100644 index 023b54ae5e4..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-data-migration.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-data-migration.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-data-migration.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-database-sequences.md b/doc/architecture/blueprints/cells/pods-feature-database-sequences.md deleted file mode 100644 index 785289081bb..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-database-sequences.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-database-sequences.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-database-sequences.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-git-access.md b/doc/architecture/blueprints/cells/pods-feature-git-access.md deleted file mode 100644 index bc44137f8d8..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-git-access.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-git-access.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-git-access.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-gitlab-pages.md b/doc/architecture/blueprints/cells/pods-feature-gitlab-pages.md deleted file mode 100644 index 8043155b1a6..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-gitlab-pages.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-gitlab-pages.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-gitlab-pages.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-global-search.md b/doc/architecture/blueprints/cells/pods-feature-global-search.md deleted file mode 100644 index d472b6a9ff6..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-global-search.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-global-search.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-global-search.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-graphql.md b/doc/architecture/blueprints/cells/pods-feature-graphql.md deleted file mode 100644 index 23d7b0bb856..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-graphql.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-graphql.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-graphql.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-organizations.md b/doc/architecture/blueprints/cells/pods-feature-organizations.md deleted file mode 100644 index 92ab37a21bf..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-organizations.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-organizations.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-organizations.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-personal-namespaces.md b/doc/architecture/blueprints/cells/pods-feature-personal-namespaces.md deleted file mode 100644 index 0f05c7d83cb..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-personal-namespaces.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-personal-namespaces.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-personal-namespaces.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-router-endpoints-classification.md b/doc/architecture/blueprints/cells/pods-feature-router-endpoints-classification.md deleted file mode 100644 index 91de121f694..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-router-endpoints-classification.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-router-endpoints-classification.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-router-endpoints-classification.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-schema-changes.md b/doc/architecture/blueprints/cells/pods-feature-schema-changes.md deleted file mode 100644 index 5f7de0eb486..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-schema-changes.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-schema-changes.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-schema-changes.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-secrets.md b/doc/architecture/blueprints/cells/pods-feature-secrets.md deleted file mode 100644 index 5c482e1c986..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-secrets.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-secrets.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-secrets.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-snippets.md b/doc/architecture/blueprints/cells/pods-feature-snippets.md deleted file mode 100644 index 867c9b49955..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-snippets.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-snippets.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-snippets.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-template.md b/doc/architecture/blueprints/cells/pods-feature-template.md deleted file mode 100644 index e1150c3426f..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-template.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-template.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-template.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/pods-feature-uploads.md b/doc/architecture/blueprints/cells/pods-feature-uploads.md deleted file mode 100644 index 7280f70ebdb..00000000000 --- a/doc/architecture/blueprints/cells/pods-feature-uploads.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'cells-feature-uploads.md' -remove_date: '2023-06-14' ---- - -This document was moved to [another location](cells-feature-uploads.md). - -<!-- This redirect file can be deleted after <2023-06-14>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/architecture/blueprints/cells/proposal-stateless-router-with-buffering-requests.md b/doc/architecture/blueprints/cells/proposal-stateless-router-with-buffering-requests.md index 12102429b03..ebf08c88ef6 100644 --- a/doc/architecture/blueprints/cells/proposal-stateless-router-with-buffering-requests.md +++ b/doc/architecture/blueprints/cells/proposal-stateless-router-with-buffering-requests.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells Stateless Router Proposal' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Pods design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Pods, and we intend to diff --git a/doc/architecture/blueprints/cells/proposal-stateless-router-with-routes-learning.md b/doc/architecture/blueprints/cells/proposal-stateless-router-with-routes-learning.md index 7f25441c75f..7602a9d3f4d 100644 --- a/doc/architecture/blueprints/cells/proposal-stateless-router-with-routes-learning.md +++ b/doc/architecture/blueprints/cells/proposal-stateless-router-with-routes-learning.md @@ -1,10 +1,11 @@ --- stage: enablement group: Tenant Scale -comments: false description: 'Cells Stateless Router Proposal' --- +<!-- vale gitlab.FutureTense = NO --> + This document is a work-in-progress and represents a very early state of the Cells design. Significant aspects are not documented, though we expect to add them in the future. This is one possible architecture for Cells, and we intend to diff --git a/doc/architecture/blueprints/ci_data_decay/index.md b/doc/architecture/blueprints/ci_data_decay/index.md index e26e7d5dbd3..2eac27def18 100644 --- a/doc/architecture/blueprints/ci_data_decay/index.md +++ b/doc/architecture/blueprints/ci_data_decay/index.md @@ -8,12 +8,14 @@ owning-stage: "~devops::verify" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # CI/CD data time decay ## Summary GitLab CI/CD is one of the most data and compute intensive components of GitLab. -Since its [initial release in November 2012](https://about.gitlab.com/blog/2012/11/13/continuous-integration-server-from-gitlab/), +Since its initial release in 2012, the CI/CD subsystem has evolved significantly. It was [integrated into GitLab in September 2015](https://about.gitlab.com/releases/2015/09/22/gitlab-8-0-released/) and has become [one of the most beloved CI/CD solutions](https://about.gitlab.com/blog/2017/09/27/gitlab-leader-continuous-integration-forrester-wave/). @@ -231,7 +233,7 @@ In progress. - 2022-02-08: Pipeline partitioning PoC [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80186) started. - 2022-02-23: Pipeline partitioning PoC [successful](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80186#note_852704724) - 2022-03-07: A way to attach an existing table as a partition [found and proven](https://gitlab.com/gitlab-org/gitlab/-/issues/353380#note_865237214). -- 2022-03-23: Pipeline partitioning design [Google Doc](https://docs.google.com/document/d/1ARdoTZDy4qLGf6Z1GIHh83-stG_ZLpqsibjKr_OXMgc) started. +- 2022-03-23: Pipeline partitioning design Google Doc (GitLab internal) started: `https://docs.google.com/document/d/1ARdoTZDy4qLGf6Z1GIHh83-stG_ZLpqsibjKr_OXMgc`. - 2022-03-29: Pipeline partitioning PoC [concluded](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80186#note_892674358). - 2022-04-15: Partitioned pipeline data associations PoC [shipped](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/84071). - 2022-04-30: Additional [benchmarking started](https://gitlab.com/gitlab-org/gitlab/-/issues/361019) to evaluate impact. diff --git a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md index 3e86d30df1d..5dea1090507 100644 --- a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md +++ b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md @@ -1,5 +1,4 @@ --- -comments: false status: ongoing creation-date: "2022-05-31" authors: [ "@grzesiek" ] @@ -9,6 +8,8 @@ owning-stage: "~devops::verify" description: 'Pipeline data partitioning design' --- +<!-- vale gitlab.FutureTense = NO --> + # Pipeline data partitioning design ## What problem are we trying to solve? diff --git a/doc/architecture/blueprints/ci_pipeline_components/dev_workflow.md b/doc/architecture/blueprints/ci_pipeline_components/dev_workflow.md index 266ea13c275..fd897781cf5 100644 --- a/doc/architecture/blueprints/ci_pipeline_components/dev_workflow.md +++ b/doc/architecture/blueprints/ci_pipeline_components/dev_workflow.md @@ -1,7 +1,6 @@ --- stage: verify group: pipeline authoring -comments: false description: 'Development workflow for a components repository' --- diff --git a/doc/architecture/blueprints/ci_pipeline_components/index.md b/doc/architecture/blueprints/ci_pipeline_components/index.md index c1094bc4290..82ff00ff4ee 100644 --- a/doc/architecture/blueprints/ci_pipeline_components/index.md +++ b/doc/architecture/blueprints/ci_pipeline_components/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::verify" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # CI/CD Catalog ## Summary @@ -98,6 +100,7 @@ identifying abstract concepts and are subject to changes as we refine the design - **Catalog resource** is the single item displayed in the catalog. A components repository is a catalog resource. - **Version** is a specific revision of catalog resource. It maps to the released tag in the project, which allows components to be pinned to a specific revision. +- **Steps** is a collection of instructions for how jobs can be executed. ## Definition of pipeline component @@ -225,7 +228,8 @@ The version of the component can be (in order of highest priority first): 1. A commit SHA - For example: `gitlab.com/gitlab-org/dast@e3262fdd0914fa823210cdb79a8c421e2cef79d8` 1. A tag - For example: `gitlab.com/gitlab-org/dast@1.0` -1. A special moving target version that points to the most recent released tag - For example: `gitlab.com/gitlab-org/dast@~latest` +1. A special moving target version that points to the most recent released tag. The target project must be +explicitly marked as a [catalog resource](#catalog-resource) - For example: `gitlab.com/gitlab-org/dast@~latest` 1. A branch name - For example: `gitlab.com/gitlab-org/dast@master` If a tag and branch exist with the same name, the tag takes precedence over the branch. @@ -703,6 +707,6 @@ Domain experts: | Area | Who |------------------------------|------------------------| | Verify / Pipeline authoring | Avielle Wolfe | -| Verify / Pipeline authoring | Laura Montemayor-Rodriguez | +| Verify / Pipeline authoring | Laura Montemayor | <!-- vale gitlab.Spelling = YES --> diff --git a/doc/architecture/blueprints/ci_scale/index.md b/doc/architecture/blueprints/ci_scale/index.md index cf7065f7c07..3a6ed4ae9b1 100644 --- a/doc/architecture/blueprints/ci_scale/index.md +++ b/doc/architecture/blueprints/ci_scale/index.md @@ -7,12 +7,14 @@ approvers: [ "@cheryl.li", "@jreporter" ] owning-stage: "~devops::verify" --- +<!-- vale gitlab.FutureTense = NO --> + # CI/CD Scaling ## Summary GitLab CI/CD is one of the most data and compute intensive components of GitLab. -Since its [initial release in November 2012](https://about.gitlab.com/blog/2012/11/13/continuous-integration-server-from-gitlab/), +Since its initial release in 2012, the CI/CD subsystem has evolved significantly. It was [integrated into GitLab in September 2015](https://about.gitlab.com/releases/2015/09/22/gitlab-8-0-released/) and has become [one of the most beloved CI/CD solutions](https://about.gitlab.com/blog/2017/09/27/gitlab-leader-continuous-integration-forrester-wave/). diff --git a/doc/architecture/blueprints/clickhouse_usage/index.md b/doc/architecture/blueprints/clickhouse_usage/index.md index 2781ea15a55..8a5530313e5 100644 --- a/doc/architecture/blueprints/clickhouse_usage/index.md +++ b/doc/architecture/blueprints/clickhouse_usage/index.md @@ -2,12 +2,14 @@ status: proposed creation-date: "2023-02-02" authors: [ "@nhxnguyen" ] -coach: "@grzesiek" +coach: "@grzesiek" approvers: [ "@dorrino", "@nhxnguyen" ] owning-stage: "~devops::data_stores" participating-stages: ["~section::ops", "~section::dev"] --- +<!-- vale gitlab.FutureTense = NO --> + # ClickHouse Usage at GitLab ## Summary @@ -50,3 +52,7 @@ Note that we are still formulating proposals and will update the blueprint accor ## Best Practices Best practices and guidelines for developing performant and scalable features using ClickHouse are located in the [ClickHouse developer documentation](../../../development/database/clickhouse/index.md). + +## Cost and maintenance analysis + +ClickHouse components cost and maintenance analysis is located in the [ClickHouse Self-Managed component costs and maintenance requirements](self_managed_costs_and_requirements/index.md). diff --git a/doc/architecture/blueprints/clickhouse_usage/self_managed_costs_and_requirements/index.md b/doc/architecture/blueprints/clickhouse_usage/self_managed_costs_and_requirements/index.md new file mode 100644 index 00000000000..d8c9c0b25d5 --- /dev/null +++ b/doc/architecture/blueprints/clickhouse_usage/self_managed_costs_and_requirements/index.md @@ -0,0 +1,65 @@ +--- +status: proposed +creation-date: "2023-04-04" +authors: [ "@niskhakova", "@dmakovey" ] +coach: "@grzesiek" +approvers: [ "@dorrino", "@nhxnguyen" ] +owning-stage: "~workinggroup::clickhouse" +participating-stages: ["~section::enablement"] +--- + +# ClickHouse Self-Managed component costs and maintenance requirements + +## Summary + +[ClickHouse](https://clickhouse.com/) requires additional cost and maintenance for self-managed customers: + +- **Resource allocation cost**: ClickHouse requires a considerable amount of resources to run optimally. + - [Minimum cost estimation](#minimum-self-managed-component-costs) shows that setting up ClickHouse can be applicable only for very large Reference Architectures: 25k and up. +- **High availability**: ClickHouse SaaS supports HA. No documented HA configuration for self-managed at the moment. +- **Geo setups**: Sync and replication complexity for GitLab Geo setups. +- **Upgrades**: An additional database to maintain and upgrade along with existing Postgres database. This also includes compatibility issues of mapping GitLab version to ClickHouse version and keeping them up-to-date. +- **Backup and restore:** Self-managed customers need to have an engineer who is familiar with backup strategies and disaster recovery process in ClickHouse or switch to ClickHouse SaaS. +- **Monitoring**: ClickHouse can use Prometheus, additional component to monitor and troubleshoot. +- **Limitations**: Azure object storage is not supported. GitLab does not have the documentation or support expertise to assist customers with deployment and operation of self-managed ClickHouse. +- **ClickHouse SaaS**: Customers using a self-managed GitLab instance with regulatory or compliance requirements, or latency concerns likely cannot use ClickHouse SaaS. + +### Minimum self-managed component costs + +Based on [ClickHouse spec requirements](https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/14384#note_1307456092) analysis +and collaborating with ClickHouse team, we identified the following minimal configurations for ClickHouse self-managed: + +1. ClickHouse High Availability (HA) + - ClickHouse - 2 machines with >=16-cores, >=64 GB RAM, SSD, 10 GB Internet. Each machine also runs Keeper. + - [Keeper](https://clickhouse.com/docs/en/guides/sre/keeper/clickhouse-keeper) - 1 machine with 2 CPU, 4 GB of RAM, SSD with high IOPS +1. ClickHouse non-HA + - ClickHouse - 1 machine with >=16-cores, >=64 GB RAM, SSD, 10 GB Internet. + +The following [cost table](https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/14384#note_1324085466) was compiled using the machine CPU and memory requirements for ClickHouse, and comparing them to the +GitLab Reference Architecture sizes and [costs](../../../../administration/reference_architectures/index.md#cost-to-run) from the GCP calculator. + +| Reference Architecture | ClickHouse type | ClickHouse cost / (GitLab cost + ClickHouse cost) | +|-------------|-----------------|-----------------------------------| +| [1k - non HA](https://cloud.google.com/products/calculator#id=a6d6a94a-c7dc-4c22-85c4-7c5747f272ed) | [non-HA](https://cloud.google.com/products/calculator#id=9af5359e-b155-451c-b090-5f0879bb591e) | 78.01% | +| [2k - non HA](https://cloud.google.com/products/calculator#id=0d3aff1f-ea3d-43f9-aa59-df49d27c35ca) | [non-HA](https://cloud.google.com/products/calculator#id=9af5359e-b155-451c-b090-5f0879bb591e) | 44.50% | +| [3k - HA](https://cloud.google.com/products/calculator/#id=15fc2bd9-5b1c-479d-bc46-d5ce096b8107) | [HA](https://cloud.google.com/products/calculator#id=9909f5af-d41a-4da2-b8cc-a0347702a823) | 37.87% | +| [5k - HA](https://cloud.google.com/products/calculator/#id=9a798136-53f2-4c35-be43-8e1e975a6663) | [HA](https://cloud.google.com/products/calculator#id=9909f5af-d41a-4da2-b8cc-a0347702a823) | 30.92% | +| [10k - HA](https://cloud.google.com/products/calculator#id=cbe61840-31a1-487f-88fa-631251c2fde5) | [HA](https://cloud.google.com/products/calculator#id=9909f5af-d41a-4da2-b8cc-a0347702a823) | 20.47% | +| [25k - HA](https://cloud.google.com/products/calculator#id=b4b8b587-508a-4433-adc8-dc506bbe924f) | [HA](https://cloud.google.com/products/calculator#id=9909f5af-d41a-4da2-b8cc-a0347702a823) | 14.30% | +| [50k - HA](https://cloud.google.com/products/calculator/#id=48b4d817-d6cd-44b8-b069-0ba9a5d123ea) | [HA](https://cloud.google.com/products/calculator#id=9909f5af-d41a-4da2-b8cc-a0347702a823) | 8.16% | + +NOTE: +The ClickHouse Self-Managed component evaluation is the minimum estimation for the costs +with a simplified architecture. + +The following components increase the cost, and were not considered in the minimum calculation: + +- Disk size - depends on data size, hard to estimate. +- Disk types - ClickHouse recommends [fast SSDs](https://clickhouse.com/docs/ru/operations/tips#storage-subsystem). +- Network usage - ClickHouse recommends using [10 GB network, if possible](https://clickhouse.com/docs/en/operations/tips#network). +- For HA we sum minimum cost across all reference architectures from 3k to 50k users, but HA specs tend to increase with user count. + +### Resources + +- [Research and understand component costs and maintenance requirements of running a ClickHouse instance with GitLab](https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/14384) +- [ClickHouse for Error Tracking on GitLab.com](https://gitlab.com/gitlab-com/gl-infra/readiness/-/blob/master/library/database/clickhouse/index.md) diff --git a/doc/architecture/blueprints/code_search_with_zoekt/index.md b/doc/architecture/blueprints/code_search_with_zoekt/index.md index ca74460b98a..db608b763b8 100644 --- a/doc/architecture/blueprints/code_search_with_zoekt/index.md +++ b/doc/architecture/blueprints/code_search_with_zoekt/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::enablement" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Use Zoekt For code search ## Summary diff --git a/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md b/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md index 12254fa7920..5b82716cb21 100644 --- a/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md +++ b/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::non_devops" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Composable GitLab Codebase NOTE: diff --git a/doc/architecture/blueprints/consolidating_groups_and_projects/index.md b/doc/architecture/blueprints/consolidating_groups_and_projects/index.md index 88034f60caf..f5bd53627cb 100644 --- a/doc/architecture/blueprints/consolidating_groups_and_projects/index.md +++ b/doc/architecture/blueprints/consolidating_groups_and_projects/index.md @@ -9,12 +9,13 @@ owning-stage: "~devops::data_stores" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Consolidating Groups and Projects -There are numerous features that exist exclusively within groups or -projects. The boundary between group and project features used to be clear. -However, there is growing demand to have group features within projects, and -project features within groups. For example, having issues in groups, and epics +Numerous features exist exclusively within groups or projects. The boundary between group and project features used to be clear. +However, there is growing demand to have group features in projects, and +project features in groups. For example, having issues in groups, and epics in projects. The [Simplify Groups & Projects Working Group](https://about.gitlab.com/company/team/structure/working-groups/simplify-groups-and-projects/) @@ -32,12 +33,12 @@ no established process in place. This results in the reimplementation of the same feature. Those implementations diverge from each other over time as they all live on their own. A few more problems with this approach: -- Features are coupled to their container. In practice it is not straight +- Features are coupled to their container. In practice, it is not straight forward to decouple a feature from its container. The degree of coupling varies across features. - Naive duplication of features will result in a more complex and fragile codebase. - Generalizing solutions across groups and projects may degrade system performance. -- The range of features span across many teams, and these changes will need to +- The range of features spans across many teams, and these changes will need to manage development interference. - The group/project hierarchy creates a natural feature hierarchy. When features exist across containers the feature hierarchy becomes ambiguous. @@ -49,33 +50,33 @@ remains consistent. ### Performance -Resources can only be queried in elaborate / complicated ways. This caused +Resources can only be queried in elaborate/complicated ways. This caused performance issues with authorization, epics, and many other places. As an example, to query the projects a user has access to, the following sources need to be considered: -- personal projects -- direct group membership -- direct project membership -- inherited group membership -- inherited project membership -- group sharing -- inherited membership via group sharing -- project sharing +- Personal projects +- Direct group membership +- Direct project membership +- Inherited group membership +- Inherited project membership +- Group sharing +- Inherited membership via group sharing +- Project sharing -Group / project membership, group / project sharing are also examples of +Group/project membership, group/project sharing are also examples of duplicated features. ## Goals -For now this blueprint strictly relates to the engineering challenges. +For now, this blueprint strictly relates to the engineering challenges. - Consolidate the group and project container architecture. - Develop a set of solutions to decouple features from their container. - Decouple engineering changes from product changes. - Develop a strategy to make architectural changes without adversely affecting other teams. -- Provide a solution for requests asking for features availability of other levels. +- Provide a solution for requests asking for features to be made available at other levels. ## Proposal @@ -103,9 +104,9 @@ New features should be implemented on `Namespace`. Similarly, when a feature need to be reimplemented on a different level, moving it to `Namespace` essentially makes it available on all levels: -- personal namespaces -- groups -- projects +- Personal namespaces +- Groups +- Projects Various traversal queries are already available on `Namespaces` to query the group hierarchy. `Projects` represent the leaf nodes in the hierarchy, but with @@ -114,14 +115,14 @@ retrieve projects as well. This also enables further simplification of some of our core features: -- routes should be generated based on the `Namespace` hierarchy, instead of - mixing project with the group hierarchy. -- there is no need to differentiate between `GroupMembers` and `ProjectMembers`. +- Routes should be generated based on the `Namespace` hierarchy, instead of + mixing the project with the group hierarchy. +- There is no need to differentiate between `GroupMembers` and `ProjectMembers`. All `Members` should be related to a `Namespace`. This can lead to simplified querying, and potentially deduplicating policies. -As more and more features will be migrated to `Namespace`, the role of `Project` -model will diminish over time to essentially a container around repository +As more and more features will be migrated to `Namespace`, the role of the `Project` +model will diminish over time to essentially a container around the repository related functionality. ## Iterations @@ -130,9 +131,103 @@ The work required to establish `Namespace` as a container for our features is tracked under [Consolidate Groups and Projects](https://gitlab.com/groups/gitlab-org/-/epics/6473) epic. +### Phase 1 (complete) + +- [Phase 1 epic](https://gitlab.com/groups/gitlab-org/-/epics/6697). +- **Goals**: + 1. Ensure every project receives a corresponding record in the `namespaces` + table with `type='Project'`. + 1. For user namespaces, the type changes from `NULL` to `User`. + +We should make sure that projects, and the project namespace, are equivalent: + +- **Create project:** Use Rails callbacks to ensure a new project namespace is + created for each project. Project namespace records should contain `created_at` and + `updated_at` attributes equal to the project's `created_at`/`updated_at` attributes. +- **Update project:** Use the `after_save` callback in Rails to ensure some + attributes are kept in sync between project and project namespaces. + Read [`project#after_save`](https://gitlab.com/gitlab-org/gitlab/blob/6d26634e864d7b748dda0e283eb2477362263bc3/app/models/project.rb#L101-L101) + for more information. +- **Delete project:** Use FKs cascade delete or Rails callbacks to ensure when a `Project` + or its `ProjectNamespace` is removed, its corresponding `ProjectNamespace` or `Project` + is also removed. +- **Transfer project to a different group:** Make sure that when a project is transferred, + its corresponding project namespace is transferred to the same group. +- **Transfer group:** Make sure when transferring a group that all of its sub-projects, + either direct or through descendant groups, have their corresponding project + namespaces transferred correctly as well. +- **Export or import project** + - **Export project** continues to export only the project, and not its project namespace, + in this phase. The project namespace does not contain any specific information + to export at this point. Eventually, we want the project namespace to be exported as well. + - **Import project** creates a new project, so the project namespace is created through + Rails `after_save` callback on the project model. +- **Export or import group:** When importing or exporting a `Group`, projects are not + included in the operation. If that feature is changed to include `Project` when its group is + imported or exported, the logic must include their corresponding project namespaces + in the import or export. + +After ensuring these points, run a database migration to create a `ProjectNamespace` +record for every `Project`. Project namespace records created during the migration +should have `created_at` and `updated_at` attributes set to the migration runtime. +The project namespaces' `created_at` and `updated_at` attributes would not match +their corresponding project's `created_at` and `updated_at` attributes. We want +the different dates to help audit any of the created project namespaces, in case we need it. +After this work completes, we must migrate data as described in +[Backfill `ProjectNamespace` for every Project](https://gitlab.com/gitlab-org/gitlab/-/issues/337100). + +### Phase 2 (complete) + +- [Phase 2 epic](https://gitlab.com/groups/gitlab-org/-/epics/6768). +- **Goal**: Link `ProjectNamespace` to other entities on the database level. + +In this phase: + +- Communicate the changes company-wide at the engineering level. We want to make + engineers aware of the upcoming changes, even though teams are not expected to + collaborate actively until phase 3. +- Raise awareness to avoid regressions and conflicting or duplicate work that + can be dealt with before phase 3. + +### Phase 3 (ongoing) + +- [Phase 3 epic](https://gitlab.com/groups/gitlab-org/-/epics/6585). + +In this phase we are migrating basic, high-priority project functionality from `Project` to `ProjectNamespace`, or directly to `Namespace`. Problems to solve as part of this phase: + +- [Unify members/members actions](https://gitlab.com/groups/gitlab-org/-/epics/8010) - on UI and API level. +- Starring: Right now only projects can be starred. We want to bring this to the group level. +- Common actions: Destroying, transferring, restoring. This can be unified on the controller level and then propagated lower. +- Archiving currently only works on the project level. This can be brought to the group level, similar to the mechanism for “pending deletion”. +- Avatar's serving and actions. + +### Phase 4 + +- [Phase 4 epic](https://gitlab.com/groups/gitlab-org/-/epics/8687) + +In this phase we are migrating additional functionality from `Project` to `ProjectNamespace`/`Namespace`: + +- Replace usages of `Project` with `ProjectNamespace` in the code. +- API changes to expose namespaces and namespace features. + - Investigate if we extend API for `groups` or we introduce a `namespaces` endpoint and slowly deprecate `groups` and `projects` endpoints. +- Break down each feature that needs to be migrated from `Project` to `ProjectNamespace` or `Namespace`. + - Investigate if we can move a feature from `Project -> Namespace` directly vs `Project -> ProjectNamespace -> Namespace`. This can be decided on a feature by feature case. +- [Migrate Project#namespace to reference ProjectNamespace](https://gitlab.com/groups/gitlab-org/-/epics/6581). +- [Routes consolidation between Project & ProjectNamespace](https://gitlab.com/gitlab-org/gitlab/-/issues/337103). +- [Policies consolidation](https://gitlab.com/groups/gitlab-org/-/epics/6689). + +### Phase 5 + +- [Phase 5 epic](https://gitlab.com/groups/gitlab-org/-/epics/6944) + +We should strive to do the code clean up as we move through the phases. However, not everything can be cleaned up while something is still being developed. For example, dropping database columns can be done as the last task when we are sure everything is working. This phase will focus on: + +- Code cleanup +- Database cleanup + ## Migrating features to Namespaces -The initial iteration will provide a framework to house features under `Namespaces`. Stage groups will eventually need to migrate their own features and functionality over to `Namespaces`. This may impact these features in unexpected ways. Therefore, to minimize UX debt and maintain product consistency, stage groups will have to consider a number of factors when migrating their features over to `Namespaces`: +The initial iteration will provide a framework to house features under `Namespaces`. Stage groups will eventually need to migrate their own features and functionality over to `Namespaces`. This may impact these features in unexpected ways. Therefore, to minimize UX debt and maintain product consistency, stage groups will have to consider several factors when migrating their features over to `Namespaces`: 1. **Conceptual model**: What are the current and future state conceptual models of these features ([see object modeling for designers](https://hpadkisson.medium.com/object-modeling-for-designers-an-introduction-7871bdcf8baf))? These should be documented in Pajamas (example: [merge requests](https://design.gitlab.com/objects/merge-request/)). 1. **Merge conflicts**: What inconsistencies are there across project, group, and administrator levels? How might these be addressed? For an example of how we rationalized this for labels, please see [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/338820). @@ -149,3 +244,4 @@ The initial iteration will provide a framework to house features under `Namespac ## Related topics - [Organization developer documentation](../../../development/organization/index.md) +- [Organization user documentation](../../../user/organization/index.md) diff --git a/doc/architecture/blueprints/container_registry_metadata_database/index.md b/doc/architecture/blueprints/container_registry_metadata_database/index.md index f3bcf1e4e59..b77aaf598e6 100644 --- a/doc/architecture/blueprints/container_registry_metadata_database/index.md +++ b/doc/architecture/blueprints/container_registry_metadata_database/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::package" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Container Registry Metadata Database ## Usage of the GitLab Container Registry diff --git a/doc/architecture/blueprints/database/clickhouse/clickhouse_dbwriter.png b/doc/architecture/blueprints/database/clickhouse/clickhouse_dbwriter.png Binary files differnew file mode 100644 index 00000000000..7aa2f0ef005 --- /dev/null +++ b/doc/architecture/blueprints/database/clickhouse/clickhouse_dbwriter.png diff --git a/doc/architecture/blueprints/database/clickhouse/ingestion_pipeline.md b/doc/architecture/blueprints/database/clickhouse/ingestion_pipeline.md new file mode 100644 index 00000000000..0fafc1bf524 --- /dev/null +++ b/doc/architecture/blueprints/database/clickhouse/ingestion_pipeline.md @@ -0,0 +1,289 @@ +--- +status: proposed +creation-date: "2023-01-10" +authors: [ "@ankitbhatnagar", "@ahegyi", "@mikolaj_wawrzyniak" ] +coach: "@grzesiek" +approvers: [ "@nhxnguyen", "@stkerr" ] +owning-stage: "~workinggroup::clickhouse" +participating-stages: [ "~section::ops", "~section::dev" ] +--- + +# Scalable data ingestion abstraction for ClickHouse + +## Table of Contents + +- [Summary](#summary) + - [Why](#why) + - [How](#how) +- [Motivation](#motivation) +- [Case Studies](#case-studies) + - [Replicating existing data into ClickHouse](#1-replicating-existing-data-into-clickhouse) + - [Ingesting large volumes of data into ClickHouse](#2-ingesting-large-volumes-of-data-into-clickhouse) +- [Goals](#goals) +- [Non-goals](#non-goals) +- [General considerations](#general-considerations) +- [Challenges building this](#major-challenges-around-building-such-a-capability) +- [Proposed solution](#proposed-solution) +- [Design & Implementation](#design--implementation) +- [References](#references) + +## Summary + +Develop a scalable & reliable data ingestion abstraction to help efficiently ingest large volumes of data from high throughput systems into ClickHouse. + +### Why + +To enable any application at GitLab to write necessary data into ClickHouse regardless of the scale at which they generate data today, or in the future. Refer to [Motivation](#motivation) for why ClickHouse in the first place. + +### How + +By building a write abstraction (API/Library) that allows a user to write data into ClickHouse and has all necessary configurations, conventions and best-practices around instrumentation, service-discovery, etc, built into it out of the box. + +## Motivation + +ClickHouse is an online, analytical processing (OLAP) database that powers use-cases that require fetching real-time, aggregated data that does not mutate a lot. ClickHouse is highly performant and can scale to large volumes of data as compared to traditional transactional relational databases (OLTP) such as Postgres, MySQL. For further reading around ClickHouse's capabilities, see [[1]](https://about.gitlab.com/blog/2022/04/29/two-sizes-fit-most-postgresql-and-clickhouse/), [[2]](https://clickhouse.com/blog/migrating-data-between-clickhouse-postgres) and [[3]](https://posthog.com/blog/clickhouse-vs-postgres). + +At GitLab, [our current and future ClickHouse uses/capabilities](https://gitlab.com/groups/gitlab-com/-/epics/2075) reference & describe multiple use-cases that could be facilitated by using ClickHouse as a backing datastore. A majority of these talk about the following two major areas of concern: + +1. Being able to leverage [ClickHouse's OLAP capabilities](https://clickhouse.com/docs/en/faq/general/olap/) enabling underlying systems to perform an aggregated analysis of data, both over short and long periods of time. +1. The fact that executing these operations with our currently existing datasets primarily in Postgres, is starting to become challenging and non-performant. + +Looking forward, assuming a larger volume of data being produced by our application(s) and the rate at which it gets produced, the ability to ingest it into a *more* capable system, both effectively and efficiently helps us scale our applications and prepare for business growth. + +## Case studies + +From an initial assessment of all (reported) use-cases that intend to utilise ClickHouse, the following broad patterns of usage can be observed: + +1. Efficiently replicating existing data from other databases into ClickHouse, most prominently Postgres. +1. Directly ingesting large volumes of data into ClickHouse for asynchronous processing, data aggregation & analysis. + +The following section(s) explain details of each problem-domain: + +### 1. Replicating existing data into ClickHouse + +With due reference to our prior work around this, it has been established that logical replication from Postgres is too slow. Instead, we'll need to be able to emit data change events within database transactions which can then get processed asynchronously to write or update corresponding data in ClickHouse. + +The following case-studies describe how these groups intend to solve the underlying problem: + +- ~group::optimize has been working towards a scalable PostgreSQL data replication strategy which can be implemented on the application layer. + + - [Proposal: Scalable data sync/replication strategy](https://gitlab.com/gitlab-org/gitlab/-/issues/382172) talks about such a strategy and the additional challenges with using Sidekiq for queueing/batching needs. + + - It has been observed that pumping data from `PostgreSQL` into `ClickHouse` directly might not be the right way to approach the problem at hand. + + - In addition to the problems described above, another class of problems when replicating data across systems is also the handling of data backfill and/or data migrations that happen upstream. + +- [group::data](https://about.gitlab.com/handbook/business-technology/data-team/) has been working around syncing data from some of our Postgres databases into a Snowflake-based data warehouse. See this issue for optioned considered: [List down all possible options for postgres to snowflake pipeline](https://gitlab.com/gitlab-data/gitlab.com-saas-data-pipeline/-/issues/13) before designing the current system in place. + + - With the work done around our [Next Gen GitLab SaaS Data Pipeline](https://docs.google.com/presentation/d/1hVaCY42YhaO5UvgLzp3mbuMYJIFuTFYFJjdhixFTxPE/edit#slide=id.g143a48de8a3_0_0), the data team owns a "custom" pipeline that does incremental data extractions based on an `updated_at` timestamp column. This helps import a significant subset of operational database relations into Snowflake data-warehouse. + + - As the volume of data grows, we can foresee this (ETL) pipeline warranting more time and resources to execute resulting in delays across the time between data being produced and being available in Snowflake data-warehouse. + + - We might also see data inconsistency/incompleteness issues emanating from the current setup since row deletions are not transferred into Snowflake, inflating data volume and skewing analysis. Any information about multiple updates happening between import interval period are also lost. + + - Having a scalable ingestion pipeline that can help replicate data from our databases into an intermediate system and/or ClickHouse in near real-time would help improve the operational characteristics around this system. + +### 2. Ingesting large volumes of data into ClickHouse + +We need to be able to ingest large volumes of potentially unaggregated data into ClickHouse which may result into a large number of small writes as well. This can have an adverse effect on how ClickHouse processes and stores incoming data. To mitigate this problem, we need to queue & batch smaller writes into larger ones to keep the ingestion pipeline efficient at all times. + +The following case-studies describe how each group intends to solve the underlying problem: + +- ~group::observability explains their need of ingesting large amounts of data into ClickHouse, with the following two issues: + + - [Proposal: GitLab Observability Platform - Data Ingestion](https://gitlab.com/gitlab-org/opstrace/opstrace/-/issues/1878) talks about using an external events store, such as Kafka, to first ingest data as received from users, then writing it into ClickHouse in larger batches thereby eliminating the need to write a large number of small writes without hampering write performance from how ClickHouse `MergeTree` processes ingested data. + + - In addition, [ClickHouse: Investigate client-side buffering to batch writes into ClickHouse](https://gitlab.com/gitlab-org/opstrace/opstrace/-/issues/2044) talks about their experimentation with using application-local queueing/batching to work around the problems mentioned above. + +- ~"group::product intelligence" has been working on building our analytics offering and recently looking at building and/or improving parts of the system. + + - [Product Analytics Collector Component](https://gitlab.com/groups/gitlab-org/-/epics/9346) talks about replacing Jitsu with Snowplow for collecting and processing tracking events. For more details of the proposal, see [Jitsu replacement](https://gitlab.com/gitlab-org/analytics-section/product-intelligence/proposals/-/blob/62d332baf5701810d9e7a0b2c00df18431e82f22/doc/jitsu_replacement.md). + + - The initial design was prototyped with [Snowplow as Jitsu Replacement PoC](https://gitlab.com/gitlab-org/analytics-section/product-analytics/devkit/-/merge_requests/37). + + - From the design, it is easy to observe how large amounts of data will be ingested into ClickHouse and could potentially benefit from the use of a scalable ingestion pipeline. + +## Goals + +### Well-defined, established client abstractions + +We want to define and establish a fully-functional application-side abstraction that can help ingest data into ClickHouse without getting in the way of how an application itself is designed while keeping the underlying code backend-agnostic. The proposed abstraction should become the default choice for any applications, core or satellite, at GitLab. + +### Support for high throughput in volume of writes + +A solution here should enable an application to write any amount of inserts (order of upto 1000-5000 writes per second) to the underlying database efficiently while also allowing for growth as the application scales out. Considering how ClickHouse processes incoming writes, a proposed solution should be able to batch a number of very small writes into larger batches. + +### Reliable, consistent delivery of data + +A solution here should also ensure reliable & consistent delivery of ingested data into the underlying database minimising undue loss of data before being eventually persisted into ClickHouse. + +## Non-goals + +### Addressing data types, schemas or formats + +At this stage of this proposal, we're not optimizing for addressing which data types, schemas or formats we receive ingested data in. It should be delegated to the backend-specific implementations themselves and not handled within the write abstraction. + +### Addressing where our data sources exist today + +We're also not addressing any client-side specific details into the design at this point. The write abstraction should only remain a tool for the language in which it is written. As long as an application can use it to write data as any other third-party library, we should be good to build on top of it. + +## General Considerations + +Having addressed the details of the two aformentioned problem-domains, we can model a proposed solution with the following logical structure: + +- Ingestion + - APIs/SDKs + - HTTP2/gRPC Sidecar +- Transport & Routing + - Multi-destination +- Digestion/Compute + - Enrichment + - Processing + - Persisting + +## Major challenges around building such a capability + +### Self-managed environments + +The single, biggest challenge around introducing ClickHouse and related systems would be the ability to make it avaiable to our users running GitLab in self-managed environments. The intended goals of this proposal are intentionally kept within those constraints. It is also prudent to establish that what we're *proposing* here be applicable to applications consuming ClickHouse from inside self-managed environments. + +There are ongoing efforts to streamline distribution and deployment of ClickHouse instances for managed environment within the larger scope of [ClickHouse Usage at GitLab](../../clickhouse_usage/index.md). A few other issues tackling parts of the aforementioned problem are: + +- [Research and understand component costs and maintenance requirements of running a ClickHouse instance with GitLab](https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/14384) +- [ClickHouse maintenance and cost research](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116669) + +### Wide variety of data sources, their structures & usage patterns + +The data that we intend to ingest into ClickHouse can come from a wide variety of data sources and be structured in different schemas or formats. With that considered, it's non-trivial effort to draft a solution that suffices all use-cases efficiently. + +Should we decide to build an intermediate ingestion system, any solution should help provide a source/schema/format-agnostic data transport layer with an established, matured client-abstraction to maximise the number of applications that can use it. + +### Building on top of our current database infrastructure + +Our current database infrastructure operates at a fairly large scale and adding more applications that continuously read/write against it adds to the pressure on the existing resources. It's important we move away any workloads and/or datasets that can be safely processed in a different context altogether. + +### Service Discovery + +We're still normalising the details around distribution and deployment of ClickHouse clusters and/or instances for our applications. Subject to how we end up doing it, for a client to be able to discover which ClickHouse cluster, shard or table would need to become a part any such solution. + +## Proposed Solution + +In light of the problems discussed earlier, it'd be in our better interests to allow the usage of an external, intermediate system subject to what one's needs might be especially around the volume & scale of data being writen from an application into ClickHouse. + +Therefore, we intend to develop an abstraction that can enable an application to store data into ClickHouse regardless of the scale that they (currently) operate at. It also: + +- Facilitates an application to switch from one *technology* to another should their performance and/or scale requirements change over time. +- Allows for backend-specific conventions, configurations & best practices such as instrumentation, service-discovery, etc. to be encoded in one place for all applications to leverage consistently. + +## Design & Implementation + +### Core assumptions + +- We're only going to focus on writing data into ClickHouse as mentioned in aforementioned non-goals. With details of how our data lands into ClickHouse, this document does not (intentionally) address where this data comes from. Some of those details are delegated to the applications generating this data i.e as long as they can consume this abstraction, they should be able to write data into ClickHouse. + +- We're going to delegate the choice of different storage backends to a following blueprint or epic since that's outside the scope of this design. With ClickHouse as the eventual destination for our data, this document only talks about writing data into it - either directly or indirectly via a queueing/batching system. + +### Architecture + + + +Having an abstraction around writing data help client-side instrumentation to stay backend-agnostic allowing them to switch code paths depending on where it runs. + +An example setup should look like: + +```ruby +Gitlab::Database::Writer.config do |config| + # + # when using sync mode, data gets written directly into ClickHouse, + # therefore, it's also assumed the backend here is ClickHouse + config.mode = :sync OR :async + config.backend = :clickhouse # optional + # OR + # + # when using async mode, data is written to an intermediate system + # first, then written into ClickHouse asynchronously + config.mode = :async + config.backend = :pubsub OR :kafka OR :otherbackend + # + # then backend-specific configurations hereafter + # + config.url = 'tcp://user:pwd@localhost:9000/database' + # e.g. a serializer helps define how data travels over the wire + config.json_serializer = ClickHouse::Serializer::JsonSerializer + # ... +end +# do application-specific processing +# eventually, write data using the object you just built +Gitlab::Database::Writer.write( + Gitlab::Database::Model::MyTable, + [{ id: 1, foo: 'bar' }], +) +``` + +We intend to keep `Gitlab::Database::Writer.backend` to be as close to the backend-specific client implementation as possible. Having a wrapper around a vanilla client helps us address peripheral concerns such as service-discovery for the backends while still allowing the user to leverage features of a given client. + +### Iterations + +Considering the large scope of this undertaking and the need for feedback around actual usage, we intend to build the proposed abstraction(s) across multiple iterations which can be described as follows: + +#### Iteration 1 - Develop write abstraction with sync mode enabled + +First, research and develop a simple write abstraction that our users can begin to use to write data into ClickHouse. This ensures our choice of the underlying client is well-researched and suffices to fulfill needs of as many reported use-cases as possible. Being able to see this running would help gather user-feedback and improve the write APIs/interfaces accordingly. + +Given this feedback and more development with how we aim to deploy ClickHouse across our environments, it'd then be prudent to build into this abstraction necessary conventions, best practices and abstract away details around connection-pooling, service-discovery, etc. + +#### Iteration 2 - Add support for schemas & data validation + +In the next iteration, we plan to add support for schema usage and validation. This helps keep model definitions sane and allows for validating data to be inserted. + +#### Iteration 3 - Add support for async mode, PoC with one backend + +With the above two iterations well-executed, we can start to scale up our write abstractions adding the support for writing data into intermediate data stores before writing it into ClickHouse asynchronously. We aim to prototype such an implementation with atleast one such backend. + +#### Further iterations + +With a backend-agnostic abstraction becoming the ingestion interface a client interacts with, there's various other use-cases that can be solved from within this abstraction. Some of them are: + +- Zero-configuration data ingestion from multiple sources +- Dynamically enriching data from multiple sources +- Offloading data to long-term retention data stores + +### Possible backend implementations + +- Applications writing directly to ClickHouse + - Application-local in-memory queueing/batching of data + - Application-local persistent queueing/batching of data +- Non-local queueing/batching of data before eventually writing into ClickHouse + - Managed cloud backends: + - [Google PubSub](https://cloud.google.com/pubsub) + - [AWS Kinesis](https://aws.amazon.com/kinesis/) + - Self-managed backends: + - [CHProxy](https://www.chproxy.org/) + - [Kafka](https://kafka.apache.org/) + - [RedPanda](https://redpanda.com/) + - [Vector](https://vector.dev/) + - [RabbitMQ](https://www.rabbitmq.com/) + +### Additional complexity when using a non-local backend + +- The need for running an additional process/sub-system that reads data from the concerned backend and writes it into ClickHouse efficiently and reliably. +- The additional hop across the backend also means that there might be potential delays in how soon this data lands into ClickHouse. + +Though the points above describe additional complexity for an application, they can be treated as valid trade-off(s) assuming their need for data ingestion at scale. + +### Comparing backends across multiple dimensions + +| Dimension | CHProxy | Redis | Google PubSub | Apache Kafka | +|---|---|---|---|---| +| Operations | Trivial | Trivial | Managed | Non-trivial, complex | +| Data Retention | Non-durable | Non-durable | Durable | Durable | +| Performance | Good | Good | High | High | +| Data Streaming | None | Minimal | Good | Best | +| Suitable for self-managed environments | Trivial | Trivial | - | Complex | + +## References + +- [ClickHouse use-cases within Manage](https://gitlab.com/groups/gitlab-org/-/epics/7964) +- [List down all possible options for postgres to snowflake pipeline](https://gitlab.com/gitlab-data/gitlab.com-saas-data-pipeline/-/issues/13) +- [Design Spike for Snowplow For Data Event capture](https://gitlab.com/gitlab-data/analytics/-/issues/12397) +- [Audit Events Performance Limits](https://gitlab.com/gitlab-org/gitlab/-/issues/375545) diff --git a/doc/architecture/blueprints/database/clickhouse/read_abstraction_layer.md b/doc/architecture/blueprints/database/clickhouse/read_abstraction_layer.md new file mode 100644 index 00000000000..82470867a30 --- /dev/null +++ b/doc/architecture/blueprints/database/clickhouse/read_abstraction_layer.md @@ -0,0 +1,307 @@ +--- +status: proposed +creation-date: "2023-02-23" +authors: [ "@mikolaj_wawrzyniak", "@jdrpereira", "@pskorupa" ] +coach: "@DylanGriffith" +approvers: [ "@nhxnguyen" ] +owning-stage: "~workinggroup::clickhouse" +participating-stages: [] +--- + +# Consider an abstraction layer to interact with ClickHouse or alternatives + +## Summary + +Provide a solution standardizing read access to ClickHouse or its alternatives for GitLab installations that will not opt-in to install ClickHouse. After analyzing different [open-source tools](#overview-of-open-source-tools) and weighing them against an option to [build a solution internally](#recommended-approach). The current recommended approach proposes to use dedicated database-level drivers to connect to each data source. Additionally, it proposes the usage of [repository pattern](https://martinfowler.com/eaaCatalog/repository.html) to confine optionally database availability complexity to a single application layer. + +## Motivation + +ClickHouse requires significant resources to be run, and smaller installations of GitLab might not get a return from investment with provided performance improvement. That creates a risk that ClickHouse might not be globally available for all installations and features might need to alternate between different data stores available. Out of all [present & future ClickHouse use cases](https://gitlab.com/groups/gitlab-com/-/epics/2075) that have been already proposed as part of the working group 7 out of 10 uses data stores different than ClickHouse. Considering that context it is important to +support those use cases in their effort to adopt ClickHouse by providing them with tools and guidelines that will standardize interactions with available data stores. + +The proposed solution can take different forms from stand-alone tooling +offering a unified interface for interactions with underlying data stores, to a set of libraries supporting each of the data stored individually backed by implementation guidelines that will describe rules and limitations placed around data stores interactions, and drawing borders of encapsulation. + +## Goals + +- Limit the impact of optionally available data stores on the overall GitLab application codebase to [single abstraction layer](../../../../development/reusing_abstractions.md#abstractions) +- Support all data store specific features +- Support communication for satellite services of the main GitLab application + +## Non-goals + +- This proposal does not directly consider write communication with database, as this is a subject of [complementary effort](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111148) +- This proposal does not directly consider schema changes and data migration challenges + +Despite above points being non goals, it is acknowledge that they might impose some alterations to final solution which is expressed at the end of this document in the [Open questions](#open-questions) section. + +## Possible Solutions + +High-level goals described in the previous paragraph can be achieved by both in-house-built solutions as well as by adopting open-source tools. +The following sections will take a closer look into both of those avenues + +### Recommended approach + +In the spirit of MVC and iteration, it is proposed to start with a solution that would rely on drivers that directly interact +with corresponding data stores, like ActiveRecord for Ruby. For this solution to be able to achieve goals set for +this exit criteria and help mitigate the issue listed in the _Motivation_ section of this document, such drivers need to be supported +by a set of development guidelines enforced with static code analysis. + +Such a solution was selected as preferred upon receiving feedback from different members of the working group concerned +about the risk of limitations that might be imposed by open-source tools, preventing groups from taking advantage of ClickHouse +features to their fullest. Members collaborating around working group criteria presented in this document, agree that +concerns around limitations could be mitigated by building a comprehensive set of prototypes, however time and effort +required to achieve that surpass the limits of this working group. It is also important to notice that ClickHouse adoption +is in an exploratory stage, and groups might not being even able to state what are their requirements just yet. + +#### Proposed drivers + +Following ClickHouse documentation there are the following drivers for Ruby and Go + +##### Ruby + +1. [ClickHouse Ruby driver](https://github.com/shlima/click_house) - Previously selected for use in GitLab as part of the Observability grup's research (see: [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/358158)) +1. [Clickhouse::Activerecord](https://github.com/PNixx/clickhouse-activerecord) + +##### Go + +1. [ClickHouse/clickhouse-go](https://github.com/ClickHouse/clickhouse-go) - Official SQL database client. +1. [uptrace/go-clickhouse](https://clickhouse.uptrace.dev/) - Alternative client. + +##### Proposed client architecture + +To keep the codebase well organized and limit coupling to any specific database engine it is important to encapsulate +interactions, including querying data to a single application layer, that would present its interface to layers above in +similar vain to [ActiveRecord interface propagation through abstraction layers](../../../../development/reusing_abstractions.md) + +Keeping underlying database engines encapsulated makes the recommended solution a good two-way door decision that +keeps the opportunity to introduce other tools later on, while giving groups time to explore and understand their use cases. + +At the lowest abstraction layer, it can be expected that there will be a family of classes directly interacting with the ClickHouse driver, those classes +following MVC pattern implemented by Rails should be classified as _Models_. + +Models-level abstraction builds well into existing patterns and guidelines but unfortunately does not solve the challenge of the optional availability of the ClickHouse database engine for self-managed instances. It is required to design a dedicated entity that will house responsibility of selecting best database to serve business logic request. +From the already mentioned existing abstraction [guidelines](../../../../development/reusing_abstractions.md) `Finders` seems to be the closest to the given requirements, due to the fact that `Finders` encapsulate database specific interaction behind their own public API, hiding database vendors detail from all layers above them. + +However, they are closely coupled to `ActiveRecord` ORM framework, and are bound by existing GitLab convention to return `ActiveRecord::Relation` objects, that might be used to compose even more complex queries. That coupling makes `Finders` unfit to deal with the optional availability of ClickHouse because returned data might come from two different databases, and might not be compatible with each other. + +With all that above in mind it might be worth considering adding a new entity into the codebase that would exist on a similar level of abstraction as `Finders` yet it would be required to return an `Array` of data objects instead. + +Required level of isolation can be achieved with usage of a [repository pattern](https://martinfowler.com/eaaCatalog/repository.html). The repository pattern is designed to separates business / domain logic from data access concerns, which is exactly what this proposal is looking for. +What is more the repository pattern does not limits operations performed on underlying databases allowing for full utilization of their features. + +To implement the repository pattern following things needs to be created: + +1. A **strategy** for each of supported databases, for example: `MyAwesomeFeature::Repository::Strategies::ClickHouseStrategy` and `MyAwesomeFeature::Repository::Strategies::PostgreSQLStrategy`. Strategies are responsible for implementing communication with underlying database ie: composing queries +1. A **repository** that is responsible for exposing high level interface to interact with database using one of available strategies selected with some predefined criteria ie: database availability. Strategies used by single repository must share the same public interface so they can be used interchangeable +1. A **Plain Old Ruby Object(PORO) Model** that represents data in business logic implemented by application layers using repository. It have to be database agnostic + +It is important to notice that the repository pattern based solution has already been implemented by Observability group (kudos to: @ahegyi, @splattael and @brodock). [`ErrorTracking::ErrorRepository`](https://gitlab.com/gitlab-org/gitlab/-/blob/1070c008b9e72626e25296480f82f2ee2b93f847/lib/gitlab/error_tracking/error_repository.rb) is being used to support migration of error tracking features from PostgreSQL to ClickHouse (integrated via API), and uses feature flag toggle as database selection criteria, that is great example of optional availability of database. + +`ErrorRepository` is using two strategies: + +1. [`OpenApiStrategy`](https://gitlab.com/gitlab-org/gitlab/-/blob/d0bdc8370ef17891fd718a4578e41fef97cf065d/lib/gitlab/error_tracking/error_repository/open_api_strategy.rb) to interact with ClickHouse using API proxy entity +1. [`ActiveRecordStrategy`](https://gitlab.com/gitlab-org/gitlab/-/blob/d0bdc8370ef17891fd718a4578e41fef97cf065d/lib/gitlab/error_tracking/error_repository/active_record_strategy.rb) to interact with PostgreSQL using `ActiveRecord` framework + +Each of those strategies return data back to abstraction layers above using following PORO Models: + +1. [`Gitlab::ErrorTracking::Error`](https://gitlab.com/gitlab-org/gitlab/-/blob/a8ea29d51ff23cd8f5b467de9063b64716c81879/lib/gitlab/error_tracking/error.rb) +1. [`Gitlab::ErrorTracking::DetailedError`](https://gitlab.com/gitlab-org/gitlab/-/blob/a8ea29d51ff23cd8f5b467de9063b64716c81879/lib/gitlab/error_tracking/detailed_error.rb) + +Additionally `ErrorRepository` is great example of remarkable flexibility offered by the repository pattern in terms of supported types of data stores, allowing to integrate solutions as different as a library and external service API under single unified interface. That example presents opportunity that the repository pattern in the future might be expanded beyond needs of ClickHouse and PostgreSQL when some use case would call for it. + +Following [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85907/diffs) documents changes done by observability group in order to migrate from using current GitLab architecture based on ActiveRecord Models, Services and Finders to the repository pattern. + +##### Possible ways to enforce client architecture + +It is not enough to propose a client-side architecture for it to fully be established as common practice it needs +to be automatically enforced, reducing the risk of developers unconsciously going against it. There are multiple ways to +introduce automated verification of repository pattern implementation including: + +1. Utilize `ActiveRecord` query subscribers in a similar way to[Database::PreventCrossJoins](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/database/prevent_cross_joins.rb) in order to detect queries to ClickHouse executed outside of _Strategies_ +1. Expanding [`CodeReuse`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/rubocop/cop/code_reuse) rubocop rules to flag all usage of ClickHouse driver outside of _Strategies_ +1. Create rubocop rule that detects calls to utility method that checks the presence of ClickHouse instance (ie: `CurrentSettings.click_house_enabled?`) that are being made outside of _Repositories_ + +At this development stage, authors see all of the listed options as viable and promising, therefore a decision about which ones to use would be deferred to the moment when the first repository pattern implementation for ClickHouse will emerge. + +### Overview of open-source tools + +In this section authors provide an overview of existing 3rd party open-source solutions that were considered as alternative approaches to achieve stated goal, but was not selected as recommended approach. + +#### Evaluation criteria + +##### 1. License (MUST HAVE) + +1. Solutions must be open source under an [acceptable license](https://about.gitlab.com/handbook/engineering/open-source/#acceptable-licenses). + +##### 2. Support for different data stores (MUST HAVE) + +1. It focuses on the fact whether the proposed abstraction layer can support both ClickHouse and PostgreSQL (must have) +1. Additional consideration might be if more than the two must-have storages are supported +1. The solution must support the [minimum required versions](../../../../install/requirements.md#postgresql-requirements) for PostgreSQL + +##### 3. Protocol compatibility + +Every abstraction layer comes at the cost of limited API compared to direct access to the tool. This exit criterion is trying to bring understanding to the degree of trade-off being made on limiting tools API for the sake of a common abstraction. + +1. List what read operations can be done via PostgreSQL and ClickHouse (`selects`, `joins`, `group by`, `order by`, `union` etc) +1. List what operations can be done with the proposed abstraction layer, how complicated it is to do such operations, and whether are there any performance concerns when compared to running operations natively +1. Does it still allow for direct access to a data source in case the required operation is not supported by the abstraction layer, eg: `ActiveRecord` allows for raw SQL strings to be run with `#execute` + +##### 4. Operational effort + +1. Deployment process: how complex is it? Is the proposed tool a library tool that is being added into the stack, or does it require additional services to be deployed independently along the GitLab system. What deployment types does the tool support (Kubernetes/VMs, SaaS/self-managed, supported OS, cloud providers). Does it support offline installation. +1. How many hardware resources does it need to operate +1. Does it require complex monitoring and operations to assure stable and performant services +1. Matured maintenance process and documentation around it: upgrades, backup and restore, scaling +1. High-availability support. Does the tool have documentation how to build HA cluster and perform failovers for self-managed? Does the tool support zero-downtime upgrade? +1. FIPS and FedRAMP compliance +1. Replication process and how the new tool would fit in GitLab Geo. + +##### 5. Developer experience + +1. Solutions must have well-structured, clear, and thoroughly documented APIs to ease adoption and reduce the learning curve. + +##### 6. Maturity (nice to have) + +1. How long does the solution exist? Is it used often? Does it have a stable community? If the license permits forking tool is also a considerable option + +##### 7. Tech fit + +1. Is the solution written in one of the programming languages we use at GitLab so that we can more easily contribute with bug fixes and new features? + +##### 8. Interoperability (Must have) + +1. Can the solution support both the main GitLab application written in Ruby on Rails also satellite services like container registry that might be written in Go + +#### Open - Source solutions + +##### 1. [Cube.dev](https://cube.dev/) + +**Evaluation** + +1. License + Apache 2.0 + MIT ✅ +1. Support for different data stores + Yes ✅ +1. Protocol compatibility + It uses OLAP theory concepts to aggregate data. This might be useful in some use cases like aggregating usage metrics, but not in others. It has APIs for both SQL queries and their own query format. +1. Operational effort + Separate service to be deployed using Docker or k8s. Uses Redis as a cache and data structure store. +1. Developer experience + Good [documentation](https://cube.dev/docs) +1. Maturity + Headless BI tools themselves are a fairly new idea, but Cube.js seems to be the leading open-source solution in this space. + The Analytics section uses it internally for our Product Analytics stack. +1. Tech fit + Uses REST and GraphQL APIs. It has its own query and data schema formats, but they are well-documented. Data definitions in either YAML or JavaScript. + +**Comment** + +The solution is already being used as a read interface for ClickHouse by ~"group::product analytics", +to gather first hand experience there was a conversation held with @mwoolf with key conclusions being: + +1. ClickHouse driver for cube.dev is community-sourced, and it does not have a maintainer as of now, which means there is no active development. It is a small and rather simple repository that should work at least until a new major version of ClickHouse will arrive with some breaking changes +1. Cube.dev is written in Type Script and JavaScript which are part of GitLab technical stack, and there are engineers here with expertise in them, however Cube.dev is expected to be mostly used by backend developers, which does not have that much experience in mentioned technologies +1. Abstraction layer for simple SQL works, based on JSON will build correct query depending on the backend +1. Data store-specific functions (like window funnel ClickHouse) are not being translated to other engines, which requires additional cube schemas to be built to represent the same data. +1. Performance so far was not an issue both on local dev and on AWS VPS millions of rows import load testing +1. It expose postgres SQL like interface for most engines, but not for ClickHouse unfortunately so for sake of working group use case JSON API might be more feasible +1. Cube.dev can automatically generate schemas on the fly, which can be used conditionally in the runtime handling optional components like ClickHouse + +There is also a [recording](https://youtu.be/iBPTCrvOBBs) of that conversation available. + +##### 2. [ClickHouse FDW](https://github.com/ildus/clickhouse_fdw) + +**Evaluation** + +A ClickHouse Foreign Data Wrapper for PostgreSQL. It allows ClickHouse tables to be queried as if they were stored in PostgreSQL. +Could be a viable option to easily introduce ClickHouse as a drop-in replacement when Postgres stops scaling. + +1. License + Apache 2.0 ✅ +1. Support for different data stores + Yes, by calling ClickHouse through a PostgreSQL instance. ✅ +1. Protocol compatibility + Supports SELECT, INSERT statements at a first glance. Not sure about joins. Allows for raw SQL by definition. +1. Operational effort + 1. A PostgreSQL extension. Requires some mapping between the two DBs. + 1. Might have adversary impact on PostgreSQL performance, when execution would wait for response from ClickHouse waisting CPU cycles on waiting + 1. Require exposing and managing connection between deployments of PostgreSQL and ClickHouse +1. Developer experience + TBD +1. Maturity + It's been around for a few years and is listed in ClickHouse docs, but doesn't seem to be widely used. +1. Tech fit + Raw SQL statements. + +**Comment** + +##### 3. [Clickhouse::Activerecord](https://github.com/PNixx/clickhouse-activerecord) + +**Evaluation** + +1. License + MIT License ✅ +1. Support for different data stores + Yes, in the sense that it provides a Clickhouse adapter for ActiveRecord in the application layer so that it can be used to query along PostgreSQL. ✅ +1. Protocol compatibility + Not sure about joins - no examples. +1. Operational effort + Ruby on Rails library tool - ORM interface in a form of an ActiveRecord adapter. +1. Developer experience + Easy to work with for developers familiar with Rails. +1. Maturity + Has been around for a few years, but repo activity is scarce (not a bad thing by itself, however). +1. Tech fit + Rails library, so yes. + +**Comment** + +##### 4. [Metriql](https://metriql.com/) + +**Evaluation** + +A headless BI solution using DBT to source data. Similar to Cube.dev in terms of defining metrics from data and transforming them with aggregations. +The authors explain the differences between Metriql and other BI tools like Cube.js in this FAQ entry. + +1. License + Apache 2.0 ✅ +1. Support for different data stores + Uses DBT to read from data sources, so CH and PostgreSQL are possible. +1. Protocol compatibility + It uses OLAP theory concepts to aggregate data. It does allow for impromptu SQL queries through a REST API. +1. Operational effort + It's a separate service to deploy and requires DBT. +1. Developer experience + I assume it requires DBT knowledge to set up and use. It has a fairly simple REST API documented here. +1. Maturity + First release May 2021, but repo activity is scarce (not a bad thing by itself). +1. Tech fit + Connects with BI tools through a REST API or JDBC Adapter. Allows querying using SQL or MQL (which is a SQL flavor/subset). + +**Comment** + +##### 5. Notable rejected 3rd party solutions + +ETL only solutions like Airflow and Meltano, as well as visualization tools like Tableau and Apache Superset, were excluded from the prospect list as they are usually clearly outside our criteria. + +**[pg2ch](https://github.com/mkabilov/pg2ch)** +PostgreSQL to ClickHouse mirroring using logical replication. +Repo archived; explicitly labeled not for production use. Logical replication might not be performant enough at our scale - we don't use it in our PostgreSQL DBs because of performance concerns. + +**Looker** +BI tooling. +Closed-source; proprietary. + +**[Hasura](https://github.com/hasura/graphql-engine)** +GraphQL interface for database sources. +No ClickHouse support yet. + +**[dbt Server](https://github.com/dbt-labs/dbt-server)** +HTTP API for dbt. MariaDB Business Source License (BSL) ❌ + +### Open questions + +1. This proposal main focus is read interface, however depending on outcome of [complementary effort](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111148) that focus on write interface similar concerns around optional availability might be applicable to write interaction. In case if ingestion pipeline would not resolve optional availability challenges for write interface it might be considerable to include write interactions into repository pattern implementation proposed in this document. +1. Concerns around ClickHouse schema changes and data migrations is not covered by any existing working group criteria, even though solving this challenges as a whole is outside of the scope of this document it is prudent to raise awareness that some alterations to proposed repository pattern based implementation might be required in order to support schema changes. diff --git a/doc/architecture/blueprints/database/scalability/patterns/index.md b/doc/architecture/blueprints/database/scalability/patterns/index.md index ec00d757377..d28734ce511 100644 --- a/doc/architecture/blueprints/database/scalability/patterns/index.md +++ b/doc/architecture/blueprints/database/scalability/patterns/index.md @@ -2,7 +2,6 @@ stage: Data Stores group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false description: 'Learn how to scale the database through the use of best-of-class database scalability patterns' --- diff --git a/doc/architecture/blueprints/database/scalability/patterns/read_mostly.md b/doc/architecture/blueprints/database/scalability/patterns/read_mostly.md index ec236c9bfe3..3a3fd2f33c2 100644 --- a/doc/architecture/blueprints/database/scalability/patterns/read_mostly.md +++ b/doc/architecture/blueprints/database/scalability/patterns/read_mostly.md @@ -2,7 +2,6 @@ stage: Data Stores group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false description: 'Learn how to scale operating on read-mostly data at scale' --- diff --git a/doc/architecture/blueprints/database/scalability/patterns/time_decay.md b/doc/architecture/blueprints/database/scalability/patterns/time_decay.md index 2b36a43a6db..24fc3f45717 100644 --- a/doc/architecture/blueprints/database/scalability/patterns/time_decay.md +++ b/doc/architecture/blueprints/database/scalability/patterns/time_decay.md @@ -2,7 +2,6 @@ stage: Data Stores group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false description: 'Learn how to operate on large time-decay data' --- diff --git a/doc/architecture/blueprints/database_scaling/size-limits.md b/doc/architecture/blueprints/database_scaling/size-limits.md index e530bd6eff0..b6b9cda8827 100644 --- a/doc/architecture/blueprints/database_scaling/size-limits.md +++ b/doc/architecture/blueprints/database_scaling/size-limits.md @@ -1,7 +1,6 @@ --- stage: Data Stores group: Database -comments: false description: 'Database Scalability / Limit table sizes' --- diff --git a/doc/architecture/blueprints/database_testing/index.md b/doc/architecture/blueprints/database_testing/index.md index fe6dcf1723d..79560dd3959 100644 --- a/doc/architecture/blueprints/database_testing/index.md +++ b/doc/architecture/blueprints/database_testing/index.md @@ -1,5 +1,5 @@ --- -status: accepted +status: implemented creation-date: "2021-02-08" authors: [ "@abrandl" ] coach: "@glopezfernandez" @@ -8,8 +8,15 @@ owning-stage: "~devops::data_stores" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Database Testing +**Notice:** This blueprint has been partially implemented. We still plan to +iterate on the tooling. The content below is a historical version of the +blueprint, written prior to incorporating database testing into our development +workflow. + We have identified [common themes of reverted migrations](https://gitlab.com/gitlab-org/gitlab/-/issues/233391) and discovered failed migrations breaking in both production and staging even when successfully tested in a developer environment. We have also experienced production incidents even with successful testing in staging. These failures are quite expensive: they can have a significant effect on availability, block deployments, and generate incident escalations. These escalations must be triaged and either reverted or fixed forward. Often, this can take place without the original author's involvement due to time zones and/or the criticality of the escalation. With our increased deployment speeds and stricter uptime requirements, the need for improving database testing is critical, particularly earlier in the development process (shift left). From a developer's perspective, it is hard, if not unfeasible, to validate a migration on a large enough dataset before it goes into production. @@ -86,13 +93,13 @@ The short-term focus is on testing regular migrations (typically schema changes) In order to secure this process and meet compliance goals, the runner environment is treated as a *production* environment and similarly locked down, monitored and audited. Only Database Maintainers have access to the CI pipeline and its job output. Everyone else can only see the results and statistics posted back on the merge request. -We implement a secured CI pipeline on <https://ops.gitlab.net> that adds the execution steps outlined above. The goal is to secure this pipeline to solve the following problem: +We implement a secured CI pipeline on [Internal GitLab for Operations](https://ops.gitlab.net/users/sign_in) that adds the execution steps outlined above. The goal is to secure this pipeline to solve the following problem: Make sure we strongly protect production data, even though we allow everyone (GitLab team/developers) to execute arbitrary code on the thin-clone which contains production data. This is in principle achieved by locking down the GitLab Runner instance executing the code and its containers on a network level, such that no data can escape over the network. We make sure no communication can happen to the outside world from within the container executing the GitLab Rails code (and its database migrations). -Furthermore, we limit the ability to view the results of the jobs (including the output printed from code) to Maintainer and Owner level on the <https://ops.gitlab.net> pipeline and provide only a high level summary back to the original MR. If there are issues or errors in one of the jobs run, the database Maintainer assigned to review the MR can check the original job for more details. +Furthermore, we limit the ability to view the results of the jobs (including the output printed from code) to Maintainer and Owner level on the [Internal GitLab for Operations](https://ops.gitlab.net/users/sign_in) pipeline and provide only a high level summary back to the original MR. If there are issues or errors in one of the jobs run, the database Maintainer assigned to review the MR can check the original job for more details. With this step implemented, we already have the ability to execute database migrations on the thin-cloned GitLab.com database automatically from GitLab CI and provide feedback back to the merge request and the developer. The content of that feedback is expected to evolve over time and we can continuously add to this. diff --git a/doc/architecture/blueprints/gitlab_agent_deployments/index.md b/doc/architecture/blueprints/gitlab_agent_deployments/index.md index 0e5b5f0b248..d8d26389d7d 100644 --- a/doc/architecture/blueprints/gitlab_agent_deployments/index.md +++ b/doc/architecture/blueprints/gitlab_agent_deployments/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::release" participating-stages: [Configure, Release] --- +<!-- vale gitlab.FutureTense = NO --> + # View and manage resources deployed by GitLab Agent For Kuberenetes ## Summary diff --git a/doc/architecture/blueprints/gitlab_observability_backend/metrics/index.md b/doc/architecture/blueprints/gitlab_observability_backend/metrics/index.md index 82d1596bc90..3edb01d9140 100644 --- a/doc/architecture/blueprints/gitlab_observability_backend/metrics/index.md +++ b/doc/architecture/blueprints/gitlab_observability_backend/metrics/index.md @@ -8,6 +8,8 @@ owning-stage: "~monitor::observability" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # GitLab Observability Backend - Metrics ## Summary diff --git a/doc/architecture/blueprints/graphql_api/index.md b/doc/architecture/blueprints/graphql_api/index.md index 95ff834cd27..2c277049434 100644 --- a/doc/architecture/blueprints/graphql_api/index.md +++ b/doc/architecture/blueprints/graphql_api/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::manage" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # GraphQL API [GraphQL](https://graphql.org/) is a data query and manipulation language for diff --git a/doc/architecture/blueprints/object_storage/index.md b/doc/architecture/blueprints/object_storage/index.md index 6718705a7c8..3f649960554 100644 --- a/doc/architecture/blueprints/object_storage/index.md +++ b/doc/architecture/blueprints/object_storage/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::data_stores" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Object storage: `direct_upload` consolidation ## Abstract @@ -124,7 +126,7 @@ infrastructure. It also makes the initial installation more complex feature after feature. Implementing a direct upload by default, with a -[consolidated object storage configuration](../../../administration/object_storage.md#consolidated-object-storage-configuration) +[consolidated object storage configuration](../../../administration/object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form) will reduce the number of merge requests needed to ship a new feature from four to only one. It will also remove the need for SRE intervention as the bucket will always be the same. diff --git a/doc/architecture/blueprints/rate_limiting/index.md b/doc/architecture/blueprints/rate_limiting/index.md index 92364040962..17808267032 100644 --- a/doc/architecture/blueprints/rate_limiting/index.md +++ b/doc/architecture/blueprints/rate_limiting/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::enablement" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Next Rate Limiting Architecture ## Summary diff --git a/doc/architecture/blueprints/remote_development/img/remote_dev_15_7.png b/doc/architecture/blueprints/remote_development/img/remote_dev_15_7.png Binary files differindex d0849ded94f..f36bfa24998 100644 --- a/doc/architecture/blueprints/remote_development/img/remote_dev_15_7.png +++ b/doc/architecture/blueprints/remote_development/img/remote_dev_15_7.png diff --git a/doc/architecture/blueprints/remote_development/index.md b/doc/architecture/blueprints/remote_development/index.md index 162ae04f6b6..f214ef56967 100644 --- a/doc/architecture/blueprints/remote_development/index.md +++ b/doc/architecture/blueprints/remote_development/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::create" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Remote Development ## Summary @@ -68,17 +70,6 @@ Container/VM-based developer machines providing all the tools and dependencies n - A workspace should be a combination of resources that support cloud-based development environment. - Workspaces are constrained by the amount of resources provided to them. -### Legacy Web IDE - -The current production [Web IDE](../../../user/project/web_ide/index.md). - -#### Legacy Web IDE properties - -An advanced editor with commit staging that currently supports: - -- [Live Preview](../../../user/project/web_ide/index.md#live-preview-removed) -- [Interactive Web Terminals](../../../user/project/web_ide/index.md#interactive-web-terminals-for-the-web-ide) - ### Web IDE VS Code for web - replacement of our current legacy Web IDE. @@ -137,145 +128,6 @@ As a zero-install development environment that runs in your browser, Remote Deve GitLab.com is only hosted within the United States of America. Organizations located in other regions have voiced demand for local SaaS offerings. BYO infrastructure helps work in conjunction with [GitLab Regions](https://gitlab.com/groups/gitlab-org/-/epics/6037) because a user's workspace may be deployed within different geographies. The ability to deploy workspaces to different geographies might also help to solve data residency and compliance problems. -## High-level architecture problems to solve - -A number of technical issues need to be resolved to implement a stable Remote Development offering. This section will be expanded. - -- Who is our main persona for BYO infrastructure? -- How do users authenticate? -- How do we support more than one IDE? -- How are workspaces provisioned? -- How can workspaces implement disaster recovery capabilities? -- If we cannot use SSH, what are the viable alternatives for establishing a secure WebSocket connection? -- Are we running into any limitations in functionality with the Web IDE by not having it running in the container itself? For example, are we going to get code completion, linting, and language server type features to work with our approach? -- How will our environments be provisioned, managed, created, destroyed, etc.? -- To what extent do we need to provide the user with a UI to interact with the provisioned environments? -- How will the files inside the workspace get live updated based on changes in the Web IDE? Are we going to use a [CRDT](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type)-like setup to patch files in a container? Are we going to generate a diff and send it though a WebSocket connection? - -## Iteration plan - -We can't ship the entire Remote Development architecture in one go - it is too large. Instead, we are adopting an iteration plan that provides value along the way. - -- Use GitLab Agent for Kubernetes Remote Development Module. -- Integrate Remote Development with the UI and Web IDE. -- Improve security and usability. - -### High-level approach - -The nuts and bolts are being worked out at [Remote Development GA4K Architecture](https://gitlab.com/gitlab-org/remote-development/gitlab-remote-development-docs/-/blob/main/doc/architecture.md) to keep a SSoT. Once we have hammered out the details, we'll replace this section with the diagram in the above repository. - -### Iteration 0: [GitLab Agent for Kubernetes Remote Development Module (plumbing)](https://gitlab.com/groups/gitlab-org/-/epics/9138) - -#### Goals - -- Use the [GitLab Agent](../../../user/clusters/agent/index.md) integration. -- Create a workspace in a Kubernetes cluster based on a `devfile` in a public repository. -- Install the IDE and dependencies as defined. -- Report the status of the environment (via the terminal or through an endpoint). -- Connect to an IDE in the workspace. - -#### Requirements - -- Remote environment running on a Kubernetes cluster based on a `devfile` in a repo. - -These are **not** part of Iteration 0: - -- Authentication/authorization with GitLab and a user. -- Integration of Remote Development with the GitLab UI and Web IDE. -- Using GA4K instead of an Ingress controller. - -#### Assumptions - -- We will use [`devworkspace-operator` v0.17.0 (latest version)](https://github.com/devfile/devworkspace-operator/releases/tag/v0.17.0). A prerequisite is [`cert-manager`](https://github.com/devfile/devworkspace-operator#with-yaml-resources). -- We have an Ingress controller ([Ingress-NGINX](https://github.com/kubernetes/ingress-nginx)), which is accessible over the network. -- The initial server is stubbed. - -#### Success criteria - -- Using GA4K to communicate with the Kubernetes API from the `remote_dev` agent module. -- All calls to the Kubernetes API are done through GA4K. -- A workspace in a Kubernetes cluster created using DevWorkspace Operator. - -### Iteration 1: [Rails endpoints, authentication, and authorization](https://gitlab.com/groups/gitlab-org/-/epics/9323) - -#### Goals - -- Add endpoints in Rails to accept work from a user. -- Poll Rails for work from KAS. -- Add authentication and authorization to the workspaces created in the Kubernetes cluster. -- Extend the GA4K `remote_dev` agent module to accept more types of work (get details of a workspace, list workspaces for a user, etc). -- Build an editor injector for the GitLab fork of VS Code. - -#### Requirements - -- [GitLab Agent for Kubernetes Remote Development Module (plumbing)](https://gitlab.com/groups/gitlab-org/-/epics/9138) is complete. - -These are **not** part of Iteration 1: - -- Integration of Remote Development with the GitLab UI and Web IDE. -- Using GA4K instead of an Ingress controller. - -#### Assumptions - -- TBA - -#### Success criteria - -- Poll Rails for work from KAS. -- Rails endpoints to create/delete/get/list workspaces. -- All requests are correctly authenticated and authorized except where the user has requested the traffic to be public (for example, opening a server while developing and making it public). -- A user can create a workspace, start a server on that workspace, and have that traffic become private/internal/public. -- We are using the GitLab fork of VS Code as an editor. - -### Iteration 2: [Integrate Remote Development with the UI and Web IDE](https://gitlab.com/groups/gitlab-org/-/epics/9169) - -#### Goals - -- Allow users full control of their workspaces via the GitLab UI. - -#### Requirements - -- [GitLab Agent for Kubernetes Remote Development Module](https://gitlab.com/groups/gitlab-org/-/epics/9138). - -These are **not** part of Iteration 2: - -- Usability improvements -- Security improvements - -#### Success criteria - -- Be able to list/create/delete/stop/start/restart workspaces from the UI. -- Be able to create workspaces for the user in the Web IDE. -- Allow the Web IDE terminal to connect to different containers in the workspace. -- Configure DevWorkspace Operator for user-expected configuration (30-minute workspace timeout, a separate persistent volume for each workspace that is deleted when the workspace is deleted, etc.). - -### Iteration 3: [Improve security and usability](https://gitlab.com/groups/gitlab-org/-/epics/9170) - -#### Goals - -- Improve security and usability of our Remote Development solution. - -#### Requirements - -- [Integrate Remote Development with the UI and Web IDE](https://gitlab.com/groups/gitlab-org/-/epics/9169) is complete. - -#### Assumptions - -- We are allowing for internal feedback and closed/early customer feedback that can be iterated on. -- We have explored or are exploring the feasibility of using GA4K with Ingresses in [Solving Ingress problems for Remote Development](https://gitlab.com/gitlab-org/gitlab/-/issues/378998). -- We have explored or are exploring Kata containers for providing root access to workspace users in [Investigate Kata Containers / Firecracker / gVisor](https://gitlab.com/gitlab-org/gitlab/-/issues/367043). -- We have explored or are exploring how Ingress/Egress requests cannot be misused from [resources within or outside the cluster](https://gitlab.com/gitlab-org/remote-development/gitlab-remote-development-docs/-/blob/main/doc/securing-the-workspace.md) (security hardening). - -#### Success criteria - -Add options to: - -- Create different classes of workspaces (1gb-2cpu, 4gb-8cpu, etc.). -- Vertically scale up workspace resources. -- Inject secrets from a GitLab user/group/repository. -- Configure timeouts of workspaces at multiple levels. -- Allow users to expose endpoints in their workspace (for example, not allow anyone in the organization to expose any endpoint publicly). - ## Market analysis We have conducted a market analysis to understand the broader market and what others can offer us by way of open-source libraries, integrations, or partnership opportunities. We have broken down the effort into a set of issues where we investigate each potential competitor/pathway/partnership as a spike. @@ -283,13 +135,15 @@ We have conducted a market analysis to understand the broader market and what ot - [Market analysis](https://gitlab.com/groups/gitlab-org/-/epics/8131) - [YouTube results](https://www.youtube.com/playlist?list=PL05JrBw4t0KrRQhnSYRNh1s1mEUypx67-) -### Next Steps +### Implementation -While our spike proved fruitful, we have paused this investigation until we reach our goals in [Viable Maturity](https://gitlab.com/groups/gitlab-org/-/epics/9190). +- [Viable Maturity Epic](https://gitlab.com/groups/gitlab-org/-/epics/9190) to track progress. +- [Documentation](https://gitlab.com/gitlab-org/remote-development/gitlab-remote-development-docs) +explaining the architecture and implementation details. -## Che versus a custom-built solution +## Che vs. DevWorkspace Operatoor vs. Custom-Built Solution -After an investigation into using [Che](https://gitlab.com/gitlab-org/gitlab/-/issues/366052) as our backend to accelerate Remote Development, we ultimately opted to [write our own custom-built solution](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97449#note_1131215629). +After an investigation into using [Che](https://gitlab.com/gitlab-org/gitlab/-/issues/366052) as our backend to accelerate Remote Development, we ultimately opted to [write our own custom-built solution using DevWorkspace Operator](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97449#note_1131215629). Some advantages of us opting to write our own custom-built solution are: @@ -297,6 +151,10 @@ Some advantages of us opting to write our own custom-built solution are: - It is easier to add support for other configurations apart from `devfile` in the future if the need arises. - We have the ability to choose which tech stack to use (for example, instead of using Traefik which is used in Che, explore NGINX itself or use GitLab Agent for Kubernetes). +After writing our own custom-built solution using DevWorkspace Operator, +we decided to [remove the dependency on DevWorkspace Operator](https://gitlab.com/groups/gitlab-org/-/epics/9895) +and thus the transitive dependency of Cert Manager. + ## Links - [Remote Development presentation](https://docs.google.com/presentation/d/1XHH_ZilZPufQoWVWViv3evipI-BnAvRQrdvzlhBuumw/edit#slide=id.g131f2bb72e4_0_8) @@ -304,7 +162,7 @@ Some advantages of us opting to write our own custom-built solution are: - [Minimal Maturity epic](https://gitlab.com/groups/gitlab-org/-/epics/9189) - [Viable Maturity epic](https://gitlab.com/groups/gitlab-org/-/epics/9190) - [Complete Maturity epic](https://gitlab.com/groups/gitlab-org/-/epics/9191) -- [Bi-weekly sync](https://docs.google.com/document/d/1hWVvksIc7VzZjG-0iSlzBnLpyr-OjwBVCYMxsBB3h_E/edit#) +- [Remote Development sync](https://docs.google.com/document/d/1hWVvksIc7VzZjG-0iSlzBnLpyr-OjwBVCYMxsBB3h_E/edit#) - [Market analysis and architecture](https://gitlab.com/groups/gitlab-org/-/epics/8131) - [GA4K Architecture](https://gitlab.com/gitlab-org/remote-development/gitlab-remote-development-docs/-/blob/main/doc/architecture.md) - [BYO infrastructure](https://gitlab.com/groups/gitlab-org/-/epics/8290) diff --git a/doc/architecture/blueprints/runner_scaling/index.md b/doc/architecture/blueprints/runner_scaling/index.md index 53401d80e34..de1203843aa 100644 --- a/doc/architecture/blueprints/runner_scaling/index.md +++ b/doc/architecture/blueprints/runner_scaling/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::verify" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Next Runner Auto-scaling Architecture ## Summary @@ -214,12 +216,12 @@ sequence diagram.  -On the diagrams above we see that currently a GitLab Runner Manager runs on a +On the diagrams above we see that currently a runner manager runs on a machine that has access to a cloud provider's API. It is using Docker Machine to provision new Virtual Machines with Docker Engine installed and it configures the Docker daemon there to allow external authenticated requests. It stores credentials to such ephemeral Docker environments on disk. Once a -machine has been provisioned and made available for GitLab Runner Manager to +machine has been provisioned and made available for the runner manager to run builds, it is using one of the existing executors to run a user-provided script. In auto-scaling, this is typically done using the Docker executor. @@ -281,7 +283,7 @@ coupled with the VM lifecycle and job routing logic. Creating idle capacity happens as a side-effect of calling [`Acquire`](https://gitlab.com/gitlab-org/gitlab-runner/-/blob/267f40d871cd260dd063f7fbd36a921fedc62241/executors/docker/machine/provider.go#L449) on the `machineProvider` while binding a job to a VM. There is also no current abstraction for in-VM job execution. VM-specific -commands are generated by the Runner Manager using the [`GenerateShellScript`](https://gitlab.com/gitlab-org/gitlab-runner/-/blob/267f40d871cd260dd063f7fbd36a921fedc62241/common/build.go#L336) +commands are generated by the runner manager using the [`GenerateShellScript`](https://gitlab.com/gitlab-org/gitlab-runner/-/blob/267f40d871cd260dd063f7fbd36a921fedc62241/common/build.go#L336) function and [injected](https://gitlab.com/gitlab-org/gitlab-runner/-/blob/267f40d871cd260dd063f7fbd36a921fedc62241/common/build.go#L373) into the VM as the manager drives the job execution stages. diff --git a/doc/architecture/blueprints/runner_tokens/index.md b/doc/architecture/blueprints/runner_tokens/index.md index 359eadb18c5..0e70b97d2c7 100644 --- a/doc/architecture/blueprints/runner_tokens/index.md +++ b/doc/architecture/blueprints/runner_tokens/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::verify" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Next GitLab Runner Token Architecture ## Summary diff --git a/doc/architecture/blueprints/secret_detection/index.md b/doc/architecture/blueprints/secret_detection/index.md index 26551367a7c..fc97ca71d7f 100644 --- a/doc/architecture/blueprints/secret_detection/index.md +++ b/doc/architecture/blueprints/secret_detection/index.md @@ -1,5 +1,5 @@ --- -status: proposed +status: ongoing creation-date: "2022-11-25" authors: [ "@theoretick" ] coach: "@DylanGriffith" @@ -8,6 +8,8 @@ owning-stage: "~devops::secure" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Secret Detection as a platform-wide experience ## Summary @@ -24,7 +26,7 @@ job logs, and project management features such as issues, epics, and MRs. ### Goals -- Support asynchronous secret detection for: +- Support asynchronous secret detection for the following scan targets: - push events - issuable creation - issuable updates @@ -47,6 +49,24 @@ Scanned object types beyond the scope of this MVC include: - Snippets - Wikis +#### Management UI + +Development of an independent interface for managing secrets is out of scope +for this blueprint. Any detections will be managed using the existing +Vulnerability Management UI. + +Management of detected secrets will remain distinct from the +[Secret Management feature capability](../../../ci/secrets/index.md) as +"detected" secrets are categorically distinct from actively "managed" secrets. +When a detected secret is identified, it has already been compromised due to +their presence in the target object (that is a repository). Alternatively, managed +secrets should be stored with stricter standards for secure storage, including +encryption and masking when visible (such as job logs or in the UI). + +As a long-term priority we should consider unifying the management of the two +secret types however that work is out of scope for the current blueprints goals, +which remain focused on active detection. + ## Proposal To achieve scalable secret detection for a variety of domain objects a dedicated @@ -67,6 +87,7 @@ as self-managed instances. - Secure authentication to GitLab.com infrastructure - Performance of scanning against large blobs - Performance of scanning against volume of domain objects (such as push frequency) +- Queueing of scan requests ## Design and implementation details @@ -74,13 +95,13 @@ The critical paths as outlined under [goals above](#goals) cover two major objec types: Git blobs (corresponding to push events) and arbitrary text blobs. The detection flow for push events relies on subscribing to the PostReceive hook -and enqueueing Sidekiq requests to the `SecretScanningService`. The `SecretScanningService` +to enqueue Sidekiq requests to the `SecretScanningService`. The `SecretScanningService` service fetches enqueued refs, queries Gitaly for the ref blob contents, scans the commit contents, and notifies the Rails application when a secret is detected. See [Push event detection flow](#push-event-detection-flow) for sequence. The detection flow for arbitrary text blobs, such as issue comments, relies on -subscribing to `Notes::PostProcessService` (or equivalent service) and enqueueing +subscribing to `Notes::PostProcessService` (or equivalent service) to enqueue Sidekiq requests to the `SecretScanningService` to process the text blob by object type and primary key of domain object. The `SecretScanningService` service fetches the relevant text blob, scans the contents, and notifies the Rails application when a secret @@ -92,7 +113,7 @@ around scanning during streaming and the added complexity in buffering lookbacks for arbitrary trace chunks. In any case of detection, the Rails application manually creates a vulnerability -using the `Vulnerabilities::ManuallyCreateService` to surface the finding within the +using the `Vulnerabilities::ManuallyCreateService` to surface the finding in the existing Vulnerability Management UI. See [technical discovery](https://gitlab.com/gitlab-org/gitlab/-/issues/376716) @@ -115,7 +136,7 @@ Token types to identify in order of importance: ### Detection engine Our current secret detection offering utilizes [Gitleaks](https://github.com/zricethezav/gitleaks/) -for all secret scanning within pipeline contexts. By using its `--no-git` configuration +for all secret scanning in pipeline contexts. By using its `--no-git` configuration we can scan arbitrary text blobs outside of a repository context and continue to utilize it for non-pipeline scanning. @@ -123,6 +144,23 @@ Given our existing familiarity with the tool and its extensibility, it should remain our engine of choice. Changes to the detection engine are out of scope unless benchmarking unveils performance concerns. +Notable alternatives include high-performance regex engines such as [hyperscan](https://github.com/intel/hyperscan) or it's portable fork [vectorscan](https://github.com/VectorCamp/vectorscan). + +### High-level architecture + +The implementation of the secret scanning service is highly dependent on the outcomes of our benchmarking +and capacity planning against both GitLab.com and our +[Reference Architectures](../../../administration/reference_architectures/index.md). +As the scanning capability must be an on-by-default component of both our SaaS and self-managed +instances [the PoC](#iterations), the deployment characteristics must be considered to determine whether +this is a standalone component or executed as a subprocess of the existing Sidekiq worker fleet +(similar to the implementation of our Elasticsearch indexing service). + +Similarly, the scan target volume will require a robust and scalable enqueueing system to limit resource consumption. + +See [this thread](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/105142#note_1194863310) +for past discussion around scaling approaches. + ### Push event detection flow ```mermaid @@ -151,17 +189,20 @@ sequenceDiagram ## Iterations -1. Requirements definition for detection coverage and actions -1. PoC of secret scanning service - 1. gRPC commit retrieval from Gitaly - 1. blob scanning - 1. benchmarking of issuables, comments, job logs and blobs to gain confidence that the total costs will be viable -1. Implementation of secret scanning service MVC (targeting individual commits) -1. Security and readiness review -1. Deployment and monitoring -1. Implementation of secret scanning service MVC (targeting arbitrary text blobs) -1. Deployment and monitoring -1. High priority domain object rollout (priority `TBD`) - 1. Issuable comments - 1. Issuable bodies - 1. Job logs +- ✓ Define [requirements for detection coverage and actions](https://gitlab.com/gitlab-org/gitlab/-/issues/376716) +- ✓ Implement [Clientside detection of GitLab tokens in comments/issues](https://gitlab.com/gitlab-org/gitlab/-/issues/368434) +- PoC of secret scanning service + - Benchmarking of issuables, comments, job logs and blobs to gain confidence that the total costs will be viable + - Capacity planning for addition of service component to Reference Architectures headroom + - Service capabilities + - gRPC commit retrieval from Gitaly + - blob scanning +- Implementation of secret scanning service MVC (targeting individual commits) +- Security and readiness review +- Deployment and monitoring +- Implementation of secret scanning service MVC (targeting arbitrary text blobs) +- Deployment and monitoring +- High priority domain object rollout (priority `TBD`) + - Issuable comments + - Issuable bodies + - Job logs diff --git a/doc/architecture/blueprints/work_items/index.md b/doc/architecture/blueprints/work_items/index.md index 2c854ecea59..f067d9fab52 100644 --- a/doc/architecture/blueprints/work_items/index.md +++ b/doc/architecture/blueprints/work_items/index.md @@ -8,6 +8,8 @@ owning-stage: "~devops::plan" participating-stages: [] --- +<!-- vale gitlab.FutureTense = NO --> + # Work Items This document is a work-in-progress. Some aspects are not documented, though we expect to add them in the future. diff --git a/doc/architecture/index.md b/doc/architecture/index.md index 2467ba33fae..9fccd1c21d1 100644 --- a/doc/architecture/index.md +++ b/doc/architecture/index.md @@ -1,6 +1,5 @@ --- feedback: false -comments: false toc: false --- diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index d7c5b089116..c4d66cbb606 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -38,7 +38,7 @@ can't link to files outside it. - Subsequent jobs in later stages of the same pipeline can use artifacts. - Different projects cannot share artifacts. - Artifacts expire after 30 days by default. You can define a custom [expiration time](../yaml/index.md#artifactsexpire_in). -- The latest artifacts do not expire if [keep latest artifacts](../pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs) is enabled. +- The latest artifacts do not expire if [keep latest artifacts](../jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs) is enabled. - Use [dependencies](../yaml/index.md#dependencies) to control which jobs fetch the artifacts. ## Good caching practices @@ -65,7 +65,7 @@ For runners to work with caches efficiently, you must do one of the following: ## Use multiple caches > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32814) in GitLab 13.10. -> - [Feature Flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/321877), in GitLab 13.12. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/321877), in GitLab 13.12. You can have a maximum of four caches: @@ -331,8 +331,8 @@ before_script: test: script: - python setup.py test - - pip install flake8 - - flake8 . + - pip install ruff + - ruff --format=gitlab . ``` ### Cache Ruby dependencies diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md index bb3c8d537f3..7be12d0c0fd 100644 --- a/doc/ci/chatops/index.md +++ b/doc/ci/chatops/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: index, concepts, howto --- @@ -108,6 +108,6 @@ ls: ## Related topics -- Download the [official GitLab ChatOps icon](img/gitlab-chatops-icon.png). -- The GitLab team maintains a [repository of common ChatOps scripts](https://gitlab.com/gitlab-com/chatops) - they use to interact with GitLab.com. +- [The official GitLab ChatOps icon](img/gitlab-chatops-icon.png) +- [A repository of common ChatOps scripts](https://gitlab.com/gitlab-com/chatops) + that GitLab uses to interact with GitLab.com diff --git a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md index 71210467018..9ea5f76733a 100644 --- a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md +++ b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/cloud_deployment/heroku.md b/doc/ci/cloud_deployment/heroku.md index 7c15bf5b8cd..5cb6682debd 100644 --- a/doc/ci/cloud_deployment/heroku.md +++ b/doc/ci/cloud_deployment/heroku.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index 79684d9d627..ce03e9e3916 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: howto --- diff --git a/doc/ci/cloud_services/google_cloud/index.md b/doc/ci/cloud_services/google_cloud/index.md index bcb2681403a..719b97c53d8 100644 --- a/doc/ci/cloud_services/google_cloud/index.md +++ b/doc/ci/cloud_services/google_cloud/index.md @@ -6,8 +6,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Configure OpenID Connect with GCP Workload Identity Federation **(FREE)** -WARNING: -The `CI_JOB_JWT_V2` variable is under development [(alpha)](../../../policy/alpha-beta-support.md#alpha-features) and is not yet suitable for production use. +NOTE: +`CI_JOB_JWT_V2` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) +and is scheduled to be removed in GitLab 16.0. Use [ID tokens](../../yaml/index.md#id_tokens) instead. This tutorial demonstrates authenticating to Google Cloud from a GitLab CI/CD job using a JSON Web Token (JWT) token and Workload Identity Federation. This configuration diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md index 88e7d88e22a..d2d609196e4 100644 --- a/doc/ci/cloud_services/index.md +++ b/doc/ci/cloud_services/index.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - `CI_JOB_JWT` variable for reading secrets from Vault [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207125) in GitLab 12.10. > - `CI_JOB_JWT_V2` variable to support additional OIDC providers [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346737) in GitLab 14.7. -> - [ID tokens](../yaml/index.md) to support any OIDC provider, including HashiCorp Vault, [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356986) in GitLab 15.7. +> - [ID tokens](../yaml/index.md#id_tokens) to support any OIDC provider, including HashiCorp Vault, [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356986) in GitLab 15.7. GitLab CI/CD supports [OpenID Connect (OIDC)](https://openid.net/connect/faq/) to give your build and deployment jobs access to cloud credentials and services. @@ -19,6 +19,10 @@ in the CI/CD job allowing you to follow a scalable and least-privilege security In GitLab 15.6 and earlier, you must use `CI_JOB_JWT_V2` instead of an ID token, but it is not customizable. In GitLab 14.6 an earlier you must use the `CI_JOB_JWT`, which has limited support. +NOTE: +`CI_JOB_JWT` and `CI_JOB_JWT_V2` were [deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) +and are scheduled to be removed in GitLab 16.0. Use [ID tokens](../yaml/index.md#id_tokens) instead. + ## Requirements - Account on GitLab. @@ -50,61 +54,7 @@ as a starting point, and for more information about supply chain attacks, see ## How it works -Each job can be configured with ID tokens, which are provided as a CI/CD variable. These JWTs can be used to authenticate with the OIDC-supported cloud provider such as AWS, Azure, GCP, or Vault. - -The following fields are included in the JWT: - -| Field | When | Description | -| ----------------------- | ------ | ----------- | -| `aud` | Always | Specified in the [ID tokens](../yaml/index.md#id_tokens) configuration | -| `jti` | Always | Unique identifier for this token | -| `iss` | Always | Issuer, the domain of your GitLab instance | -| `iat` | Always | Issued at | -| `nbf` | Always | Not valid before | -| `exp` | Always | Expires at | -| `sub` | Always |`project_path:{group}/{project}:ref_type:{type}:ref:{branch_name}` | -| `namespace_id` | Always | Use this to scope to group or user level namespace by ID | -| `namespace_path` | Always | Use this to scope to group or user level namespace by path | -| `project_id` | Always | Use this to scope to project by ID | -| `project_path` | Always | Use this to scope to project by path | -| `user_id` | Always | ID of the user executing the job | -| `user_login` | Always | Username of the user executing the job | -| `user_email` | Always | Email of the user executing the job | -| `pipeline_id` | Always | ID of this pipeline | -| `pipeline_source` | Always | [Pipeline source](../jobs/job_control.md#common-if-clauses-for-rules) | -| `job_id` | Always | ID of this job | -| `ref` | Always | Git ref for this job | -| `ref_type` | Always | Git ref type, either `branch` or `tag` | -| `ref_protected` | Always | `true` if this Git ref is protected, `false` otherwise | -| `environment` | Job is creating a deployment | Environment this job deploys to ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9) | -| `environment_protected` | Job is creating a deployment |`true` if deployed environment is protected, `false` otherwise ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9) | - -```json -{ - "jti": "c82eeb0c-5c6f-4a33-abf5-4c474b92b558", - "iss": "https://gitlab.example.com", - "aud": "https://vault.example.com", - "iat": 1585710286, - "nbf": 1585798372, - "exp": 1585713886, - "sub": "project_path:mygroup/myproject:ref_type:branch:ref:main", - "namespace_id": "1", - "namespace_path": "mygroup", - "project_id": "22", - "project_path": "mygroup/myproject", - "user_id": "42", - "user_login": "myuser", - "user_email": "myuser@example.com", - "pipeline_id": "1212", - "pipeline_source": "web", - "job_id": "1212", - "ref": "auto-deploy-2020-04-01", - "ref_type": "branch", - "ref_protected": "true", - "environment": "production", - "environment_protected": "true" -} -``` +Each job can be configured with ID tokens, which are provided as a CI/CD variable containing the [token payload](../secrets/id_token_authentication.md#token-payload). These JWTs can be used to authenticate with the OIDC-supported cloud provider such as AWS, Azure, GCP, or Vault. ### Authorization workflow diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index 7c741ef3842..f0384fb29c7 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -81,7 +81,7 @@ are certain use cases that you may need to work around. For more information, ch ## Needs Visualization -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](../../policy/alpha-beta-support.md#beta-features). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](../../policy/alpha-beta-support.md#beta). > - It became a [standard feature](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38517) in 13.3. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52208) in GitLab 13.9. diff --git a/doc/ci/docker/authenticate_registry.md b/doc/ci/docker/authenticate_registry.md new file mode 100644 index 00000000000..224d0cdf7aa --- /dev/null +++ b/doc/ci/docker/authenticate_registry.md @@ -0,0 +1,144 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: concepts, howto +--- + +# Authenticate with registry in Docker-in-Docker + +When you use Docker-in-Docker, the +[standard authentication methods](using_docker_images.md#access-an-image-from-a-private-container-registry) +do not work, because a fresh Docker daemon is started with the service. + +## Option 1: Run `docker login` + +In [`before_script`](../yaml/index.md#before_script), run `docker +login`: + +```yaml +image: docker:20.10.16 + +variables: + DOCKER_TLS_CERTDIR: "/certs" + +services: + - docker:20.10.16-dind + +build: + stage: build + before_script: + - echo "$DOCKER_REGISTRY_PASS" | docker login $DOCKER_REGISTRY --username $DOCKER_REGISTRY_USER --password-stdin + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests +``` + +To sign in to Docker Hub, leave `$DOCKER_REGISTRY` +empty or remove it. + +## Option 2: Mount `~/.docker/config.json` on each job + +If you are an administrator for GitLab Runner, you can mount a file +with the authentication configuration to `~/.docker/config.json`. +Then every job that the runner picks up is already authenticated. If you +are using the official `docker:20.10.16` image, the home directory is +under `/root`. + +If you mount the configuration file, any `docker` command +that modifies the `~/.docker/config.json` fails. For example, `docker login` +fails, because the file is mounted as read-only. Do not change it from +read-only, because this causes problems. + +Here is an example of `/opt/.docker/config.json` that follows the +[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determine-your-docker_auth_config-data) +documentation: + +```json +{ + "auths": { + "https://index.docker.io/v1/": { + "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" + } + } +} +``` + +### Docker + +Update the +[volume mounts](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#volumes-in-the-runnersdocker-section) +to include the file. + +```toml +[[runners]] + ... + executor = "docker" + [runners.docker] + ... + privileged = true + volumes = ["/opt/.docker/config.json:/root/.docker/config.json:ro"] +``` + +### Kubernetes + +Create a [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) with the content +of this file. You can do this with a command like: + +```shell +kubectl create configmap docker-client-config --namespace gitlab-runner --from-file /opt/.docker/config.json +``` + +Update the [volume mounts](https://docs.gitlab.com/runner/executors/kubernetes.html#using-volumes) +to include the file. + +```toml +[[runners]] + ... + executor = "kubernetes" + [runners.kubernetes] + image = "alpine:3.12" + privileged = true + [[runners.kubernetes.volumes.config_map]] + name = "docker-client-config" + mount_path = "/root/.docker/config.json" + # If you are running GitLab Runner 13.5 + # or lower you can remove this + sub_path = "config.json" +``` + +## Option 3: Use `DOCKER_AUTH_CONFIG` + +If you already have +[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determine-your-docker_auth_config-data) +defined, you can use the variable and save it in +`~/.docker/config.json`. + +You can define this authentication in several ways: + +- In [`pre_build_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) + in the runner configuration file. +- In [`before_script`](../yaml/index.md#before_script). +- In [`script`](../yaml/index.md#script). + +The following example shows [`before_script`](../yaml/index.md#before_script). +The same commands apply for any solution you implement. + +```yaml +image: docker:20.10.16 + +variables: + DOCKER_TLS_CERTDIR: "/certs" + +services: + - docker:20.10.16-dind + +build: + stage: build + before_script: + - mkdir -p $HOME/.docker + - echo $DOCKER_AUTH_CONFIG > $HOME/.docker/config.json + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests +``` diff --git a/doc/ci/docker/docker_layer_caching.md b/doc/ci/docker/docker_layer_caching.md new file mode 100644 index 00000000000..145ead9a212 --- /dev/null +++ b/doc/ci/docker/docker_layer_caching.md @@ -0,0 +1,61 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: concepts, howto +--- + +# Make Docker-in-Docker builds faster with Docker layer caching + +When using Docker-in-Docker, Docker downloads all layers of your image every +time you create a build. Recent versions of Docker (Docker 1.13 and later) can +use a pre-existing image as a cache during the `docker build` step. This significantly +accelerates the build process. + +## How Docker caching works + +When running `docker build`, each command in `Dockerfile` creates a layer. +These layers are retained as a cache and can be reused if there have been no changes. Change in one layer causes the recreation of all subsequent layers. + +To specify a tagged image to be used as a cache source for the `docker build` +command, use the `--cache-from` argument. Multiple images can be specified +as a cache source by using multiple `--cache-from` arguments. Any image that's used +with the `--cache-from` argument must be pulled +(using `docker pull`) before it can be used as a cache source. + +## Docker caching example + +This example `.gitlab-ci.yml` file shows how to use Docker caching: + +```yaml +image: docker:20.10.16 + +services: + - docker:20.10.16-dind + +variables: + # Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled + DOCKER_HOST: tcp://docker:2376 + DOCKER_TLS_CERTDIR: "/certs" + +before_script: + - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY + +build: + stage: build + script: + - docker pull $CI_REGISTRY_IMAGE:latest || true + - docker build --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest . + - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA + - docker push $CI_REGISTRY_IMAGE:latest +``` + +In the `script` section for the `build` stage: + +1. The first command tries to pull the image from the registry so that it can be + used as a cache for the `docker build` command. +1. The second command builds a Docker image by using the pulled image as a + cache (see the `--cache-from $CI_REGISTRY_IMAGE:latest` argument) if + available, and tags it. +1. The last two commands push the tagged Docker images to the container registry + so that they can also be used as cache for subsequent builds. diff --git a/doc/ci/docker/index.md b/doc/ci/docker/index.md index e976700c160..162d67286ad 100644 --- a/doc/ci/docker/index.md +++ b/doc/ci/docker/index.md @@ -2,7 +2,6 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false type: index --- diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 674f9b6035a..1b2b1aef3b6 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -570,196 +570,11 @@ build: ## Authenticate with registry in Docker-in-Docker -When you use Docker-in-Docker, the -[standard authentication methods](using_docker_images.md#access-an-image-from-a-private-container-registry) -do not work, because a fresh Docker daemon is started with the service. - -### Option 1: Run `docker login` - -In [`before_script`](../yaml/index.md#before_script), run `docker -login`: - -```yaml -image: docker:20.10.16 - -variables: - DOCKER_TLS_CERTDIR: "/certs" - -services: - - docker:20.10.16-dind - -build: - stage: build - before_script: - - echo "$DOCKER_REGISTRY_PASS" | docker login $DOCKER_REGISTRY --username $DOCKER_REGISTRY_USER --password-stdin - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests -``` - -To sign in to Docker Hub, leave `$DOCKER_REGISTRY` -empty or remove it. - -### Option 2: Mount `~/.docker/config.json` on each job - -If you are an administrator for GitLab Runner, you can mount a file -with the authentication configuration to `~/.docker/config.json`. -Then every job that the runner picks up is already authenticated. If you -are using the official `docker:20.10.16` image, the home directory is -under `/root`. - -If you mount the configuration file, any `docker` command -that modifies the `~/.docker/config.json` fails. For example, `docker login` -fails, because the file is mounted as read-only. Do not change it from -read-only, because this causes problems. - -Here is an example of `/opt/.docker/config.json` that follows the -[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determine-your-docker_auth_config-data) -documentation: - -```json -{ - "auths": { - "https://index.docker.io/v1/": { - "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" - } - } -} -``` - -#### Docker - -Update the -[volume mounts](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#volumes-in-the-runnersdocker-section) -to include the file. - -```toml -[[runners]] - ... - executor = "docker" - [runners.docker] - ... - privileged = true - volumes = ["/opt/.docker/config.json:/root/.docker/config.json:ro"] -``` - -#### Kubernetes - -Create a [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) with the content -of this file. You can do this with a command like: - -```shell -kubectl create configmap docker-client-config --namespace gitlab-runner --from-file /opt/.docker/config.json -``` - -Update the [volume mounts](https://docs.gitlab.com/runner/executors/kubernetes.html#using-volumes) -to include the file. - -```toml -[[runners]] - ... - executor = "kubernetes" - [runners.kubernetes] - image = "alpine:3.12" - privileged = true - [[runners.kubernetes.volumes.config_map]] - name = "docker-client-config" - mount_path = "/root/.docker/config.json" - # If you are running GitLab Runner 13.5 - # or lower you can remove this - sub_path = "config.json" -``` - -### Option 3: Use `DOCKER_AUTH_CONFIG` - -If you already have -[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determine-your-docker_auth_config-data) -defined, you can use the variable and save it in -`~/.docker/config.json`. - -You can define this authentication in several ways: - -- In [`pre_build_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) - in the runner configuration file. -- In [`before_script`](../yaml/index.md#before_script). -- In [`script`](../yaml/index.md#script). - -The following example shows [`before_script`](../yaml/index.md#before_script). -The same commands apply for any solution you implement. - -```yaml -image: docker:20.10.16 - -variables: - DOCKER_TLS_CERTDIR: "/certs" - -services: - - docker:20.10.16-dind - -build: - stage: build - before_script: - - mkdir -p $HOME/.docker - - echo $DOCKER_AUTH_CONFIG > $HOME/.docker/config.json - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests -``` +When you use Docker-in-Docker, the [standard authentication methods](using_docker_images.md#access-an-image-from-a-private-container-registry) do not work, because a fresh Docker daemon is started with the service. You should [authenticate with registry](authenticate_registry.md). ## Make Docker-in-Docker builds faster with Docker layer caching -When using Docker-in-Docker, Docker downloads all layers of your image every -time you create a build. Recent versions of Docker (Docker 1.13 and later) can -use a pre-existing image as a cache during the `docker build` step. This significantly -accelerates the build process. - -### How Docker caching works - -When running `docker build`, each command in `Dockerfile` creates a layer. -These layers are retained as a cache and can be reused if there have been no changes. Change in one layer causes the recreation of all subsequent layers. - -To specify a tagged image to be used as a cache source for the `docker build` -command, use the `--cache-from` argument. Multiple images can be specified -as a cache source by using multiple `--cache-from` arguments. Any image that's used -with the `--cache-from` argument must be pulled -(using `docker pull`) before it can be used as a cache source. - -### Docker caching example - -This example `.gitlab-ci.yml` file shows how to use Docker caching: - -```yaml -image: docker:20.10.16 - -services: - - docker:20.10.16-dind - -variables: - # Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled - DOCKER_HOST: tcp://docker:2376 - DOCKER_TLS_CERTDIR: "/certs" - -before_script: - - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - -build: - stage: build - script: - - docker pull $CI_REGISTRY_IMAGE:latest || true - - docker build --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest . - - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA - - docker push $CI_REGISTRY_IMAGE:latest -``` - -In the `script` section for the `build` stage: - -1. The first command tries to pull the image from the registry so that it can be - used as a cache for the `docker build` command. -1. The second command builds a Docker image by using the pulled image as a - cache (see the `--cache-from $CI_REGISTRY_IMAGE:latest` argument) if - available, and tags it. -1. The last two commands push the tagged Docker images to the container registry - so that they can also be used as cache for subsequent builds. +When using Docker-in-Docker, Docker downloads all layers of your image every time you create a build. You can [make your builds faster with Docker layer caching](docker_layer_caching.md). ## Use the OverlayFS driver diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index b19e65ee396..ca33fcf09bd 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -113,7 +113,7 @@ build: You can build [multi-arch images](https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/) inside a container by using [`manifest-tool`](https://github.com/estesp/manifest-tool). -For a detailed guide on how to build a multi-arch image, read [Building a multi-arch container image in unprivileged containers](https://ingenuity.siemens.com/2022/07/building-a-multi-arch-container-image-in-unprivileged-containers/). +For a detailed guide on how to build a multi-arch image, read [Building a multi-arch container image in unprivileged containers](https://blog.siemens.com/2022/07/building-a-multi-arch-container-image-in-unprivileged-containers/). ## Using a registry with a custom certificate diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index c4f685b18ec..2e40a80c307 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: Require approvals prior to deploying to a Protected Environment --- @@ -62,9 +62,11 @@ co-exist and multiple approval rules takes the precedence over the unified appro #### Unified approval setting -There are two ways to configure approvals for a protected environment: +> - UI configuration [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/378447) in GitLab +> 15.11. + +To configure approvals for a protected environment: -- Using the [UI](protected_environments.md#protecting-environments), set the **Required approvals** field to 1 or more. - Using the [REST API](../../api/protected_environments.md#protect-a-single-environment), set the `required_approval_count` field to 1 or more. @@ -87,14 +89,18 @@ Maintainer role. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) in GitLab 14.10 with a flag named `deployment_approval_rules`. Disabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) in GitLab 15.0. [Feature flag `deployment_approval_rules`](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) removed. +> - UI configuration [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378445) in GitLab 15.11. - Using the [REST API](../../api/group_protected_environments.md#protect-a-single-environment). - `deploy_access_levels` represents which entity can execute the deployment job. - `approval_rules` represents which entity can approve the deployment job. +- Using the [UI](protected_environments.md#protecting-environments). + - **Allowed to deploy** sets which entities can execute the deployment job. + - **Approvers** sets which entities can approve the deployment job. After this is configured, all jobs deploying to this environment automatically go into a blocked state and wait for approvals before running. Ensure that the number of required approvals is less than the number of users allowed to deploy. -Example: +A configuration that uses the REST API might look like: ```shell curl --header 'Content-Type: application/json' --request POST \ diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 327118c8a86..0cfaf2b3fe9 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md index 2ee0dc2ee11..fd2c85819fe 100644 --- a/doc/ci/environments/environments_dashboard.md +++ b/doc/ci/environments/environments_dashboard.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/ci/environments/external_deployment_tools.md b/doc/ci/environments/external_deployment_tools.md index 89a1f8565a9..37c4cf05f13 100644 --- a/doc/ci/environments/external_deployment_tools.md +++ b/doc/ci/environments/external_deployment_tools.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- @@ -14,7 +14,7 @@ GitLab can receive deployment events from these external tools and allows you to For example, the following features are available by setting up tracking: - [See when an merge request has been deployed, and to which environment](../../user/project/merge_requests/widgets.md#post-merge-pipeline-status). -- [Filter merge requests by environment or deployment date](../../user/project/merge_requests/index.md#filter-merge-requests-by-environment-or-deployment-date). +- [Filter merge requests by environment or deployment date](../../user/project/merge_requests/index.md#by-environment-or-deployment-date). - [DevOps Research and Assessment (DORA) metrics](../../user/analytics/dora_metrics.md). - [View environments and deployments](index.md#view-environments-and-deployments). - [Track newly included merge requests per deployment](index.md#track-newly-included-merge-requests-per-deployment). diff --git a/doc/ci/environments/img/environment_auto_stop_v13_10.png b/doc/ci/environments/img/environment_auto_stop_v13_10.png Binary files differdeleted file mode 100644 index 50f268da27f..00000000000 --- a/doc/ci/environments/img/environment_auto_stop_v13_10.png +++ /dev/null diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md index 8df2d8ead6c..c032ae3be60 100644 --- a/doc/ci/environments/incremental_rollouts.md +++ b/doc/ci/environments/incremental_rollouts.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index c9b212c2915..577cbce2943 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -1,9 +1,8 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference -disqus_identifier: 'https://docs.gitlab.com/ee/ci/environments.html' --- # Environments and deployments **(FREE)** @@ -168,7 +167,7 @@ You cannot rename an environment by using the UI, and the API method was depreca To achieve the same result as renaming an environment: -1. [Stop the existing environment](#stop-an-environment-through-the-ui). +1. [Stop the existing environment](#stop-an-environment-by-using-the-ui). 1. [Delete the existing environment](#delete-an-environment). 1. [Create a new environment](#create-a-static-environment) with the desired name. @@ -453,28 +452,31 @@ For example: With GitLab [Route Maps](../review_apps/index.md#route-maps), you can go directly from source files to public pages in the environment set for Review Apps. -### Stop an environment +### Stopping an environment -When you stop an environment: +Stopping an environment means its deployments are not accessible on the target server. You must stop +an environment before it can be deleted. -- On the **Environments** page, it moves from the list of **Available** environments - to the list of **Stopped** environments. -- An [`on_stop` action](../yaml/index.md#environmenton_stop), if defined, is executed. - -There are multiple ways to clean up [dynamic environments](#create-a-dynamic-environment): - -- If you use [merge request pipelines](../pipelines/merge_request_pipelines.md), GitLab stops an environment [when a merge request is merged or closed](#stop-an-environment-when-a-merge-request-is-merged-or-closed). -- If you do _NOT_ use [merge request pipelines](../pipelines/merge_request_pipelines.md), GitLab stops an environment [when the associated feature branch is deleted](#stop-an-environment-when-a-branch-is-deleted). -- If you set [an expiry period to an environment](../yaml/index.md#environmentauto_stop_in), GitLab stops an environment [when it's expired](#stop-an-environment-after-a-certain-time-period). - -To stop stale environments, you can [use the API](../../api/environments.md#stop-stale-environments). +If the environment has an [`on_stop` action](../yaml/index.md#environmenton_stop) defined, it's +executed to stop the environment. #### Stop an environment when a branch is deleted You can configure environments to stop when a branch is deleted. -The following example shows a `deploy_review` job that calls a `stop_review` job -to clean up and stop the environment. +In the following example, a `deploy_review` job calls a `stop_review` job to clean up and stop the +environment. + +- Both jobs must have the same [`rules`](../yaml/index.md#rules) + or [`only/except`](../yaml/index.md#only--except) configuration. Otherwise, + the `stop_review` job might not be included in all pipelines that include the + `deploy_review` job, and you cannot trigger `action: stop` to stop the environment automatically. +- The job with [`action: stop` might not run](#the-job-with-action-stop-doesnt-run) + if it's in a later stage than the job that started the environment. +- If you can't use [merge request pipelines](../pipelines/merge_request_pipelines.md), + set the [`GIT_STRATEGY`](../runners/configure_runners.md#git-strategy) to `none` in the + `stop_review` job. Then the [runner](https://docs.gitlab.com/runner/) doesn't + try to check out the code after the branch is deleted. ```yaml deploy_review: @@ -496,30 +498,13 @@ stop_review: when: manual ``` -Both jobs must have the same [`rules`](../yaml/index.md#rules) -or [`only/except`](../yaml/index.md#only--except) configuration. Otherwise, -the `stop_review` job might not be included in all pipelines that include the -`deploy_review` job, and you cannot trigger `action: stop` to stop the environment automatically. - -The job with [`action: stop` might not run](#the-job-with-action-stop-doesnt-run) -if it's in a later stage than the job that started the environment. - -If you can't use [merge request pipelines](../pipelines/merge_request_pipelines.md), -set the [`GIT_STRATEGY`](../runners/configure_runners.md#git-strategy) to `none` in the -`stop_review` job. Then the [runner](https://docs.gitlab.com/runner/) doesn't -try to check out the code after the branch is deleted. - -Read more in the [`.gitlab-ci.yml` reference](../yaml/index.md#environmenton_stop). - #### Stop an environment when a merge request is merged or closed -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/60885) in GitLab 11.10. - -You can configure environments to stop when a merge request is merged or closed. -This stop trigger is automatically enabled when you use [merge request pipelines](../pipelines/merge_request_pipelines.md). +When you use the [merge request pipelines](../pipelines/merge_request_pipelines.md) configuration, +the `stop` trigger is automatically enabled. -The following example shows a `deploy_review` job that calls a `stop_review` job -to clean up and stop the environment. +In the following example, the `deploy_review` job calls a `stop_review` job to clean up and stop +the environment. ```yaml deploy_review: @@ -544,38 +529,35 @@ stop_review: when: manual ``` -#### Stop an environment when another job is finished - -You can set an environment to stop when another job is finished. - -In your `.gitlab-ci.yml` file, specify in the [`on_stop`](../yaml/index.md#environmenton_stop) -keyword the name of the job that stops the environment. +#### Run a pipeline job when environment is stopped -The following example shows a `review_app` job that calls a `stop_review_app` job after the first -job is finished. The `stop_review_app` is triggered based on what is defined under `when`. In this -case, it is set to `manual`, so it needs a -[manual action](../jobs/job_control.md#create-a-job-that-must-be-run-manually) -from the GitLab UI to run. +You can specify a job to run when an environment is stopped. -Both jobs must have the same rules or only/except configuration. -In this example, if the configuration is not identical: +Prerequisites: -- The `stop_review_app` job might not be included in all pipelines that include the `review_app` job. -- It is not possible to trigger the `action: stop` to stop the environment automatically. +- Both jobs must have the same rules or only/except configuration. +- The `stop_review_app` job **must** have the following keywords defined: + - `when`, defined at either: + - [The job level](../yaml/index.md#when). + - [In a rules clause](../yaml/index.md#rules). If you use `rules` and `when: manual`, you should + also set [`allow_failure: true`](../yaml/index.md#allow_failure) so the pipeline can complete + even if the job doesn't run. + - `environment:name` + - `environment:action` -Also in the example, `GIT_STRATEGY` is set to `none`. If the -`stop_review_app` job is [automatically triggered](../environments/index.md#stop-an-environment), -the runner doesn't try to check out the code after the branch is deleted. +In your `.gitlab-ci.yml` file, specify in the [`on_stop`](../yaml/index.md#environmenton_stop) +keyword the name of the job that stops the environment. -The `stop_review_app` job **must** have the following keywords defined: +In the following example: -- `when`, defined at either: - - [The job level](../yaml/index.md#when). - - [In a rules clause](../yaml/index.md#rules). If you use `rules` and `when: manual`, you should - also set [`allow_failure: true`](../yaml/index.md#allow_failure) so the pipeline can complete - even if the job doesn't run. -- `environment:name` -- `environment:action` +- A `review_app` job calls a `stop_review_app` job after the first job is finished. +- The `stop_review_app` is triggered based on what is defined under `when`. In this + case, it is set to `manual`, so it needs a + [manual action](../jobs/job_control.md#create-a-job-that-must-be-run-manually) + from the GitLab UI to run. +- The `GIT_STRATEGY` is set to `none`. If the `stop_review_app` job is + [automatically triggered](../environments/index.md#stopping-an-environment), + the runner doesn't try to check out the code after the branch is deleted. ```yaml review_app: @@ -599,21 +581,23 @@ stop_review_app: #### Stop an environment after a certain time period -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20956) in GitLab 12.8. +You can set an environment to stop automatically after a certain time period. -You can set environments to stop automatically after a certain time period. +NOTE: +Due to resource limitations, a background worker for stopping environments runs only once every +hour. This means that environments may not be stopped after the exact time period specified, but are +instead stopped when the background worker detects expired environments. In your `.gitlab-ci.yml` file, specify the [`environment:auto_stop_in`](../yaml/index.md#environmentauto_stop_in) -keyword. You can specify a human-friendly date as the value, such as `1 hour and 30 minutes` or `1 day`. +keyword. Specify the time period in natural language, such as `1 hour and 30 minutes` or `1 day`. After the time period passes, GitLab automatically triggers a job to stop the environment. -Due to resource limitations, a background worker for stopping environments only runs once -every hour. This means that environments aren't stopped at the exact timestamp specified, but are -instead stopped when the hourly cron worker detects expired environments. +In the following example: -In the following example, each merge request creates a Review App environment. -Each push triggers the `review_app` job and an environment named `review/your-branch-name` -is created or updated. The environment runs until `stop_review_app` is executed: +- Each commit on a merge request triggers a `review_app` job that deploys the latest change to the + environment and resets its expiry period. +- If the environment is inactive for more than a week, GitLab automatically triggers the + `stop_review_app` job to stop the environment. ```yaml review_app: @@ -635,13 +619,33 @@ stop_review_app: when: manual ``` -As long as the merge request is active and keeps getting new commits, -the Review App doesn't stop. Developers don't need to worry about -re-initiating Review App. +##### View an environment's scheduled stop date and time + +When a environment has been [scheduled to stop after a specified time period](#stop-an-environment-after-a-certain-time-period), +you can view its expiration date and time. + +To view an environment's expiration date and time: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. +1. Select the name of the environment. + +The expiration date and time is displayed in the upper-left corner, next to the environment's name. + +##### Override a environment's scheduled stop date and time + +When a environment has been [scheduled to stop after a specified time period](#stop-an-environment-after-a-certain-time-period), +you can override its expiration. -Because `stop_review_app` is set to `auto_stop_in: 1 week`, -if a merge request is inactive for more than a week, -GitLab automatically triggers the `stop_review_app` job to stop the environment. +To override an environment's expiration: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. +1. Select the deployment name. +1. in the upper-right corner, select the thumbtack (**{thumbtack}**). + +The `auto_stop_in` setting is overridden and the environment remains active until it's stopped +manually. #### Stop an environment without running the `on_stop` action @@ -653,7 +657,7 @@ To stop an environment without running the defined `on_stop` action, execute the [Stop an environment API](../../api/environments.md#stop-an-environment) with the parameter `force=true`. -#### Stop an environment through the UI +#### Stop an environment by using the UI NOTE: To trigger an `on_stop` action and manually stop an environment from the @@ -672,15 +676,20 @@ To stop an environment in the GitLab UI: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22456) in GitLab 14.10 [with a flag](../../administration/feature_flags.md) named `environment_multiple_stop_actions`. Disabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/358911) in GitLab 15.0. [Feature flag `environment_multiple_stop_actions`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86685) removed. -This feature is useful when you need to perform multiple **parallel** stop actions on an environment. +To configure multiple **parallel** stop actions on an environment, specify the +[`on_stop`](../yaml/index.md#environmenton_stop) keyword across multiple +[deployment jobs](../jobs/index.md#deployment-jobs) for the same `environment`, as defined in the +`.gitlab-ci.yml` file. + +When an environment is stopped, the matching `on_stop` actions from only successful deployment jobs are run in parallel, in no particular order. -To configure multiple stop actions on an environment, specify the [`on_stop`](../yaml/index.md#environmenton_stop) -keyword across multiple [deployment jobs](../jobs/index.md#deployment-jobs) for the same `environment`, as defined in the `.gitlab-ci.yml` file. +In the following example, for the `test` environment there are two deployment jobs: -When an environment is stopped, the matching `on_stop` actions from *successful deployment jobs* alone are run in parallel in no particular order. +- `deploy-to-cloud-a` +- `deploy-to-cloud-b` -In the following example, for the `test` environment there are two deployment jobs `deploy-to-cloud-a` -and `deploy-to-cloud-b`. +When the environment is stopped, the system runs `on_stop` actions `teardown-cloud-a` and +`teardown-cloud-b` in parallel. ```yaml deploy-to-cloud-a: @@ -710,32 +719,6 @@ teardown-cloud-b: when: manual ``` -When the environment is stopped, the system runs `on_stop` actions -`teardown-cloud-a` and `teardown-cloud-b` in parallel. - -#### View a deployment's scheduled stop time - -You can view a deployment's expiration date in the GitLab UI. - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. -1. Select the name of the deployment. - -In the upper-left corner, next to the environment name, the expiration date is displayed. - -#### Override a deployment's scheduled stop time - -You can manually override a deployment's expiration date. - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. -1. Select the deployment name. -1. in the upper-right corner, select the thumbtack (**{thumbtack}**). - - - -The `auto_stop_in` setting is overwritten and the environment remains active until it's stopped manually. - ### Delete an environment Delete an environment when you want to remove it and all its deployments. @@ -743,7 +726,7 @@ Delete an environment when you want to remove it and all its deployments. Prerequisites: - You must have at least the Developer role. -- You must [stop](#stop-an-environment) the environment before it can be deleted. +- You must [stop](#stopping-an-environment) the environment before it can be deleted. To delete an environment: diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index c81a5ab1378..f752e2179df 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -42,9 +42,22 @@ To protect an environment: - You can select groups that are already associated with the project only. - Users must have at least the Developer role to appear in the **Allowed to deploy** list. -1. In the **Required approvals** list, select the number of approvals required to deploy to this environment. - - Ensure that this number is less than the number of users allowed to deploy. +1. In the **Approvers** list, select the role, users, or groups you + want to give deploy access to. Keep in mind that: + + - There are two roles to choose from: + - **Maintainers**: Allows access to all of the project's users with the Maintainer role. + - **Developers**: Allows access to all of the project's users with the Maintainer and Developer role. + - You can select groups that are already associated with the project only. + - Users must have at least the Developer role to appear in + the **Allowed to deploy** list. + +1. In the **Approval rules** section: + + - Ensure that this number is less than or equal to the number of members in + the rule. - See [Deployment Approvals](deployment_approvals.md) for more information about this feature. + 1. Select **Protect**. The protected environment now appears in the list of protected environments. @@ -124,7 +137,7 @@ access to a protected environment through any of these methods: If the user also has push or merge access to the branch deployed on production, they have the following privileges: -- [Stop an environment](index.md#stop-an-environment). +- [Stop an environment](index.md#stopping-an-environment). - [Delete an environment](index.md#delete-an-environment). - [Create an environment terminal](index.md#web-terminals-deprecated). diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index 6295c131fb9..f59bb8ed931 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -5,15 +5,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Authenticating and reading secrets with HashiCorp Vault **(PREMIUM)** +# Authenticating and reading secrets with HashiCorp Vault (Deprecated) **(PREMIUM)** This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD. NOTE: -[GitLab Premium](https://about.gitlab.com/pricing/) supports read access to a -HashiCorp Vault, and enables you to -[use Vault secrets in a CI job](../../secrets/index.md#use-vault-secrets-in-a-ci-job). -For more information, see [Using external secrets in CI](../../secrets/index.md). +Authenticating with HashiCorp Vault by using `CI_JOB_JWT` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) +and the token is scheduled to be removed in GitLab 16.0. Use [ID tokens to authenticate with HashiCorp Vault](../../secrets/id_token_authentication.md#automatic-id-token-authentication-with-hashicorp-vault) instead. ## Requirements diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 81e70e5cf82..20083413f66 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/deployment/index.md b/doc/ci/examples/deployment/index.md index 74b4581af99..7499bf042f7 100644 --- a/doc/ci/examples/deployment/index.md +++ b/doc/ci/examples/deployment/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/index.md b/doc/ci/examples/index.md index b05c872d624..f2871e50617 100644 --- a/doc/ci/examples/index.md +++ b/doc/ci/examples/index.md @@ -2,7 +2,6 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false type: index --- diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index d56f14617db..cf12943d279 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -2,7 +2,6 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/articles/laravel_with_gitlab_and_envoy/index.html' author: Mehran Rasulian author_gitlab: mehranrasulian --- diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index 0b4b2054610..2146d76e4d6 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -168,3 +168,18 @@ Then, install the module: ```shell npm install --save @gitlab-examples/semantic-release-npm ``` + +## Troubleshooting + +### Deleted Git tags reappear + +A [Git tag](../../user/project/repository/tags/index.md) deleted from the repository +can sometimes be recreated by `semantic-release` when GitLab runners use a cached +version of the repository. If the job runs on a runner with a cached repository that +still has the tag, `semantic-release` recreates the tag in the main repository. + +To avoid this behavior, you can either: + +- Configure the runner with [`GIT_STRATEGY: clone`](../runners/configure_runners.md#git-strategy). +- Include the [`git fetch --prune-tags` command](https://git-scm.com/docs/git-fetch#Documentation/git-fetch.txt---prune-tags) + in your CI/CD script. diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 07ba3d8f916..31cb6bc9946 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -14,9 +14,13 @@ repository into your project and keep your commits separate. ## Configure the `.gitmodules` file When you use Git submodules, your project should have a file named `.gitmodules`. -You might need to modify it to work in a GitLab CI/CD job. +You have multiple options to configure it to work in a GitLab CI/CD job. -For example, your `.gitmodules` configuration might look like the following if: +### Using absolute URLs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3198) in GitLab Runner 15.11. + +For example, your generated `.gitmodules` configuration might look like the following if: - Your project is located at `https://gitlab.com/secret-group/my-project`. - Your project depends on `https://gitlab.com/group/project`, which you want @@ -26,19 +30,43 @@ For example, your `.gitmodules` configuration might look like the following if: ```ini [submodule "project"] path = project - url = ../../group/project.git + url = git@gitlab.com:secret-group/project.git +``` + +In this case, use the [`GIT_SUBMODULE_FORCE_HTTPS`](runners/configure_runners.md#rewrite-submodule-urls-to-https) variable +to instruct GitLab Runner to convert the URL to HTTPS before it clones the submodules. + +Alternatively, if you also use HTTPS locally, you can configure an HTTPS URL: + +```ini +[submodule "project"] + path = project + url = https://gitlab.com/secret-group/project.git ``` -When your submodule is on the same GitLab server, you should use relative URLs in -your `.gitmodules` file. Then you can clone with HTTPS in all your CI/CD jobs. You -can also use SSH for all your local checkouts. +You do not need to configure additional variables in this case, but you need to use a +[personal access token](../user/profile/personal_access_tokens.md) to clone it locally. + +### Using relative URLs + +WARNING: +If you use relative URLs, submodules may resolve incorrectly in forking workflows. +Use absolute URLs instead if you expect your project to have forks. + +When your submodule is on the same GitLab server, you can also use relative URLs in +your `.gitmodules` file: + +```ini +[submodule "project"] + path = project + url = ../../project.git +``` The above configuration instructs Git to automatically deduce the URL to -use when cloning sources. Git uses the same configuration for both HTTPS and SSH. -GitLab CI/CD uses HTTPS for cloning your sources, and you can continue to use SSH -to clone locally. +use when cloning sources. You can clone with HTTPS in all your CI/CD jobs, and you +can continue to use SSH to clone locally. -For submodules not located on the same GitLab server, use the full URL: +For submodules not located on the same GitLab server, always use the full URL: ```ini [submodule "project-x"] @@ -50,8 +78,6 @@ For submodules not located on the same GitLab server, use the full URL: To make submodules work correctly in CI/CD jobs: -1. Make sure you use [relative URLs](#configure-the-gitmodules-file) - for submodules located in the same GitLab server. 1. You can set the `GIT_SUBMODULE_STRATEGY` variable to either `normal` or `recursive` to tell the runner to [fetch your submodules before the job](runners/configure_runners.md#git-submodule-strategy): @@ -60,6 +86,9 @@ To make submodules work correctly in CI/CD jobs: GIT_SUBMODULE_STRATEGY: recursive ``` +1. For submodules located on the same GitLab server and configured with a Git or SSH URL, make sure + you set the [`GIT_SUBMODULE_FORCE_HTTPS`](runners/configure_runners.md#rewrite-submodule-urls-to-https) variable. + 1. Use `GIT_SUBMODULE_DEPTH` to configure the cloning depth of submodules independently of the [`GIT_DEPTH`](runners/configure_runners.md#shallow-cloning) variable: ```yaml diff --git a/doc/ci/index.md b/doc/ci/index.md index ea48a5e461d..65e6394b439 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -2,7 +2,6 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false description: "Learn how to use GitLab CI/CD, the GitLab built-in Continuous Integration, Continuous Deployment, and Continuous Delivery toolset to build, test, and deploy your application." type: index --- @@ -47,7 +46,7 @@ GitLab CI/CD uses a number of concepts to describe and run your build and deploy | [Pipelines](pipelines/index.md) | Structure your CI/CD process through pipelines. | | [CI/CD variables](variables/index.md) | Reuse values based on a variable/value key pair. | | [Environments](environments/index.md) | Deploy your application to different environments (for example, staging, production). | -| [Job artifacts](pipelines/job_artifacts.md) | Output, use, and reuse job artifacts. | +| [Job artifacts](jobs/job_artifacts.md) | Output, use, and reuse job artifacts. | | [Cache dependencies](caching/index.md) | Cache your dependencies for a faster execution. | | [GitLab Runner](https://docs.gitlab.com/runner/) | Configure your own runners to execute your scripts. | | [Pipeline efficiency](pipelines/pipeline_efficiency.md) | Configure your pipelines to run quickly and efficiently. | @@ -96,7 +95,7 @@ GitLab CI/CD features, grouped by DevOps stage, include: | [Building Docker images](docker/using_docker_build.md) | Maintain Docker-based projects using GitLab CI/CD. | | [Canary Deployments](../user/project/canary_deployments.md) | Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature. | | [Deploy boards](../user/project/deploy_boards.md) | Check the current health and status of each CI/CD environment running on Kubernetes. | -| [Feature Flags](../operations/feature_flags.md) | Deploy your features behind Feature Flags. | +| [Feature flags](../operations/feature_flags.md) | Deploy your features behind Feature flags. | | [GitLab Pages](../user/project/pages/index.md) | Deploy static websites. | | [GitLab Releases](../user/project/releases/index.md) | Add release notes to Git tags. | | [Cloud deployment](cloud_deployment/index.md) | Deploy your application to a main cloud provider. | @@ -120,7 +119,7 @@ using GitLab CI/CD with various: You can change the default behavior of GitLab CI/CD for: -- An entire GitLab instance in the [CI/CD administration settings](../administration/index.md#cicd-settings). +- An entire GitLab instance in the [CI/CD administration settings](../administration/cicd.md). - Specific projects in the [pipelines settings](pipelines/settings.md). See also: @@ -129,13 +128,13 @@ See also: ## Related topics -- [Why you might choose GitLab CI/CD](https://about.gitlab.com/blog/2016/10/17/gitlab-ci-oohlala/). -- [Reasons you might migrate from another platform](https://about.gitlab.com/blog/2016/07/22/building-our-web-app-on-gitlab-ci/). -- [Five teams that made the switch to GitLab CI/CD](https://about.gitlab.com/blog/2019/04/25/5-teams-that-made-the-switch-to-gitlab-ci-cd/). +- [Why you might choose GitLab CI/CD](https://about.gitlab.com/blog/2016/10/17/gitlab-ci-oohlala/) +- [Reasons you might migrate from another platform](https://about.gitlab.com/blog/2016/07/22/building-our-web-app-on-gitlab-ci/) +- [Five teams that made the switch to GitLab CI/CD](https://about.gitlab.com/blog/2019/04/25/5-teams-that-made-the-switch-to-gitlab-ci-cd/) - If you use VS Code to edit your GitLab CI/CD configuration, the [GitLab Workflow VS Code extension](../user/project/repository/vscode.md) helps you [validate your configuration](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow#validate-gitlab-ci-configuration) - and [view your pipeline status](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow#information-about-your-branch-pipelines-mr-closing-issue). + and [view your pipeline status](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow#information-about-your-branch-pipelines-mr-closing-issue) See also the [Why CI/CD?](https://docs.google.com/presentation/d/1OGgk2Tcxbpl7DJaIOzCX4Vqg3dlwfELC3u2jEeCBbDk) presentation. diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index db916d8d125..3f1c72c2bd6 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -12,7 +12,7 @@ When a pipeline job is about to run, GitLab generates a unique token and injects You can use a GitLab CI/CD job token to authenticate with specific API endpoints: - Packages: - - [Package Registry](../../user/packages/package_registry/index.md#use-gitlab-cicd-to-build-packages). + - [Package Registry](../../user/packages/package_registry/index.md#to-build-packages). - [Packages API](../../api/packages.md) (project-level). - [Container Registry](../../user/packages/container_registry/build_and_push_images.md#use-gitlab-cicd) (the `$CI_REGISTRY_PASSWORD` is `$CI_JOB_TOKEN`). @@ -30,7 +30,7 @@ job to run. A user can cause a job to run by taking action like pushing a commit triggering a manual job, or being the owner of a scheduled pipeline. Therefore, this user must be assigned to [a role that has the required privileges](../../user/permissions.md#gitlab-cicd-permissions). -The token is valid only while the pipeline job runs. After the job finishes, you can't +The token is valid only while the pipeline job runs. After the job finishes, you cannot use the token anymore. A job token can access a project's resources without any configuration, but it might @@ -76,39 +76,44 @@ be accessed unless projects are explicitly authorized. There is a proposal to add more strategic control of the access permissions, see [epic 3559](https://gitlab.com/groups/gitlab-org/-/epics/3559). +NOTE: +Because `CI_REGISTRY_TOKEN` uses `CI_JOB_TOKEN` to authenticate, the access configuration +also applies to `CI_REGISTRY_TOKEN`. + ### Allow access to your project with a job token -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346298/) in GitLab 15.9. [Deployed behind the `:inbound_ci_scoped_job_token` feature flag](../../user/feature_flags.md), enabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346298/) in GitLab 15.9. [Deployed behind the `:inbound_ci_scoped_job_token` feature flag](../../user/feature_flags.md), enabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/346298/) in GitLab 15.10. -Create an **inbound** allowlist of projects which can access your project through +Create an allowlist of projects which can access your project through their `CI_JOB_TOKEN`. -For example, project `A` can add project `B` to the inbound allowlist. CI/CD jobs +For example, project `A` can add project `B` to the allowlist. CI/CD jobs in project `B` (the "allowed project") can now use their CI/CD job token to authenticate API calls to access project `A`. If project `A` is public or internal, -the project can be accessed by project `B` without adding it to the inbound allowlist. +the project can be accessed by project `B` without adding it to the allowlist. -By default the inbound allowlist of any project only includes itself. +By default, the allowlist of any project only includes itself. It is a security risk to disable this feature, so project maintainers or owners should keep this setting enabled at all times. Add projects to the allowlist only when cross-project access is needed. -### Disable the inbound job token scope allowlist +### Disable the job token scope allowlist WARNING: It is a security risk to disable the allowlist. A malicious user could try to compromise a pipeline created in an unauthorized project. If the pipeline was created by one of your maintainers, the job token could be used in an attempt to access your project. -You can disable the inbound job token scope allowlist for testing or a similar reason, +You can disable the job token scope allowlist for testing or a similar reason, but you should enable it again as soon as possible. Prerequisite: - You must have at least the Maintainer role for the project. -To disable the inbound job token scope allowlist: +To disable the job token scope allowlist: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. @@ -118,9 +123,9 @@ To disable the inbound job token scope allowlist: You can also disable the allowlist [with the API](../../api/graphql/reference/index.md#mutationprojectcicdsettingsupdate). -### Add a project to the inbound job token scope allowlist +### Add a project to the job token scope allowlist -You can add projects to the inbound allowlist for a project. Projects added to the allowlist +You can add projects to the allowlist for a project. Projects added to the allowlist can make API calls from running pipelines by using the CI/CD job token. Prerequisite: @@ -148,9 +153,9 @@ You can also add a target project to the allowlist [with the API](../../api/grap NOTE: This feature is disabled by default for all new projects and is [scheduled for removal](https://gitlab.com/gitlab-org/gitlab/-/issues/383084) -in GitLab 16.0. Project maintainers or owners should enable the **inbound** access control instead. +in GitLab 16.0. Project maintainers or owners should enable the access control instead. -Control your project's job token scope by creating an **outbound** allowlist of projects which +Control your project's job token scope by creating an allowlist of projects which can be accessed by your project's job token. By default, the allowlist includes your current project. @@ -165,13 +170,13 @@ to make an API request to a private project `B`, then `B` must be added to the a If project `B` is public or internal, you do not need to add `B` to the allowlist to grant access. -### Configure the outbound job token scope +### Configure the job token scope Prerequisite: - You must not have more than 100 projects added to the token's scope. -To configure the outbound job token scope: +To configure the job token scope: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. @@ -237,10 +242,12 @@ CI job token failures are usually shown as responses like `404 Not Found` or sim While troubleshooting CI/CD job token authentication issues, be aware that: +- A [GraphQL example mutation](../../api/graphql/getting_started.md#update-project-settings) + is available to toggle the scope settings per project. - When the [CI/CD job token scopes](#configure-cicd-job-token-access) are enabled, and the job token is being used to access a different project: - The user that executes the job must be a member of the project that is being accessed. - The user must have the [permissions](../../user/permissions.md) to perform the action. - - The accessed project must have the project attempting to access it [added to the inbound allowlist](#add-a-project-to-the-inbound-job-token-scope-allowlist). + - The accessed project must have the project attempting to access it [added to the allowlist](#add-a-project-to-the-job-token-scope-allowlist). - The CI job token becomes invalid if the job is no longer running, has been erased, or if the project is in the process of being deleted. diff --git a/doc/ci/jobs/job_artifacts.md b/doc/ci/jobs/job_artifacts.md new file mode 100644 index 00000000000..a6838f2e87a --- /dev/null +++ b/doc/ci/jobs/job_artifacts.md @@ -0,0 +1,335 @@ +--- +stage: Verify +group: Pipeline Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Job artifacts **(FREE)** + +Jobs can output an archive of files and directories. This output is known as a job artifact. + +You can download job artifacts by using the GitLab UI or the [API](../../api/job_artifacts.md#get-job-artifacts). + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview of job artifacts, watch the video [GitLab CI pipelines, artifacts, and environments](https://www.youtube.com/watch?v=PCKDICEe10s). +Or, for an introduction, watch [GitLab CI pipeline tutorial for beginners](https://www.youtube.com/watch?v=Jav4vbUrqII). + +For administrator information about job artifact storage, see [administering job artifacts](../../administration/job_artifacts.md). + +## Create job artifacts + +To create job artifacts, use the [`artifacts`](../yaml/index.md#artifacts) keyword in your `.gitlab-ci.yml` file: + +```yaml +pdf: + script: xelatex mycv.tex + artifacts: + paths: + - mycv.pdf +``` + +In this example, a job named `pdf` calls the `xelatex` command to build a PDF file from the +LaTeX source file, `mycv.tex`. + +The [`paths`](../yaml/index.md#artifactspaths) keyword determines which files to add to the job artifacts. +All paths to files and directories are relative to the repository where the job was created. + +### With wildcards + +You can use wildcards for paths and directories. For example, to create an artifact +with all the files inside the directories that end with `xyz`: + +```yaml +job: + script: echo "build xyz project" + artifacts: + paths: + - path/*xyz/* +``` + +### With an expiry + +The [`expire_in`](../yaml/index.md#artifactsexpire_in) keyword determines how long +GitLab keeps the job artifacts. For example: + +```yaml +pdf: + script: xelatex mycv.tex + artifacts: + paths: + - mycv.pdf + expire_in: 1 week +``` + +If `expire_in` is not defined, the [instance-wide setting](../../user/admin_area/settings/continuous_integration.md#default-artifacts-expiration) +is used. + +To prevent artifacts from expiring, you can select **Keep** from the job details page. +The option is not available when an artifact has no expiry set. + +### With a dynamically defined name + +You can use [CI/CD variables](../variables/index.md) to dynamically define the +artifacts file's name. + +For example, to create an archive with a name of the current job: + +```yaml +job: + artifacts: + name: "$CI_JOB_NAME" + paths: + - binaries/ +``` + +To create an archive with a name of the current branch or tag including only +the binaries directory: + +```yaml +job: + artifacts: + name: "$CI_COMMIT_REF_NAME" + paths: + - binaries/ +``` + +If your branch-name contains forward slashes +(for example `feature/my-feature`) use `$CI_COMMIT_REF_SLUG` +instead of `$CI_COMMIT_REF_NAME` for proper naming of the artifact. + +### With a Windows runner or shell executor + +If you use Windows Batch to run your shell scripts you must replace `$` with `%`: + +```yaml +job: + artifacts: + name: "%CI_JOB_STAGE%-%CI_COMMIT_REF_NAME%" + paths: + - binaries/ +``` + +If you use Windows PowerShell to run your shell scripts you must replace `$` with `$env:`: + +```yaml +job: + artifacts: + name: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_NAME" + paths: + - binaries/ +``` + +### Without excluded files + +Use [`artifacts:exclude`](../yaml/index.md#artifactsexclude) to prevent files from +being added to an artifacts archive. + +For example, to store all files in `binaries/`, but not `*.o` files located in +subdirectories of `binaries/`. + +```yaml +artifacts: + paths: + - binaries/ + exclude: + - binaries/**/*.o +``` + +Unlike [`artifacts:paths`](../yaml/index.md#artifactspaths), `exclude` paths are not recursive. +To exclude all of the contents of a directory, match them explicitly rather +than matching the directory itself. + +For example, to store all files in `binaries/` but nothing located in the `temp/` subdirectory: + +```yaml +artifacts: + paths: + - binaries/ + exclude: + - binaries/temp/**/* +``` + +### With untracked files + +Use [`artifacts:untracked`](../yaml/index.md#artifactsuntracked) to add all Git untracked +files as artifacts (along with the paths defined in [`artifacts:paths`](../yaml/index.md#artifactspaths)). Untracked +files are those that haven't been added to the repository but exist in the repository checkout. + +For example, to save all Git untracked files and files in `binaries`: + +```yaml +artifacts: + untracked: true + paths: + - binaries/ +``` + +For example, to save all untracked files but [exclude](../yaml/index.md#artifactsexclude) `*.txt` files: + +```yaml +artifacts: + untracked: true + exclude: + - "*.txt" +``` + +## Prevent a job from fetching artifacts + +Jobs downloads all artifacts from the completed jobs in previous stages by default. +To prevent a job from downloading any artifacts, set [`dependencies`](../yaml/index.md#dependencies) +to an empty array (`[]`): + +```yaml +job: + stage: test + script: make build + dependencies: [] +``` + +## View all job artifacts in a project + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254938) in GitLab 15.11 [with a flag](../../administration/feature_flags.md) named `artifacts_management_page`. Disabled by default. + +You can view all artifacts stored in a project from the **CI/CD > Artifacts** page. +This list displays all jobs and their associated artifacts. Expand an entry to access +all artifacts associated with a job, including: + +- Artifacts created with the `artifacts:` keyword. +- [Report artifacts](../yaml/artifacts_reports.md). +- Job logs and metadata, which are stored internally as separate artifacts. + +You can download or delete individual artifacts from this list. + +## Download job artifacts + +You can download job artifacts from: + +- Any **Pipelines** list. On the right of the pipeline, select **Download artifacts** (**{download}**). +- Any **Jobs** list. On the right of the job, select **Download artifacts** (**{download}**). +- A job's detail page. On the right of the page, select **Download**. +- A merge request **Overview** page. On the right of the latest pipeline, select **Artifacts** (**{download}**). +- The [**Artifacts**](#view-all-job-artifacts-in-a-project) page. On the right of the job, select **Download** (**{download}**). +- The [artifacts browser](#browse-the-contents-of-the-artifacts-archive). On the top of the page, + select **Download artifacts archive** (**{download}**). + +[Report artifacts](../yaml/artifacts_reports.md) can only be downloaded from the **Pipelines** list +or **Artifacts** page. + +You can download job artifacts from the latest successful pipeline by using [the job artifacts API](../../api/job_artifacts.md). +You cannot download [artifact reports](../yaml/artifacts_reports.md) with the job artifacts API, +unless the report is added as a regular artifact with `artifacts:paths`. + +### From a URL + +You can download the artifacts archive for a specific job with a publicly accessible +URL for the [job artifacts API](../../api/job_artifacts.md#download-the-artifacts-archive). + +For example, to download the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com: + +```plaintext +https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/download?job=build +``` + +For example, to download the file `review/index.html` from the latest job named `build` in the `main` branch of a project on GitLab.com: + +```plaintext +https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/raw/review/index.html?job=build +``` + +In both examples, replace `<project-id>` with a valid project ID, found at the top of the project details page. + +Artifacts for [parent and child pipelines](../pipelines/downstream_pipelines.md#parent-child-pipelines) +are searched in hierarchical order from parent to child. For example, if both parent and +child pipelines have a job with the same name, the job artifacts from the parent pipeline are returned. + +## Browse the contents of the artifacts archive + +You can browse the contents of the artifacts from the UI without downloading the artifact locally, +from: + +- Any **Jobs** list. On the right of the job, select **Browse** (**{folder-open}**). +- A job's detail page. On the right of the page, select **Browse**. +- The **Artifacts** page. On the right of the job, select **Browse** (**{folder-open}**). + +If [GitLab Pages](../../administration/pages/index.md) is enabled in the project, you can preview +HTML files in the artifacts directly in your browser. If the project is internal or private, you must +enable [GitLab Pages access control](../../administration/pages/index.md#access-control) to preview +HTML files. + +### From a URL + +You can browse the job artifacts of the latest successful pipeline for a specific job +with a publicly accessible URL. + +For example, to browse the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com: + +```plaintext +https://gitlab.com/<full-project-path>/-/jobs/artifacts/main/browse?job=build +``` + +Replace `<full-project-path>` with a valid project path, you can find it in the URL for your project. + +## Delete job log and artifacts + +WARNING: +Deleting the job log and artifacts is a destructive action that cannot be reverted. Use with caution. + +You can delete a job's artifacts and log. + +Prerequisites: + +- You must be the owner of the job or a user with at least the Maintainer role for the project. + +To delete a job: + +1. Go to a job's detail page. +1. In the upper-right corner of the job's log, select **Erase job log and artifacts** (**{remove}**). + +## Link to job artifacts in the merge request UI + +Use the [`artifacts:expose_as`](../yaml/index.md#artifactsexpose_as) keyword to display +a link to job artifacts in the [merge request](../../user/project/merge_requests/index.md) UI. + +For example, for an artifact with a single file: + +```yaml +test: + script: ["echo 'test' > file.txt"] + artifacts: + expose_as: 'artifact 1' + paths: ['file.txt'] +``` + +With this configuration, GitLab adds **artifact 1** as a link to `file.txt` to the +**View exposed artifact** section of the relevant merge request. + +## Keep artifacts from most recent successful jobs + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16267) in GitLab 13.0. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/229936) in GitLab 13.4. +> - [Made optional with a CI/CD setting](https://gitlab.com/gitlab-org/gitlab/-/issues/241026) in GitLab 13.8. + +By default artifacts are always kept for successful pipelines for the most recent commit on +each ref. This means that the latest artifacts do not immediately expire according +to the `expire_in` specification. + +If a pipeline for a new commit on the same ref completes successfully, the previous pipeline's +artifacts are deleted according to the `expire_in` configuration. The artifacts +of the new pipeline are kept automatically. If multiple pipelines run for the most +recent commit on the ref, all artifacts are kept. + +Keeping the latest artifacts can use a large amount of storage space in projects +with a lot of jobs or large artifacts. If the latest artifacts are not needed in +a project, you can disable this behavior to save space: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Artifacts**. +1. Clear the **Keep artifacts from most recent successful jobs** checkbox. + +You can disable this behavior for all projects on a self-managed instance in the +[instance's CI/CD settings](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). + +When **Keep artifacts from most recent successful jobs** is enabled, artifacts are always kept for [blocked](job_control.md#types-of-manual-jobs) +pipelines. These artifacts expire only after the blocking job is triggered and the pipeline completes. +[Issue 387087](https://gitlab.com/gitlab-org/gitlab/-/issues/387087) proposes to change this behavior. diff --git a/doc/ci/jobs/job_artifacts_troubleshooting.md b/doc/ci/jobs/job_artifacts_troubleshooting.md new file mode 100644 index 00000000000..f5951350085 --- /dev/null +++ b/doc/ci/jobs/job_artifacts_troubleshooting.md @@ -0,0 +1,149 @@ +--- +stage: Verify +group: Pipeline Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Troubleshooting job artifacts + +When working with [job artifacts](job_artifacts.md), you might encounter the following issues. + +## Job does not retrieve certain artifacts + +By default, jobs fetch all artifacts from previous stages, but jobs using `dependencies` +or `needs` do not fetch artifacts from all jobs by default. + +If you use these keywords, artifacts are fetched from only a subset of jobs. Review +the keyword reference for information on how to fetch artifacts with these keywords: + +- [`dependencies`](../yaml/index.md#dependencies) +- [`needs`](../yaml/index.md#needs) +- [`needs:artifacts`](../yaml/index.md#needsartifacts) + +## Job artifacts use too much disk space + +If job artifacts are using too much disk space, see the +[job artifacts administration documentation](../../administration/job_artifacts.md#job-artifacts-using-too-much-disk-space). + +## Error message `No files to upload` + +This message appears in job logs when a the runner can't find the file to upload. Either +the path to the file is incorrect, or the file was not created. You can check the job +log for other errors or warnings that specify the filename and why it wasn't +generated. + +For more detailed job logs, you can [enable CI/CD debug logging](../variables/index.md#enable-debug-logging) +and try the job again. This logging might provide more information about why the file +wasn't created. + +## Error message `Missing /usr/bin/gitlab-runner-helper. Uploading artifacts is disabled.` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3068) in GitLab 15.2, GitLab Runner uses `RUNNER_DEBUG` instead of `DEBUG`, fixing this issue. + +In GitLab 15.1 and earlier, setting a CI/CD variable named `DEBUG` can cause artifact uploads to fail. + +To work around this, you can: + +- Update to GitLab and GitLab Runner 15.2 +- Use a different variable name +- Set it as an environment variable in a `script` command: + + ```yaml + failing_test_job: # This job might fail due to issue gitlab-org/gitlab-runner#3068 + variables: + DEBUG: true + script: bin/mycommand + artifacts: + paths: + - bin/results + + successful_test_job: # This job does not define a CI/CD variable named `DEBUG` and is not affected by the issue + script: DEBUG=true bin/mycommand + artifacts: + paths: + - bin/results + ``` + +## Error message `FATAL: invalid argument` when uploading a dotenv artifact on a Windows runner + +The PowerShell `echo` command writes files with UCS-2 LE BOM (Byte Order Mark) encoding, +but only UTF-8 is supported. If you try to create a [`dotenv`](../yaml/artifacts_reports.md) +artifact with `echo`, it causes a `FATAL: invalid argument` error. + +Use PowerShell `Add-Content` instead, which uses UTF-8: + +```yaml +test-job: + stage: test + tags: + - windows + script: + - echo "test job" + - Add-Content -Path build.env -Value "MY_ENV_VAR=true" + artifacts: + reports: + dotenv: build.env +``` + +## Job artifacts do not expire + +If some job artifacts are not expiring as expected, check if the +[**Keep artifacts from most recent successful jobs**](job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs) +setting is enabled. + +When this setting is enabled, job artifacts from the latest successful pipeline +of each ref do not expire and are not deleted. + +## Error message `This job could not start because it could not retrieve the needed artifacts.` + +A job configured with the [`needs:artifacts`](../yaml/index.md#needsartifacts) keyword +fails to start and returns this error message if: + +- The job's dependencies cannot be found. +- The job cannot access the relevant resources due to insufficient permissions. + +The troubleshooting steps to follow differ based on the syntax the job uses: + +- [`needs:project`](#for-a-job-configured-with-needsproject) +- [`needs:pipeline:job`](#for-a-job-configured-with-needspipelinejob) + +### For a job configured with `needs:project` + +The `could not retrieve the needed artifacts.` error can happen for a job using +[`needs:project`](../yaml/index.md#needsproject) with a configuration similar to: + +```yaml +rspec: + needs: + - project: my-group/my-project + job: dependency-job + ref: master + artifacts: true +``` + +To troubleshoot this error, verify that: + +- Project `my-group/my-project` is in a group with a Premium subscription plan. +- The user running the job can access resources in `my-group/my-project`. +- The `project`, `job`, and `ref` combination exists and results in the desired dependency. +- Any variables in use evaluate to the correct values. + +### For a job configured with `needs:pipeline:job` + +The `could not retrieve the needed artifacts.` error can happen for a job using +[`needs:pipeline:job`](../yaml/index.md#needspipelinejob) with a configuration similar to: + +```yaml +rspec: + needs: + - pipeline: $UPSTREAM_PIPELINE_ID + job: dependency-job + artifacts: true +``` + +To troubleshoot this error, verify that: + +- The `$UPSTREAM_PIPELINE_ID` CI/CD variable is available in the current pipeline's + parent-child pipeline hierarchy. +- The `pipeline` and `job` combination exists and resolves to an existing pipeline. +- `dependency-job` has run and finished successfully. diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index c1e9f97a169..b50695e8e50 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -2,7 +2,6 @@ stage: Verify group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false type: index, howto --- @@ -292,8 +291,8 @@ Self-managed runners: GitLab.com shared runners: - Linux -- [Windows](../runners/saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). -- [macOS](../runners/saas/macos_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). +- [Windows](../runners/saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta)). +- [macOS](../runners/saas/macos_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta)). ### Machine and specific build environments @@ -321,16 +320,14 @@ Example of the same job using `tags` in GitLab CI/CD: ```yaml windows job: - stage: - - build + stage: build tags: - windows script: - echo Hello, %USERNAME%! osx job: - stage: - - build + stage: build tags: - osx script: diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index 71deaadf9ec..dc1944e65c6 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -2,7 +2,6 @@ stage: Verify group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false type: index, howto --- @@ -179,7 +178,7 @@ rspec: Artifacts may work a bit differently than you've used them with Jenkins. In GitLab, any job can define a set of artifacts to be saved by using the `artifacts` keyword. This can be configured to point to a file or set of files that can then be persisted from job to job. Read more on our detailed -[artifacts documentation](../pipelines/job_artifacts.md): +[artifacts documentation](../jobs/job_artifacts.md): ```yaml pdf: diff --git a/doc/ci/mobile_devops.md b/doc/ci/mobile_devops.md index 1359191f6a6..c4316a33980 100644 --- a/doc/ci/mobile_devops.md +++ b/doc/ci/mobile_devops.md @@ -17,22 +17,23 @@ Mobile DevOps is still in development, but you can: - [Report a bug](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=report_bug). - [Share feedback](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=general_feedback). -## Code Signing +## Code signing -[Project-level Secure Files](secure_files/index.md) makes it easier to manage key stores, provision profiles, +With [project-level secure files](secure_files/index.md), you can manage key stores and provision profiles and signing certificates directly in a GitLab project. -For a guided walkthrough of this feature, watch the [video demo](https://youtu.be/O7FbJu3H2YM). +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Project-level secure files demo](https://youtu.be/O7FbJu3H2YM). -## Review Apps for Mobile +## Review apps for mobile -You can use [Review Apps](review_apps/index.md) to preview changes directly from a merge request. -Review Apps for Mobile brings that capability to mobile developers through an integration -with [Appetize](https://appetize.io/). +You can use [review apps](review_apps/index.md) to preview changes directly from a merge request. +This feature is possible through an integration with [Appetize.io](https://appetize.io/). -Watch a [video walkthrough](https://youtu.be/X15mI19TXa4) of this feature, or visit the -[setup instructions](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/readme/-/issues/15) -to get started. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Review apps for mobile setup instructions](https://youtu.be/X15mI19TXa4). + +To get started, see the [setup instructions](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/readme/-/issues/15). ## Mobile SAST @@ -40,3 +41,12 @@ You can use [Static Application Security Testing (SAST)](../user/application_sec to run static analyzers on code to check for known security vulnerabilities. Mobile SAST expands this functionality for mobile teams with an [experimental SAST feature](../user/application_security/sast/index.md#experimental-features) based on [Mobile Security Framework (MobSF)](https://github.com/MobSF/Mobile-Security-Framework-MobSF). + +## Automated releases + +With the [Apple App Store integration](../user/project/integrations/apple_app_store.md), you can configure your CI/CD pipelines to connect to [App Store Connect](https://appstoreconnect.apple.com/) to build and release apps for iOS, iPadOS, macOS, tvOS, and watchOS. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Apple App Store integration demo](https://youtu.be/CwzAWVgJeK8). + +With the [Google Play integration](../user/project/integrations/google_play.md), you can configure your CI/CD pipelines to connect to the [Google Play Console](https://play.google.com/console) to build and release apps for Android devices. diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index e1eb2f53910..f23ee5fad10 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -7,6 +7,9 @@ type: reference # CI/CD minutes quota **(PREMIUM)** +NOTE: +The term `CI/CD minutes` is being renamed to `compute credits`. During this transition, you might see references in the UI and documentation to `CI/CD minutes`, `CI minutes`, `pipeline minutes`, `CI pipeline minutes`, `pipeline minutes quota`, and `compute credits`. For more information, see [issue 5218](https://gitlab.com/gitlab-com/Product/-/issues/5218). + Administrators can limit the amount of time that projects can use to run jobs on [shared runners](../runners/runners_scope.md#shared-runners) each month. This limit is tracked with a quota of CI/CD minutes. @@ -83,11 +86,16 @@ NOTE: You can set a quota of CI/CD minutes for only top-level groups or user namespaces. If you set a quota for a subgroup, it is not used. -## View CI/CD minutes used by a group +## View CI/CD minutes -> Displaying shared runners duration per project [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355666) in GitLab 15.0. +Prerequisite: -You can view the number of CI/CD minutes being used by a group. +- You must have access to the build to view the total usage and quota summary for a namespace associated with a build. +- Access to **Usage Quotas** page is based on your role in the associated namespace or group. + +### View Usage Quota Reports for a group + +> Displaying shared runners duration per project [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355666) in GitLab 15.0. Prerequisite: @@ -105,10 +113,14 @@ The projects list shows projects with CI/CD minute usage or shared runners usage in the current month only. The list includes all projects in the namespace and its subgroups, sorted in descending order of CI/CD minute usage. -## View CI/CD minutes used by a personal namespace +### View Usage Quota reports for a personal namespace > Displaying shared runners duration [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345795) in GitLab 15.0. +Prerequisite: + +- The namespace must be your personal namespace. + You can view the number of CI/CD minutes being used by a personal namespace: 1. On the top bar, in the upper-right corner, select your avatar. @@ -144,6 +156,10 @@ You can find pricing for additional CI/CD minutes on the ### Purchase CI/CD minutes for a group **(FREE SAAS)** +Prerequisite: + +- You must have the Owner role for the group. + You can purchase additional CI/CD minutes for your group. You cannot transfer purchased CI/CD minutes from one group to another, so be sure to select the correct group. @@ -159,6 +175,10 @@ namespace. ### Purchase CI/CD minutes for a personal namespace **(FREE SAAS)** +Prerequisite: + +- The namespace must be your personal namespace. + To purchase additional minutes for your personal namespace: 1. On the top bar, in the upper-right corner, select your avatar. @@ -203,8 +223,8 @@ The cost factors for jobs running on shared runners on GitLab.com are: - `1` for internal, public, and private projects. - Exceptions for public projects: - - `0.5` for projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). - - `0.008` for forks of projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). For every 125 minutes of job execution time, + - `0.5` for projects in the [GitLab for Open Source program](../../subscriptions/community_programs.md#gitlab-for-open-source). + - `0.008` for forks of projects in the [GitLab for Open Source program](../../subscriptions/community_programs.md#gitlab-for-open-source). For every 125 minutes of job execution time, you use 1 CI/CD minute. - Discounted dynamically for [community contributions to GitLab projects](#cost-factor-for-community-contributions-to-gitlab-projects). @@ -297,6 +317,13 @@ On GitLab SaaS an email notification is sent to the namespace owners when: - The available CI/CD minutes are below 5% of the quota. - All CI/CD minutes have been used. +### Special quota limits + +In some cases, the quota limit is replaced by one of the following labels: + +- **Unlimited minutes**: For namespaces with unlimited CI/CD minutes +- **Not supported minutes**: For namespaces where active shared runners are not enabled + ## Reduce consumption of CI/CD minutes If your project consumes too many CI/CD minutes, there are some strategies you can @@ -318,3 +345,19 @@ If you manage an open source project, these improvements can also reduce CI/CD m consumption for contributor fork projects, enabling more contributions. See our [pipeline efficiency guide](pipeline_efficiency.md) for more details. + +## Reset CI/CD minutes used **(PREMIUM SELF)** + +An administrator can reset the number of minutes used by a namespace for the current month. + +### Reset minutes for a personal namespace + +1. Find the [user in the admin area](../../user/admin_area/index.md#administering-users). +1. Select **Edit**. +1. In **Limits**, select **Reset pipeline minutes**. + +### Reset minutes for a group namespace + +1. Find the [group in the admin area](../../user/admin_area/index.md#administering-groups). +1. Select **Edit**. +1. In **Permissions and group features**, select **Reset pipeline minutes**. diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md index 1e4654e69fe..ffcfee98f05 100644 --- a/doc/ci/pipelines/downstream_pipelines.md +++ b/doc/ci/pipelines/downstream_pipelines.md @@ -334,7 +334,8 @@ To retry failed and canceled jobs, select **Retry** (**{retry}**): ### Recreate a downstream pipeline -> Retry trigger job from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367547) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `ci_recreate_downstream_pipeline`. Disabled by default. +> - Retry trigger job from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367547) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `ci_recreate_downstream_pipeline`. Disabled by default. +> - [Generally available](https://gitlab.com/groups/gitlab-org/-/epics/6947) in GitLab 15.11. Feature flag `ci_recreate_downstream_pipeline` removed. You can recreate a downstream pipeline by retrying its corresponding trigger job. The newly created downstream pipeline replaces the current downstream pipeline in the pipeline graph. diff --git a/doc/ci/pipelines/img/job_artifacts_browser_button_v13_11.png b/doc/ci/pipelines/img/job_artifacts_browser_button_v13_11.png Binary files differdeleted file mode 100644 index 710a82075ce..00000000000 --- a/doc/ci/pipelines/img/job_artifacts_browser_button_v13_11.png +++ /dev/null diff --git a/doc/ci/pipelines/img/job_artifacts_browser_v13_11.png b/doc/ci/pipelines/img/job_artifacts_browser_v13_11.png Binary files differdeleted file mode 100644 index 6a8ea49b833..00000000000 --- a/doc/ci/pipelines/img/job_artifacts_browser_v13_11.png +++ /dev/null diff --git a/doc/ci/pipelines/img/job_artifacts_jobs_page_v13_11.png b/doc/ci/pipelines/img/job_artifacts_jobs_page_v13_11.png Binary files differdeleted file mode 100644 index 9f09e36b927..00000000000 --- a/doc/ci/pipelines/img/job_artifacts_jobs_page_v13_11.png +++ /dev/null diff --git a/doc/ci/pipelines/img/job_artifacts_merge_request_v13_11.png b/doc/ci/pipelines/img/job_artifacts_merge_request_v13_11.png Binary files differdeleted file mode 100644 index f4b875de471..00000000000 --- a/doc/ci/pipelines/img/job_artifacts_merge_request_v13_11.png +++ /dev/null diff --git a/doc/ci/pipelines/img/job_artifacts_pipelines_page_v13_11.png b/doc/ci/pipelines/img/job_artifacts_pipelines_page_v13_11.png Binary files differdeleted file mode 100644 index c7b0b91d488..00000000000 --- a/doc/ci/pipelines/img/job_artifacts_pipelines_page_v13_11.png +++ /dev/null diff --git a/doc/ci/pipelines/img/job_latest_artifacts_browser.png b/doc/ci/pipelines/img/job_latest_artifacts_browser.png Binary files differdeleted file mode 100644 index c6d8856078b..00000000000 --- a/doc/ci/pipelines/img/job_latest_artifacts_browser.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index d2038dbcb62..b9c8e9e1e09 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -2,7 +2,6 @@ stage: Verify group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/ci/pipelines.html' type: reference --- @@ -94,7 +93,7 @@ This table lists the refspecs injected for each pipeline type: The refs `refs/heads/<name>` and `refs/tags/<name>` exist in your project repository. GitLab generates the special ref `refs/pipelines/<id>` during a running pipeline job. This ref can be created even after the associated branch or tag has been -deleted. It's therefore useful in some features such as [automatically stopping an environment](../environments/index.md#stop-an-environment), +deleted. It's therefore useful in some features such as [automatically stopping an environment](../environments/index.md#stopping-an-environment), and [merge trains](../pipelines/merge_trains.md) that might run pipelines after branch deletion. diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index 7db4596bb1d..c2630d6810d 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -1,523 +1,11 @@ --- -stage: Verify -group: Pipeline Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/job_artifacts.html' +redirect_to: '../jobs/job_artifacts.md' +remove_date: '2023-07-04' --- -# Job artifacts **(FREE)** +This document was moved to [another location](../jobs/job_artifacts.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16675) in GitLab 12.4, artifacts in internal and private projects can be previewed when [GitLab Pages access control](../../administration/pages/index.md#access-control) is enabled. - -Jobs can output an archive of files and directories. This output is known as a job artifact. - -You can download job artifacts by using the GitLab UI or the [API](../../api/job_artifacts.md#get-job-artifacts). - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview of job artifacts, watch the video [GitLab CI pipelines, artifacts, and environments](https://www.youtube.com/watch?v=PCKDICEe10s). -Or, for an introduction, watch [GitLab CI pipeline tutorial for beginners](https://www.youtube.com/watch?v=Jav4vbUrqII). - -For administrator information about job artifact storage, see [administering job artifacts](../../administration/job_artifacts.md). - -## Create job artifacts - -To create job artifacts, use the `artifacts` keyword in your `.gitlab-ci.yml` file: - -```yaml -pdf: - script: xelatex mycv.tex - artifacts: - paths: - - mycv.pdf - expire_in: 1 week -``` - -In this example, a job named `pdf` calls the `xelatex` command to build a PDF file from the -LaTeX source file, `mycv.tex`. - -The `paths` keyword determines which files to add to the job artifacts. -All paths to files and directories are relative to the repository where the job was created. - -The `expire_in` keyword determines how long GitLab keeps the job artifacts. -You can also [use the UI to keep job artifacts from expiring](#download-job-artifacts). -If `expire_in` is not defined, the -[instance-wide setting](../../user/admin_area/settings/continuous_integration.md#default-artifacts-expiration) -is used. - -If you run two types of pipelines (like branch and scheduled) for the same ref, -the pipeline that finishes later creates the job artifact. - -To disable artifact passing, define the job with empty [dependencies](../yaml/index.md#dependencies): - -```yaml -job: - stage: build - script: make build - dependencies: [] -``` - -You may want to create artifacts only for tagged releases to avoid filling the -build server storage with temporary build artifacts. For example, use [`rules`](../yaml/index.md#rules) -to create artifacts only for tags: - -```yaml -default-job: - script: - - mvn test -U - rules: - - if: $CI_COMMIT_BRANCH - -release-job: - script: - - mvn package -U - artifacts: - paths: - - target/*.war - rules: - - if: $CI_COMMIT_TAG -``` - -You can use wildcards for directories too. For example, if you want to get all the -files inside the directories that end with `xyz`: - -```yaml -job: - artifacts: - paths: - - path/*xyz/* -``` - -### Use CI/CD variables to define the artifacts name - -You can use [CI/CD variables](../variables/index.md) to dynamically define the -artifacts file's name. - -For example, to create an archive with a name of the current job: - -```yaml -job: - artifacts: - name: "$CI_JOB_NAME" - paths: - - binaries/ -``` - -To create an archive with a name of the current branch or tag including only -the binaries directory: - -```yaml -job: - artifacts: - name: "$CI_COMMIT_REF_NAME" - paths: - - binaries/ -``` - -If your branch-name contains forward slashes -(for example `feature/my-feature`) it's advised to use `$CI_COMMIT_REF_SLUG` -instead of `$CI_COMMIT_REF_NAME` for proper naming of the artifact. - -To create an archive with a name of the current job and the current branch or -tag including only the binaries directory: - -```yaml -job: - artifacts: - name: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME" - paths: - - binaries/ -``` - -To create an archive with a name of the current [stage](../yaml/index.md#stages) and branch name: - -```yaml -job: - artifacts: - name: "$CI_JOB_STAGE-$CI_COMMIT_REF_NAME" - paths: - - binaries/ -``` - -If you use **Windows Batch** to run your shell scripts you must replace -`$` with `%`: - -```yaml -job: - artifacts: - name: "%CI_JOB_STAGE%-%CI_COMMIT_REF_NAME%" - paths: - - binaries/ -``` - -If you use **Windows PowerShell** to run your shell scripts you must replace -`$` with `$env:`: - -```yaml -job: - artifacts: - name: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_NAME" - paths: - - binaries/ -``` - -### Exclude files from job artifacts - -Use [`artifacts:exclude`](../yaml/index.md#artifactsexclude) to prevent files from -being added to an artifacts archive. - -For example, to store all files in `binaries/`, but not `*.o` files located in -subdirectories of `binaries/`. - -```yaml -artifacts: - paths: - - binaries/ - exclude: - - binaries/**/*.o -``` - -Unlike [`artifacts:paths`](../yaml/index.md#artifactspaths), `exclude` paths are not recursive. -To exclude all of the contents of a directory, match them explicitly rather -than matching the directory itself. - -For example, to store all files in `binaries/` but nothing located in the `temp/` subdirectory: - -```yaml -artifacts: - paths: - - binaries/ - exclude: - - binaries/temp/**/* -``` - -### Add untracked files to artifacts - -Use [`artifacts:untracked`](../yaml/index.md#artifactsuntracked) to add all Git untracked -files as artifacts (along with the paths defined in [`artifacts:paths`](../yaml/index.md#artifactspaths)). Untracked -files are those that haven't been added to the repository but exist in the repository checkout. - -Save all Git untracked files and files in `binaries`: - -```yaml -artifacts: - untracked: true - paths: - - binaries/ -``` - -Save all untracked files but [exclude](../yaml/index.md#artifactsexclude) `*.txt`: - -```yaml -artifacts: - untracked: true - exclude: - - "*.txt" -``` - -## Download job artifacts - -You can download job artifacts or view the job archive: - -- On the **Pipelines** page, to the right of the pipeline: - -  - -- On the **Jobs** page, to the right of the job: - -  - -- On a job's detail page. The **Keep** button indicates an `expire_in` value was set: - -  - -- On a merge request, by the pipeline details: - -  - -- When browsing an archive: - -  - - If [GitLab Pages](../../administration/pages/index.md) is enabled in the project, you can preview - HTML files in the artifacts directly in your browser. If the project is internal or private, you must - enable [GitLab Pages access control](../../administration/pages/index.md#access-control) to preview - HTML files. - -## View failed job artifacts - -If the latest job has failed to upload the artifacts, you can see that -information in the UI. - - - -## Delete job artifacts - -WARNING: -This is a destructive action that leads to data loss. Use with caution. - -You can delete a single job, which also removes the job's -artifacts and log. You must be: - -- The owner of the job. -- A user with at least the Maintainer role for the project. - -To delete a job: - -1. Go to a job's detail page. -1. In the upper-right corner of the job's log, select **Erase job log** (**{remove}**). -1. On the confirmation dialog, select **OK**. - -## Expose job artifacts in the merge request UI - -Use the [`artifacts:expose_as`](../yaml/index.md#artifactsexpose_as) keyword to expose -[job artifacts](../pipelines/job_artifacts.md) in the [merge request](../../user/project/merge_requests/index.md) UI. - -For example, to match a single file: - -```yaml -test: - script: ["echo 'test' > file.txt"] - artifacts: - expose_as: 'artifact 1' - paths: ['file.txt'] -``` - -With this configuration, GitLab adds a link **artifact 1** to the relevant merge request -that points to `file.txt`. To access the link, select **View exposed artifact** -below the pipeline graph in the merge request overview. - -An example that matches an entire directory: - -```yaml -test: - script: ["mkdir test && echo 'test' > test/file.txt"] - artifacts: - expose_as: 'artifact 1' - paths: ['test/'] -``` - -## Retrieve job artifacts for other projects - -To retrieve a job artifact from a different project, you might need to use a -private token to [authenticate and download](../../api/job_artifacts.md#get-job-artifacts) -the artifact. - -## How searching for job artifacts works - -In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201784), artifacts -for [parent and child pipelines](downstream_pipelines.md#parent-child-pipelines) are searched in hierarchical -order from parent to child. For example, if both parent and child pipelines have a -job with the same name, the job artifact from the parent pipeline is returned. - -## Access the latest job artifacts - -You can download job artifacts from the latest successful pipeline by using [the job artifacts API](../../api/job_artifacts.md). -You cannot download [artifact reports](../yaml/artifacts_reports.md) with the job artifacts API, -unless the report is added as a regular artifact with `artifacts:paths`. - -### Download the whole artifacts archive for a specific job - -You can download the artifacts archive for a specific job with [the job artifacts API](../../api/job_artifacts.md#download-the-artifacts-archive). - -For example, to download the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com: - -```plaintext -https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/download?job=build -``` - -Replace `<project-id>` with a valid project ID, found at the top of the project details page. - -### Download a single file from the artifacts - -You can download a specific file from the artifacts archive for a specific job with [the job artifacts API](../../api/job_artifacts.md#download-a-single-artifact-file-by-job-id). - -For example, to download the file `review/index.html` from the latest job named `build` in the `main` branch of the `gitlab` project in the `gitlab-org` namespace: - -```plaintext -https://gitlab.com/api/v4/projects/27456355/jobs/artifacts/main/raw/review/index.html?job=build -``` - -### Browse job artifacts - -To browse the job artifacts of the latest successful pipeline for a specific job you can use the following URL: - -```plaintext -https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/browse?job=<job_name> -``` - -For example, to browse the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com: - -```plaintext -https://gitlab.com/<full-project-path>/-/jobs/artifacts/main/browse?job=build -``` - -Replace `<full-project-path>` with a valid project path, you can find it in the URL for your project. - -## When job artifacts are deleted - -See the [`expire_in`](../yaml/index.md#artifactsexpire_in) documentation for information on when -job artifacts are deleted. - -### Keep artifacts from most recent successful jobs - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16267) in GitLab 13.0. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/229936) in GitLab 13.4. -> - [Made optional with a CI/CD setting](https://gitlab.com/gitlab-org/gitlab/-/issues/241026) in GitLab 13.8. - -By default artifacts are always kept for successful pipelines for the most recent commit on -each ref. This means that the latest artifacts do not immediately expire according -to the `expire_in` specification. - -If a pipeline for a new commit on the same ref completes successfully, the previous pipeline's -artifacts are deleted according to the `expire_in` configuration. The artifacts -of the new pipeline are kept automatically. If multiple pipelines run for the most -recent commit on the ref, all artifacts are kept. - -Keeping the latest artifacts can use a large amount of storage space in projects -with a lot of jobs or large artifacts. If the latest artifacts are not needed in -a project, you can disable this behavior to save space: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. -1. Expand **Artifacts**. -1. Clear the **Keep artifacts from most recent successful jobs** checkbox. - -You can disable this behavior for all projects on a self-managed instance in the -[instance's CI/CD settings](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). - -When **Keep artifacts from most recent successful jobs** is enabled, artifacts are always kept for [blocked](../jobs/job_control.md#types-of-manual-jobs) -pipelines. These artifacts expire only after the blocking job is triggered and the pipeline completes. -For more information, see [issue 387087](https://gitlab.com/gitlab-org/gitlab/-/issues/387087). - -## Troubleshooting - -### Job does not retrieve certain artifacts - -By default, jobs fetch all artifacts from previous stages, but jobs using `dependencies` -or `needs` do not fetch artifacts from all jobs by default. - -If you use these keywords, artifacts are fetched from only a subset of jobs. Review -the keyword reference for information on how to fetch artifacts with these keywords: - -- [`dependencies`](../yaml/index.md#dependencies) -- [`needs`](../yaml/index.md#needs) -- [`needs:artifacts`](../yaml/index.md#needsartifacts) - -### Job artifacts using too much disk space - -There are a number of potential causes for this. -[Read more in the job artifacts administration documentation](../../administration/job_artifacts.md#job-artifacts-using-too-much-disk-space). - -### Error message `No files to upload` - -This message is often preceded by other errors or warnings that specify the filename and why it wasn't -generated. Check the job log for these messages. - -If you find no helpful messages, retry the failed job after activating -[CI/CD debug logging](../variables/index.md#enable-debug-logging). -This logging should provide information to help you investigate further. - -### Error message `Missing /usr/bin/gitlab-runner-helper. Uploading artifacts is disabled.` - -There is a [known issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3068) where setting a CI/CD variable named `DEBUG` can cause artifact uploads to fail. - -To work around this, either use a different variable name or set it inline with `script`: - -```yaml -# This job might fail due to issue gitlab-org/gitlab-runner#3068 -failing_test_job: - variables: - DEBUG: true - script: bin/mycommand - artifacts: - paths: - - bin/results - -# This job does not define a CI/CD variable named `DEBUG` and is not affected by the issue -successful_test_job: - script: DEBUG=true bin/mycommand - artifacts: - paths: - - bin/results -``` - -### Error message `FATAL: invalid argument` when uploading a dotenv artifact on a windows runner - -The PowerShell `echo` command writes files with UCS-2 LE BOM (Byte Order Mark) encoding, -but only UTF-8 is supported. If you try create the dotenv artifact with `echo`, it causes a -`FATAL: invalid argument` error. - -Use PowerShell `Add-Content` instead, which uses UTF-8: - -```yaml -test-job: - stage: test - tags: - - windows - script: - - echo "test job" - - Add-Content -Path build.env -Value "MY_ENV_VAR=true" - artifacts: - reports: - dotenv: build.env -``` - -### Job artifacts are not expired - -If some job artifacts are not expiring as expected, check if the -[**Keep artifacts from most recent successful jobs**](#keep-artifacts-from-most-recent-successful-jobs) -setting is enabled. - -When this setting is enabled, job artifacts from the latest successful pipeline -of each ref do not expire and are not deleted. - -### Error message `This job could not start because it could not retrieve the needed artifacts.` - -A job configured with [`needs:artifacts`](../yaml/index.md#needsartifacts) keyword -fails to start and returns this error message if: - -- The job's dependencies cannot be found. -- The job cannot access the relevant resources due to insufficient permissions. - -The troubleshooting steps to follow are determined by the syntax used in the job configuration. - -#### Job configured with `needs:project` - -The `could not retrieve the needed artifacts.` error can happen for a job using -[`needs:project`](../yaml/index.md#needsproject), with a configuration similar to: - -```yaml -rspec: - needs: - - project: org/another-project - job: dependency-job - ref: master - artifacts: true -``` - -To troubleshoot this job, verify that: - -- Project `org/another-project` is in a group with a Premium subscription plan. -- The user running the job has permissions to access resources in `org/another-project`. -- The `project`, `job`, and `ref` combination exists and results in the desired dependency. -- Any variables in use evaluate to the correct values. - -#### Job configured with `needs:pipeline:job` - -The `could not retrieve the needed artifacts.` error can happen for a job using -[`needs:pipeline:job`](../yaml/index.md#needspipelinejob), with a configuration similar to: - -```yaml -rspec: - needs: - - pipeline: $UPSTREAM_PIPELINE_ID - job: dependency-job - artifacts: true -``` - -To troubleshoot this job, verify that: - -- The `$UPSTREAM_PIPELINE_ID` CI/CD variable is available in the current pipeline's - parent-child pipeline hierarchy. -- The `pipeline` and `job` combination exists and resolves to an existing pipeline. -- `dependency-job` has run and finished successfully. +<!-- This redirect file can be deleted after <2023-07-04>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 881e94a6559..d42f35627a2 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -202,16 +202,20 @@ It's possible to have both branch pipelines and merge request pipelines in the **Pipelines** tab of a single merge request. This might be [by configuration](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines), or [by accident](#two-pipelines-when-pushing-to-a-branch). -If both types of pipelines are in one merge request, the merge request's pipeline -is not considered successful if: - -- The branch pipeline succeeds. -- The merge request pipeline fails. - When using the [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) feature and both pipelines types are present, the merge request pipelines are checked, not the branch pipelines. +Therefore, the MR pipeline result is marked as unsuccessful if the +**merge request pipeline** fails, independently of the **branch pipeline** result. + +However: + +- These conditions are not enforced. +- A race condition determines which pipeline's result is used to either block or pass merge requests. + +This bug is tracked on [issue 384927](https://gitlab.com/gitlab-org/gitlab/-/issues/384927). + ### `An error occurred while trying to run a new pipeline for this merge request.` This error can happen when you select **Run pipeline** in a merge request, but the diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index 370d81dacc4..af29c8105ee 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -6,6 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Merge trains **(PREMIUM)** +FLAG: +In GitLab 15.11 and later, the **Start merge train** button is **Set to auto-merge** and the **Add to merge train** button is **Merge**. On self-managed GitLab, by default these changes are not available. To make them available, +ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `auto_merge_labels_mr_widget`. On GitLab.com, this feature is not available. + Use merge trains to queue merge requests and verify their changes work together before they are merged to the target branch. diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md index 097f6bad87c..316d7819f55 100644 --- a/doc/ci/pipelines/merged_results_pipelines.md +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -74,3 +74,10 @@ is not found in the merge ref. This behavior was improved in GitLab 12.4 by introducing [persistent pipeline refs](../troubleshooting.md#fatal-reference-is-not-a-tree-error). Upgrade to GitLab 12.4 or later to resolve the problem. + +### Successful merged results pipeline overrides a failed branch pipeline + +A failed branch pipeline is sometimes ignored when the +[**Pipelines must succeed** setting](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) +is activated. +[Issue 385841](https://gitlab.com/gitlab-org/gitlab/-/issues/385841) is open to track this. diff --git a/doc/ci/pipelines/pipeline_artifacts.md b/doc/ci/pipelines/pipeline_artifacts.md index 31c848d5274..d8db79a54dc 100644 --- a/doc/ci/pipelines/pipeline_artifacts.md +++ b/doc/ci/pipelines/pipeline_artifacts.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Pipeline artifacts **(FREE)** Pipeline artifacts are files created by GitLab after a pipeline finishes. Pipeline artifacts are -different to [job artifacts](job_artifacts.md) because they are not explicitly managed by +different to [job artifacts](../jobs/job_artifacts.md) because they are not explicitly managed by `.gitlab-ci.yml` definitions. Pipeline artifacts are used by the [test coverage visualization feature](../testing/test_coverage_visualization.md) diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md index 0795005aa8e..c0c8d60b9d6 100644 --- a/doc/ci/pipelines/pipeline_efficiency.md +++ b/doc/ci/pipelines/pipeline_efficiency.md @@ -147,7 +147,7 @@ with embedded metric charts and all valuable details to analyze the problem. Review the storage use of the following to help analyze costs and efficiency: -- [Job artifacts](job_artifacts.md) and their [`expire_in`](../yaml/index.md#artifactsexpire_in) +- [Job artifacts](../jobs/job_artifacts.md) and their [`expire_in`](../yaml/index.md#artifactsexpire_in) configuration. If kept for too long, storage usage grows and could slow pipelines down. - [Container registry](../../user/packages/container_registry/index.md) usage. - [Package registry](../../user/packages/package_registry/index.md) usage. diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 02728d5e1e0..3ff748644cf 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -2,7 +2,6 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/schedules.html' type: reference, howto --- @@ -84,8 +83,8 @@ To take ownership of a pipeline created by a different user: ## Related topics -- Pipeline schedules can be maintained by using the [Pipeline schedules API](../../api/pipeline_schedules.md). -- You can [control which jobs are added to scheduled pipelines](../jobs/job_control.md#run-jobs-for-scheduled-pipelines). +- [Pipeline schedules API](../../api/pipeline_schedules.md) +- [Adding jobs to scheduled pipelines](../jobs/job_control.md#run-jobs-for-scheduled-pipelines) <!-- ## Troubleshooting diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 98dc02c1041..3e9e6c50f64 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -2,7 +2,6 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/settings.html' type: reference, howto --- @@ -209,119 +208,7 @@ Jobs that exceed the timeout are marked as failed. You can override this value [for individual runners](../runners/configure_runners.md#set-maximum-job-timeout-for-a-runner). -## Merge request test coverage results - -If you use test coverage in your code, you can use a regular expression to -find coverage results in the job log. You can then include these results -in the merge request in GitLab. - -If the pipeline succeeds, the coverage is shown in the merge request widget and -in the jobs table. If multiple jobs in the pipeline have coverage reports, they are -averaged. - - - - - -### Add test coverage results using `coverage` keyword - -To add test coverage results to a merge request using the project's `.gitlab-ci.yml` file, provide a regular expression -using the [`coverage`](../yaml/index.md#coverage) keyword. - -### Test coverage examples - -Use this regex for commonly used test tools. - -<!-- vale gitlab.Spelling = NO --> - -- Simplecov (Ruby). Example: `/\(\d+.\d+\%\) covered/`. -- pytest-cov (Python). Example: `/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/`. -- Scoverage (Scala). Example: `/Statement coverage[A-Za-z\.*]\s*:\s*([^%]+)/`. -- `pest --coverage --colors=never` (PHP). Example: `/^\s*Cov:\s*\d+\.\d+?%$/`. -- `phpunit --coverage-text --colors=never` (PHP). Example: `/^\s*Lines:\s*\d+.\d+\%/`. -- gcovr (C/C++). Example: `/^TOTAL.*\s+(\d+\%)$/`. -- `tap --coverage-report=text-summary` (NodeJS). Example: `/^Statements\s*:\s*([^%]+)/`. -- `nyc npm test` (NodeJS). Example: `/All files[^|]*\|[^|]*\s+([\d\.]+)/`. -- `jest --ci --coverage` (NodeJS). Example: `/All files[^|]*\|[^|]*\s+([\d\.]+)/`. -- excoveralls (Elixir). Example: `/\[TOTAL\]\s+(\d+\.\d+)%/`. -- `mix test --cover` (Elixir). Example: `/\d+.\d+\%\s+\|\s+Total/`. -- JaCoCo (Java/Kotlin). Example: `/Total.*?([0-9]{1,3})%/`. -- `go test -cover` (Go). Example: `/coverage: \d+.\d+% of statements/`. -- .NET (OpenCover). Example: `/(Visited Points).*\((.*)\)/`. -- .NET (`dotnet test` line coverage). Example: `/Total\s*\|\s*(\d+(?:\.\d+)?)/`. -- tarpaulin (Rust). Example: `/^\d+.\d+% coverage/`. -- Pester (PowerShell). Example: `/Covered (\d+\.\d+%)/`. - -<!-- vale gitlab.Spelling = YES --> - -### View code coverage history - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/209121) the ability to download a `.csv` in GitLab 12.10. -> - Graph [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33743) in GitLab 13.1. - -To see the evolution of your project code coverage over time, -you can view a graph or download a CSV file with this data. - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Repository**. - -The historic data for each job is listed in the dropdown list above the graph. - -To view a CSV file of the data, select **Download raw data (`.csv`)**. - - - -Code coverage data is also [available at the group level](../../user/group/repositories_analytics/index.md). - -### Coverage check approval rule **(PREMIUM)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15765) in GitLab 14.0. -> - [Made configurable in Project Settings](https://gitlab.com/gitlab-org/gitlab/-/issues/331001) in GitLab 14.1. - -You can implement merge request approvals to require approval by selected users or a group -when merging a merge request would cause the project's test coverage to decline. - -Follow these steps to enable the `Coverage-Check` MR approval rule: - -1. Set up a [`coverage`](../yaml/index.md#coverage) regular expression for all jobs you want to include in the overall coverage value. -1. Go to your project and select **Settings > Merge requests**. -1. Under **Merge request approvals**, select **Enable** next to the `Coverage-Check` approval rule. -1. Select the **Target branch**. -1. Set the number of **Approvals required** to greater than zero. -1. Select the users or groups to provide approval. -1. Select **Add approval rule**. - - - -### Remove color codes from code coverage - -Some test coverage tools output with ANSI color codes that aren't -parsed correctly by the regular expression. This causes coverage -parsing to fail. - -Some coverage tools don't provide an option to disable color -codes in the output. If so, pipe the output of the coverage tool through a -small one line script that strips the color codes off. - -For example: - -```shell -lein cloverage | perl -pe 's/\e\[?.*?[\@-~]//g' -``` - ## Pipeline badges You can use [pipeline badges](../../user/project/badges.md) to indicate the pipeline status and test coverage of your projects. These badges are determined by the latest successful pipeline. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index b9e0e39396c..3e1cbca8bfa 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -156,7 +156,7 @@ For the complete `.gitlab-ci.yml` syntax, see [the full `.gitlab-ci.yml` keyword ## Related topics -- [Follow this guide to migrate from CircleCI](../migration/circleci.md). -- [Follow this guide to migrate from Jenkins](../migration/jenkins.md). +- [Migrate from CircleCI](../migration/circleci.md) +- [Migrate from Jenkins](../migration/jenkins.md) - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch [First time GitLab & CI/CD](https://www.youtube.com/watch?v=kTNfi5z6Uvk&t=553s). This includes a quick introduction to GitLab, the first steps with CI/CD, building a Go project, running tests, using the CI/CD pipeline editor, detecting secrets and security vulnerabilities and offers more exercises for asynchronous practice. - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch [Intro to GitLab CI](https://www.youtube.com/watch?v=l5705U8s_nQ&t=358s). This workshop uses the Web IDE to quickly get going with building source code using CI/CD, and run unit tests. diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md index 3835b5f2df2..118c94b4d15 100644 --- a/doc/ci/resource_groups/index.md +++ b/doc/ci/resource_groups/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: Control the job concurrency in GitLab CI/CD --- diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 9e69ab1ccb8..1b423f1df70 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -59,7 +59,7 @@ The process of configuring review apps is as follows: 1. Set up a job in `.gitlab-ci.yml` that uses the [predefined CI/CD variable](../variables/index.md) `${CI_COMMIT_REF_SLUG}` to create dynamic environments and restrict it to run only on branches. Alternatively, you can get a YAML template for this job by [enabling review apps](#enable-review-app-button) for your project. -1. Optionally, set a job that [manually stops](../environments/index.md#stop-an-environment) the review apps. +1. Optionally, set a job that [manually stops](../environments/index.md#stopping-an-environment) the review apps. ### Enable review app button @@ -193,13 +193,18 @@ After you have the route mapping set up, it takes effect in the following locati  -## Visual Reviews **(PREMIUM)** +<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> +## Visual Reviews (deprecated) **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10761) in GitLab 12.0. > - [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. > - It's [deployed behind a feature flag](../../user/feature_flags.md), `anonymous_visual_review_feedback`, disabled by default. > - It's disabled on GitLab.com. +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387751) in GitLab 15.8 +and is planned for removal in 17.0. This change is a breaking change. + FLAG: On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `anonymous_visual_review_feedback`. @@ -305,3 +310,5 @@ the user must enter a [personal access token](../../user/profile/personal_access with `api` scope before submitting feedback. This same method can be used to require authentication for any public projects. + +<!--- end_remove --> diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index 41dda5da0b2..cf4b95f511d 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -249,6 +249,29 @@ Example 2: 1. A job that has no tags defined is executed and run. 1. A second job that has a `docker` tag defined is stuck. +### A runner and a job have multiple tags + +The selection logic that matches the job and runner is based on the list of `tags` +defined in the job. + +The following examples illustrate the impact of a runner and a job having multiple tags. For a runner to be +selected to run a job, it must have all of the tags defined in the job script block. + +Example 1: + +1. The runner is configured with the tags `[docker, shell, gpu]`. +1. The job has the tags `[docker, shell, gpu]` and is executed and run. + +Example 2: + +1. The runner is configured with the tags `[docker, shell, gpu]`. +1. The job has the tags `[docker, shell,]` and is executed and run. + +Example 3: + +1. The runner is configured with the tags `[docker, shell]`. +1. The job has the tags `[docker, shell, gpu]` and is not executed. + ### Use tags to run jobs on different platforms You can use tags to run different jobs on different platforms. For @@ -257,16 +280,14 @@ example, if you have an OS X runner with tag `osx` and a Windows runner with tag ```yaml windows job: - stage: - - build + stage: build tags: - windows script: - echo Hello, %USERNAME%! osx job: - stage: - - build + stage: build tags: - osx script: @@ -313,6 +334,7 @@ globally or for individual jobs: - [`GIT_CLEAN_FLAGS`](#git-clean-flags) - [`GIT_FETCH_EXTRA_FLAGS`](#git-fetch-extra-flags) - [`GIT_SUBMODULE_UPDATE_FLAGS`](#git-submodule-update-flags) +- [`GIT_SUBMODULE_FORCE_HTTPS`](#rewrite-submodule-urls-to-https) - [`GIT_DEPTH`](#shallow-cloning) (shallow cloning) - [`GIT_SUBMODULE_DEPTH`](#git-submodule-depth) - [`GIT_CLONE_PATH`](#custom-build-directories) (custom build directories) @@ -569,6 +591,23 @@ You should be aware of the implications for the security, stability, and reprodu your builds when using the `--remote` flag. In most cases, it is better to explicitly track submodule commits as designed, and update them using an auto-remediation/dependency bot. +### Rewrite submodule URLs to HTTPS + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3198) in GitLab Runner 15.11. + +Use the `GIT_SUBMODULE_FORCE_HTTPS` variable to force a rewrite of all Git and SSH submodule URLs to HTTPS. +This allows you to clone submodules on the same GitLab instance that use absolute URLs, even if they were +configured with a Git or SSH protocol. + +```yaml +variables: + GIT_SUBMODULE_STRATEGY: recursive + GIT_SUBMODULE_FORCE_HTTPS: "true" +``` + +When enabled, GitLab Runner uses a [CI/CD job token](../jobs/ci_job_token.md) to clone the submodules with +the permissions of the user executing the job, and does not require SSH credentials. + ### Shallow cloning > Introduced in GitLab 8.9 as an experimental feature. @@ -782,7 +821,7 @@ variables: NOTE: Zip archives are the only supported artifact type. Follow [the issue for details](https://gitlab.com/gitlab-org/gitlab/-/issues/367203). -GitLab Runner can generate and produce attestation metadata for all build artifacts. To enable this feature, you must set the `RUNNER_GENERATE_ARTIFACTS_METADATA` environment variable to `true`. This variable can either be set globally or it can be set for individual jobs. The metadata is in rendered in a plain text `.json` file that's stored with the artifact. The filename is as follows: `{ARTIFACT_NAME}-metadata.json` where `ARTIFACT_NAME` is what was defined as the [name for the artifact](../pipelines/job_artifacts.md#use-cicd-variables-to-define-the-artifacts-name) in the CI file. The filename, however, defaults to `artifacts-metadata.json` if no name was given to the build artifacts. +GitLab Runner can generate and produce attestation metadata for all build artifacts. To enable this feature, you must set the `RUNNER_GENERATE_ARTIFACTS_METADATA` environment variable to `true`. This variable can either be set globally or it can be set for individual jobs. The metadata is in rendered in a plain text `.json` file that's stored with the artifact. The filename is as follows: `{ARTIFACT_NAME}-metadata.json` where `ARTIFACT_NAME` is what was defined as the [name for the artifact](../jobs/job_artifacts.md#with-a-dynamically-defined-name) in the CI file. The filename, however, defaults to `artifacts-metadata.json` if no name was given to the build artifacts. ### Attestation format diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index d3bbd7d8816..6883f0727dd 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -11,8 +11,8 @@ If you use GitLab SaaS (GitLab.com), your [untagged](../../ci/runners/configure_ As long as shared runners are enabled for your project, no configuration is required. Your jobs can run on: - [Linux runners](saas/linux_saas_runner.md). -- [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). -- [macOS runners](saas/macos_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). +- [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta)). +- [macOS runners](saas/macos_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta)). The number of minutes you can use on these runners depends on the [maximum number of CI/CD minutes](../pipelines/cicd_minutes.md) @@ -22,7 +22,7 @@ in your [subscription plan](https://about.gitlab.com/pricing/). GitLab SaaS runners on Linux and Windows run on Google Compute Platform. The [Google Infrastructure Security Design Overview whitepaper](https://cloud.google.com/docs/security/infrastructure/design/resources/google_infrastructure_whitepaper_fa.pdf) provides an overview of how Google designs security into its technical infrastructure. The GitLab [Trust Center](https://about.gitlab.com/security/) and [GitLab Security Compliance Controls](https://about.staging.gitlab.com/handbook/engineering/security/security-assurance/security-compliance/sec-controls.html) pages provide an overview of the security and compliance controls that govern the GitLab SaaS runners. -The runner that serves as a Runner Manager automatically initiates the creation and deletion of the virtual machines (VMs) used for CI jobs. When the Runner Manager picks up a GitLab SaaS CI job, it automatically executes that job on a new VM. There is no human or manual intervention in this process. The following section provides an overview of the additional built-in layers that harden the security of the GitLab Runner SaaS CI build environment. +The runner that serves as a runner manager automatically initiates the creation and deletion of the virtual machines (VMs) used for CI jobs. When the runner manager picks up a GitLab SaaS CI job, it automatically executes that job on a new VM. There is no human or manual intervention in this process. The following section provides an overview of the additional built-in layers that harden the security of the GitLab Runner SaaS CI build environment. ### Security of CI job execution on GitLab Runner SaaS (Linux, Windows) @@ -39,4 +39,4 @@ GitLab sends the command to remove the temporary runner VM to the Google Compute - Firewall rules only allow outbound communication from the temporary VM to the public internet. - Inbound communication from the public internet to the temporary VM is not allowed. - Firewall rules do not permit communication between VMs. -- The only internal communication allowed to the temporary VMs is from the Runner Manager. +- The only internal communication allowed to the temporary VMs is from the runner manager. diff --git a/doc/ci/runners/register_runner.md b/doc/ci/runners/register_runner.md index ec10a1299dc..7b22e6215ba 100644 --- a/doc/ci/runners/register_runner.md +++ b/doc/ci/runners/register_runner.md @@ -4,9 +4,21 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Register a shared runner **(FREE)** +# Generate runner tokens **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383139) in GitLab 15.10. [Deployed behind the `create_runner_workflow_for_admin` flag](../../administration/feature_flags.md), disabled by default. +To register a runner, you can use either: + +- An authentication token assigned to the runner when you create the runner in the UI. The runner uses the token to authenticate with GitLab when picking up jobs from the job queue. +- A registration token (deprecated). + +## Generate an authentication token + +Registration with an authentication token is only available for shared runners. Support for project and group +runners is proposed in this [epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). + +### For a shared runner + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383139) in GitLab 15.10. Deployed behind the `create_runner_workflow_for_admin` [flag](../../administration/feature_flags.md), disabled by default. FLAG: On self-managed GitLab, this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `create_runner_workflow_for_admin`. @@ -14,9 +26,9 @@ On GitLab.com, this feature is available but can be configured by GitLab.com adm Prerequisites: -- You must be an administrator to register a shared runner. +- You must be an administrator. -To register a [shared runner](../runners/runners_scope.md#shared-runners): +To generate an authentication token for a shared runner: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **CI/CD > Runners**. @@ -31,3 +43,32 @@ To register a [shared runner](../runners/runners_scope.md#shared-runners): NOTE: The token only displays in the UI for a short period of time during registration, and is then saved in `config.toml`. + +## Generate a registration token (deprecated) + +WARNING: +The ability to pass a runner registration token was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6 and is +planned for removal in 17.0, along with support for certain configuration arguments. This change is a breaking change. GitLab plans to introduce a new +[GitLab Runner token architecture](../../architecture/blueprints/runner_tokens/index.md), which introduces +a new method for registering runners and eliminates the legacy +[runner registration token](../../security/token_overview.md#runner-registration-tokens-deprecated). + +### For a shared runner + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **CI/CD > Runners**. +1. Select **Register an instance runner**. +1. Copy the registration token. + +### For a group runner + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **CI/CD > Runners**. +1. Copy the registration token. + +### For a project runner + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand the **Runners** section. +1. Copy the registration token. diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index 21120732fbe..20be2f2a147 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # SaaS runners on macOS (Beta) **(PREMIUM SAAS)** -SaaS runners on macOS are in [Beta](../../../policy/alpha-beta-support.md#beta-features) for approved open source programs and customers in Premium and Ultimate plans. +SaaS runners on macOS are in [Beta](../../../policy/alpha-beta-support.md#beta) for approved open source programs and customers in Premium and Ultimate plans. SaaS runners on macOS provide an on-demand macOS build environment integrated with GitLab SaaS [CI/CD](../../../ci/index.md). diff --git a/doc/ci/runners/saas/windows_saas_runner.md b/doc/ci/runners/saas/windows_saas_runner.md index 2c6540833b0..6e4ffc036b0 100644 --- a/doc/ci/runners/saas/windows_saas_runner.md +++ b/doc/ci/runners/saas/windows_saas_runner.md @@ -4,9 +4,9 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# SaaS runners on Windows (beta) **(FREE SAAS)** +# SaaS runners on Windows (Beta) **(FREE SAAS)** -SaaS runners on Windows are in [beta](../../../policy/alpha-beta-support.md#beta-features) +SaaS runners on Windows are in [Beta](../../../policy/alpha-beta-support.md#beta) and shouldn't be used for production workloads. During this beta period, the [shared runner quota for CI/CD minutes](../../pipelines/cicd_minutes.md) @@ -126,7 +126,7 @@ test: ## Limitations and known issues -- All the limitations mentioned in our [beta definition](../../../policy/alpha-beta-support.md#beta-features). +- All the limitations mentioned in our [Beta definition](../../../policy/alpha-beta-support.md#beta). - The average provisioning time for a new Windows VM is 5 minutes. This means that you may notice slower build start times on the Windows runner fleet during the beta. In a future diff --git a/doc/ci/secrets/id_token_authentication.md b/doc/ci/secrets/id_token_authentication.md index b10763b40d6..d6d1f62e47d 100644 --- a/doc/ci/secrets/id_token_authentication.md +++ b/doc/ci/secrets/id_token_authentication.md @@ -35,60 +35,64 @@ services with which a token can authenticate. This reduces the severity of havin ### Token payload -The following fields are included in each ID token: +The following standard claims are included in each ID token: + +| Field | Description | +|--------------------------------------------------------------------|-------------| +| [`iss`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.1) | Issuer of the token, which is the domain of the GitLab instance ("issuer" claim). | +| [`sub`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.2) | `project_path:{group}/{project}:ref_type:{type}:ref:{branch_name}` ("subject" claim). | +| [`aud`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.3) | Intended audience for the token ("audience" claim). Specified in the [ID tokens](../yaml/index.md#id_tokens) configuration. The domain of the GitLab instance by default. | +| [`exp`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.4) | The expiration time ("expiration time" claim). | +| [`nbf`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.5) | The time after which the token becomes valid ("not before" claim). | +| [`iat`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.6) | The time the JWT was issued ("issued at" claim). | +| [`jti`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.7) | Unique identifier for the token ("JWT ID" claim). | + +The token also includes custom claims provided by GitLab: | Field | When | Description | |-------------------------|------------------------------|-------------| -| [`aud`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.3) | Always | Intended audience for the token ("audience" claim). Configured in GitLab the CI/CD configuration. The domain of the GitLab instance by default. | -| [`exp`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.4) | Always | The expiration time ("expiration time" claim). | -| [`iat`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.6) | Always | The time the JWT was issued ("issued at" claim). | -| [`iss`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.1) | Always | Issuer of the token, which is the domain of the GitLab instance ("issuer" claim). | -| [`jti`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.7) | Always | Unique identifier for the token ("JWT ID" claim). | -| [`nbf`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.5) | Always | The time after which the token becomes valid ("not before" claim). | -| [`sub`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.2) | Always | `project_path:{group}/{project}:ref_type:{type}:ref:{branch_name}` ("subject" claim). | -| `deployment_tier` | Job specifies an environment | [Deployment tier](../environments/index.md#deployment-tier-of-environments) of the environment the job specifies. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363590) in GitLab 15.2. | -| `environment_protected` | Job specifies an environment | `true` if specified environment is protected, `false` otherwise. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9. | -| `environment` | Job specifies an environment | Environment the job specifies. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9. | -| `job_id` | Always | ID of the job. | -| `namespace_id` | Always | Use to scope to group or user level namespace by ID. | -| `namespace_path` | Always | Use to scope to group or user level namespace by path. | +| `namespace_id` | Always | Use this to scope to group or user level namespace by ID. | +| `namespace_path` | Always | Use this to scope to group or user level namespace by path. | +| `project_id` | Always | Use this to scope to project by ID. | +| `project_path` | Always | Use this to scope to project by path. | +| `user_id` | Always | ID of the user executing the job. | +| `user_login` | Always | Username of the user executing the job. | +| `user_email` | Always | Email of the user executing the job. | | `pipeline_id` | Always | ID of the pipeline. | | `pipeline_source` | Always | [Pipeline source](../jobs/job_control.md#common-if-clauses-for-rules). | -| `project_id` | Always | Use to scope to project by ID. | -| `project_path` | Always | Use to scope to project by path. | -| `ref_protected` | Always | `true` if the Git ref is protected, `false` otherwise. | -| `ref_type` | Always | Git ref type, either `branch` or `tag`. | +| `job_id` | Always | ID of the job. | | `ref` | Always | Git ref for the job. | -| `user_email` | Always | Email of the user executing the job. | -| `user_id` | Always | ID of the user executing the job. | -| `user_login` | Always | Username of the user executing the job. | - -Example ID token payload: +| `ref_type` | Always | Git ref type, either `branch` or `tag`. | +| `ref_protected` | Always | `true` if the Git ref is protected, `false` otherwise. | +| `environment` | Job specifies an environment | Environment this job deploys to ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9). | +| `environment_protected` | Job specifies an environment | `true` if deployed environment is protected, `false` otherwise ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9). | +| `deployment_tier` | Job specifies an environment | [Deployment tier](../environments/index.md#deployment-tier-of-environments) of the environment the job specifies. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363590) in GitLab 15.2. | ```json { - "jti": "c82eeb0c-5c6f-4a33-abf5-4c474b92b558", - "aud": "hashicorp.example.com", - "iss": "gitlab.example.com", - "iat": 1585710286, - "nbf": 1585798372, - "exp": 1585713886, - "sub": "job_1212", - "namespace_id": "1", - "namespace_path": "mygroup", - "project_id": "22", - "project_path": "mygroup/myproject", - "user_id": "42", - "user_login": "myuser", - "user_email": "myuser@example.com", - "pipeline_id": "1212", - "pipeline_source": "web", - "job_id": "1212", - "ref": "auto-deploy-2020-04-01", + "namespace_id": "72", + "namespace_path": "my-group", + "project_id": "20", + "project_path": "my-group/my-project", + "user_id": "1", + "user_login": "sample-user", + "user_email": "sample-user@example.com", + "pipeline_id": "574", + "pipeline_source": "push", + "job_id": "302", + "ref": "feature-branch-1", "ref_type": "branch", - "ref_protected": "true", - "environment": "production", - "environment_protected": "true" + "ref_protected": "false", + "environment": "test-environment2", + "environment_protected": "false", + "deployment_tier": "testing", + "jti": "235b3a54-b797-45c7-ae9a-f72d7bc6ef5b", + "iss": "https://gitlab.example.com", + "iat": 1681395193, + "nbf": 1681395188, + "exp": 1681398793, + "sub": "project_path:my-group/my-project:ref_type:branch:ref:feature-branch-1", + "aud": "https://vault.example.com" } ``` diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md index fa8fe2a36a8..6966a6baadf 100644 --- a/doc/ci/secrets/index.md +++ b/doc/ci/secrets/index.md @@ -23,14 +23,7 @@ GitLab has selected [Vault by HashiCorp](https://www.vaultproject.io) as the first supported provider, and [KV-V2](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2) as the first supported secrets engine. -By default, GitLab authenticates using Vault's -[JSON Web Token (JWT) authentication method](https://developer.hashicorp.com/vault/docs/auth/jwt#jwt-authentication), using -the [JSON Web Token](https://gitlab.com/gitlab-org/gitlab/-/issues/207125) (`CI_JOB_JWT`). - -[ID tokens](../yaml/index.md#id_tokens) is the preferred secure way to authenticate with Vault, -because ID tokens are defined per-job. GitLab can also authenticate with Vault by using the `CI_JOB_JWT`, -but that token is provided to every job, which can be a security risk. - +Use [ID tokens](../yaml/index.md#id_tokens) to [authenticate with Vault](https://developer.hashicorp.com/vault/docs/auth/jwt#jwt-authentication). The [Authenticating and Reading Secrets With HashiCorp Vault](../examples/authenticating-with-hashicorp-vault/index.md) tutorial has more details about authenticating with ID tokens. @@ -40,7 +33,7 @@ can use [use Vault secrets in a CI job](#use-vault-secrets-in-a-ci-job). The flow for using GitLab with HashiCorp Vault is summarized by this diagram: - + 1. Configure your vault and secrets. 1. Generate your JWT and provide it to your CI job. diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index eca4c62deb2..5107a62f734 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -2,7 +2,6 @@ stage: Verify group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false type: index --- diff --git a/doc/ci/test_cases/index.md b/doc/ci/test_cases/index.md index 8ad9e27de8b..edc962c9921 100644 --- a/doc/ci/test_cases/index.md +++ b/doc/ci/test_cases/index.md @@ -16,6 +16,12 @@ Test cases in GitLab can help your teams create testing scenarios in their exist Now your Implementation and Testing teams can collaborate better, as they no longer have to use external test planning tools, which require additional overhead, context switching, and expense. +NOTE: +[Requirements](../../user/project/requirements/index.md) and test cases are being +[migrated to work items](https://gitlab.com/groups/gitlab-org/-/epics/5171). +[Issue 323790](https://gitlab.com/gitlab-org/gitlab/-/issues/323790) proposes to link requirements to test cases. +For more information, see [Product Stage Direction - Plan](https://about.gitlab.com/direction/plan/). + ## Create a test case Prerequisite: diff --git a/doc/ci/testing/accessibility_testing.md b/doc/ci/testing/accessibility_testing.md index 58bb4308095..b03e4a23153 100644 --- a/doc/ci/testing/accessibility_testing.md +++ b/doc/ci/testing/accessibility_testing.md @@ -4,10 +4,15 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Accessibility testing **(FREE)** +<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> +# Accessibility testing (deprecated) **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25144) in GitLab 12.8. +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/390424) in GitLab 15.9 +and is planned for removal in 17.0. This change is a breaking change. + If your application offers a web interface, you can use [GitLab CI/CD](../index.md) to determine the accessibility impact of pending code changes. @@ -25,7 +30,7 @@ As of [GitLab 14.5](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73309) ## Accessibility merge request widget > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39425) in GitLab 13.0 behind the disabled [feature flag](../../administration/feature_flags.md) `:accessibility_report_view`. -> - [Feature Flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217372) in GitLab 13.1. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217372) in GitLab 13.1. GitLab displays an **Accessibility Report** in the merge request widget area: @@ -63,7 +68,7 @@ The `a11y` job in your CI/CD pipeline generates these files: file is named `gl-accessibility.json`. In GitLab versions 12.10 and earlier, this file is named [`accessibility.json`](https://gitlab.com/gitlab-org/ci-cd/accessibility/-/merge_requests/9). -You can [view job artifacts in your browser](../pipelines/job_artifacts.md#download-job-artifacts). +You can [view job artifacts in your browser](../jobs/job_artifacts.md#download-job-artifacts). NOTE: For GitLab versions earlier than 12.9, use `include:remote` and @@ -74,3 +79,5 @@ The job definition provided by the template does not support Kubernetes. You cannot pass configurations into Pa11y via CI configuration. To change the configuration, edit a copy of the template in your CI file. + +<!--- end_remove -->
\ No newline at end of file diff --git a/doc/ci/testing/browser_performance_testing.md b/doc/ci/testing/browser_performance_testing.md index 175ecef7f09..600b1a2cf4b 100644 --- a/doc/ci/testing/browser_performance_testing.md +++ b/doc/ci/testing/browser_performance_testing.md @@ -4,10 +4,15 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- +<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> # Browser Performance Testing **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3507) in GitLab 10.3. +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/388719) in GitLab 15.9 +and is planned for removal in 17.0. This change is a breaking change. + If your application offers a web interface and you're using [GitLab CI/CD](../index.md), you can quickly determine the rendering performance impact of pending code changes in the browser. diff --git a/doc/ci/testing/code_coverage.md b/doc/ci/testing/code_coverage.md new file mode 100644 index 00000000000..bd1246d2f78 --- /dev/null +++ b/doc/ci/testing/code_coverage.md @@ -0,0 +1,129 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Code coverage **(FREE)** + +Use code coverage to provide insights on what source code is being validated by a test suite. Code coverage is one of many test metrics that can determine software performance and quality. + +## View Code Coverage results + +Code Coverage results are shown in: + +- Merge request widget +- Project repository analytics +- Group repository analytics +- Repository badge + +For more information on test coverage visualization in the file diff of the MR, see [Test Coverage Visualization](test_coverage_visualization.md). + +### View code coverage results in the MR + +If you use test coverage in your code, you can use a regular expression to +find coverage results in the job log. You can then include these results +in the merge request in GitLab. + +If the pipeline succeeds, the coverage is shown in the merge request widget and +in the jobs table. If multiple jobs in the pipeline have coverage reports, they are +averaged. + + + + + +#### Add test coverage results using `coverage` keyword + +To add test coverage results to a merge request using the project's `.gitlab-ci.yml` file, provide a regular expression +using the [`coverage`](../yaml/index.md#coverage) keyword. + +#### Test coverage examples + +Use this regex for commonly used test tools. + +<!-- vale gitlab.Spelling = NO --> + +- Simplecov (Ruby). Example: `/\(\d+.\d+\%\) covered/`. +- pytest-cov (Python). Example: `/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/`. +- Scoverage (Scala). Example: `/Statement coverage[A-Za-z\.*]\s*:\s*([^%]+)/`. +- `pest --coverage --colors=never` (PHP). Example: `/^\s*Cov:\s*\d+\.\d+?%$/`. +- `phpunit --coverage-text --colors=never` (PHP). Example: `/^\s*Lines:\s*\d+.\d+\%/`. +- gcovr (C/C++). Example: `/^TOTAL.*\s+(\d+\%)$/`. +- `tap --coverage-report=text-summary` (NodeJS). Example: `/^Statements\s*:\s*([^%]+)/`. +- `nyc npm test` (NodeJS). Example: `/All files[^|]*\|[^|]*\s+([\d\.]+)/`. +- `jest --ci --coverage` (NodeJS). Example: `/All files[^|]*\|[^|]*\s+([\d\.]+)/`. +- excoveralls (Elixir). Example: `/\[TOTAL\]\s+(\d+\.\d+)%/`. +- `mix test --cover` (Elixir). Example: `/\d+.\d+\%\s+\|\s+Total/`. +- JaCoCo (Java/Kotlin). Example: `/Total.*?([0-9]{1,3})%/`. +- `go test -cover` (Go). Example: `/coverage: \d+.\d+% of statements/`. +- .NET (OpenCover). Example: `/(Visited Points).*\((.*)\)/`. +- .NET (`dotnet test` line coverage). Example: `/Total\s*\|\s*(\d+(?:\.\d+)?)/`. +- tarpaulin (Rust). Example: `/^\d+.\d+% coverage/`. +- Pester (PowerShell). Example: `/Covered (\d+\.\d+%)/`. + +<!-- vale gitlab.Spelling = YES --> + +### View history of project code coverage + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/209121) the ability to download a `.csv` in GitLab 12.10. +> - Graph [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33743) in GitLab 13.1. + +To see the evolution of your project code coverage over time, +you can view a graph or download a CSV file with this data. + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Analytics > Repository**. + +The historic data for each job is listed in the dropdown list above the graph. + +To view a CSV file of the data, select **Download raw data (`.csv`)**. + + + +### View history of group code coverage **(PREMIUM)** + +To see the all the project's code coverage under a group over time, you can find view [group repository analytics](../../user/group/repositories_analytics/index.md). + + + +### Pipeline badges + +You can use [pipeline badges](../../user/project/badges.md#test-coverage-report-badges) to indicate the pipeline status and +test coverage of your projects. These badges are determined by the latest successful pipeline. + +## Coverage check approval rule **(PREMIUM)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15765) in GitLab 14.0. +> - [Made configurable in Project Settings](https://gitlab.com/gitlab-org/gitlab/-/issues/331001) in GitLab 14.1. + +When merging a request that would cause the project's test coverage to decline, you can stipulate that such merge requests require approval by selected users or a group. + +Follow these steps to enable the `Coverage-Check` MR approval rule: + +1. Set up a [`coverage`](../yaml/index.md#coverage) regular expression for all jobs you want to include in the overall coverage value. +1. Go to your project and select **Settings > Merge requests**. +1. Under **Merge request approvals**, select **Enable** next to the `Coverage-Check` approval rule. +1. Select the **Target branch**. +1. Set the number of **Approvals required** to greater than zero. +1. Select the users or groups to provide approval. +1. Select **Add approval rule**. + + + +## Troubleshooting + +### Remove color codes from code coverage + +Some test coverage tools output with ANSI color codes that aren't +parsed correctly by the regular expression. This causes coverage +parsing to fail. + +Some coverage tools do not provide an option to disable color +codes in the output. If so, pipe the output of the coverage tool through a one-line script that strips the color codes. + +For example: + +```shell +lein cloverage | perl -pe 's/\e\[?.*?[\@-~]//g' +``` diff --git a/doc/ci/testing/code_quality.md b/doc/ci/testing/code_quality.md index ff7e11e9c71..98915de1385 100644 --- a/doc/ci/testing/code_quality.md +++ b/doc/ci/testing/code_quality.md @@ -74,9 +74,9 @@ tab of the pipeline's details page.  -### Project quality view +### Project quality view **(ULTIMATE)** -The project quality view displays an overview of the code quality findings. +The project quality view displays an overview of the code quality findings. The view can be found under **Analytics > CI/CD**, and requires [`project_quality_summary_page`](../../user/feature_flags.md) feature flag to be enabled for this particular project.  @@ -260,7 +260,7 @@ code_quality: ``` The full JSON file is available as a -[downloadable artifact](../pipelines/job_artifacts.md#download-job-artifacts) of the `code_quality` +[downloadable artifact](../jobs/job_artifacts.md#download-job-artifacts) of the `code_quality` job. ### Download output in JSON and HTML format @@ -287,7 +287,7 @@ code_quality_html: ``` Both the JSON and HTML files are available as -[downloadable artifacts](../pipelines/job_artifacts.md#download-job-artifacts) of the `code_quality` +[downloadable artifacts](../jobs/job_artifacts.md#download-job-artifacts) of the `code_quality` job. ### Download output in only HTML format @@ -311,7 +311,7 @@ code_quality: ``` The HTML file is available as a -[downloadable artifact](../pipelines/job_artifacts.md#download-job-artifacts) of the `code_quality` +[downloadable artifact](../jobs/job_artifacts.md#download-job-artifacts) of the `code_quality` job. ## Use Code Quality with merge request pipelines diff --git a/doc/ci/pipelines/img/code_coverage_graph_v13_1.png b/doc/ci/testing/img/code_coverage_graph_v13_1.png Binary files differindex da92a1b4a86..da92a1b4a86 100644 --- a/doc/ci/pipelines/img/code_coverage_graph_v13_1.png +++ b/doc/ci/testing/img/code_coverage_graph_v13_1.png diff --git a/doc/ci/testing/img/code_coverage_group_report.png b/doc/ci/testing/img/code_coverage_group_report.png Binary files differnew file mode 100644 index 00000000000..df8ca613f8c --- /dev/null +++ b/doc/ci/testing/img/code_coverage_group_report.png diff --git a/doc/ci/pipelines/img/coverage_check_approval_rule_14_1.png b/doc/ci/testing/img/coverage_check_approval_rule_14_1.png Binary files differindex 8c1d005e074..8c1d005e074 100644 --- a/doc/ci/pipelines/img/coverage_check_approval_rule_14_1.png +++ b/doc/ci/testing/img/coverage_check_approval_rule_14_1.png diff --git a/doc/ci/pipelines/img/pipelines_test_coverage_build.png b/doc/ci/testing/img/pipelines_test_coverage_build.png Binary files differindex 7eaba1a256f..7eaba1a256f 100644 --- a/doc/ci/pipelines/img/pipelines_test_coverage_build.png +++ b/doc/ci/testing/img/pipelines_test_coverage_build.png diff --git a/doc/ci/pipelines/img/pipelines_test_coverage_mr_widget.png b/doc/ci/testing/img/pipelines_test_coverage_mr_widget.png Binary files differindex fbcd612f3f2..fbcd612f3f2 100644 --- a/doc/ci/pipelines/img/pipelines_test_coverage_mr_widget.png +++ b/doc/ci/testing/img/pipelines_test_coverage_mr_widget.png diff --git a/doc/ci/testing/index.md b/doc/ci/testing/index.md index 5fd86561b9e..a8fb6d688d7 100644 --- a/doc/ci/testing/index.md +++ b/doc/ci/testing/index.md @@ -14,8 +14,9 @@ display reports or link to important information directly from [merge requests]( | [Accessibility Testing](accessibility_testing.md) | Automatically report A11y violations for changed pages in merge requests. | | [Browser Performance Testing](browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | | [Load Performance Testing](load_performance_testing.md) | Quickly determine the server performance impact of pending code changes. | +| [Code Coverage](code_coverage.md) | See code coverage results in the MR, project or group. | | [Code Quality](code_quality.md) | Analyze your source code quality using the [Code Climate](https://codeclimate.com/) analyzer and show the Code Climate report right in the merge request widget area. | -| [Display arbitrary job artifacts](../yaml/index.md#artifactsexpose_as) | Configure CI pipelines with the `artifacts:expose_as` parameter to directly link to selected [artifacts](../pipelines/job_artifacts.md) in merge requests. | +| [Display arbitrary job artifacts](../yaml/index.md#artifactsexpose_as) | Configure CI pipelines with the `artifacts:expose_as` parameter to directly link to selected [artifacts](../jobs/job_artifacts.md) in merge requests. | | [Unit test reports](unit_test_reports.md) | Configure your CI jobs to use Unit test reports, and let GitLab display a report on the merge request so that it's easier and faster to identify the failure without having to check the entire job log. | | [License Compliance](../../user/compliance/license_compliance/index.md) | Manage the licenses of your dependencies. | | [Metrics Reports](metrics_reports.md) | Display the Metrics Report on the merge request so that it's fast and easier to identify changes to important metrics. | diff --git a/doc/ci/testing/load_performance_testing.md b/doc/ci/testing/load_performance_testing.md index cf870bcec55..2897d7fe0ab 100644 --- a/doc/ci/testing/load_performance_testing.md +++ b/doc/ci/testing/load_performance_testing.md @@ -4,10 +4,15 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Load Performance Testing **(PREMIUM)** +<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> +# Load Performance Testing (deprecated) **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10683) in GitLab 13.2. +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/388723) in GitLab 15.9 +and is planned for removal in 17.0. This change is a breaking change. + With Load Performance Testing, you can test the impact of any pending code changes to your application's backend in [GitLab CI/CD](../index.md). @@ -161,7 +166,7 @@ For example: 1. In the `review` job: 1. Capture the dynamic URL and save it into a `.env` file, for example, `echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env`. - 1. Set the `.env` file to be a [job artifact](../pipelines/job_artifacts.md). + 1. Set the `.env` file to be a [job artifact](../jobs/job_artifacts.md). 1. In the `load_performance` job: 1. Set it to depend on the review job, so it inherits the environment file. 1. Set the `K6_DOCKER_OPTIONS` variable with the [Docker CLI option for environment files](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file), for example `--env-file review.env`. @@ -199,3 +204,5 @@ load_performance: rules: - if: $CI_COMMIT_BRANCH # Modify to match your pipeline rules, or use `only/except` if needed. ``` + +<!--- end_remove --> diff --git a/doc/ci/testing/test_coverage_visualization.md b/doc/ci/testing/test_coverage_visualization.md index 1b29304d1af..94ee4d2dce2 100644 --- a/doc/ci/testing/test_coverage_visualization.md +++ b/doc/ci/testing/test_coverage_visualization.md @@ -54,8 +54,8 @@ of times the line was checked by tests. Uploading a test coverage report does not enable: -- [Test coverage results in merge requests](../pipelines/settings.md#merge-request-test-coverage-results). -- [Code coverage history](../pipelines/settings.md#view-code-coverage-history). +- [Test coverage results in merge requests](code_coverage.md#view-code-coverage-results-in-the-mr). +- [Code coverage history](code_coverage.md#view-history-of-project-code-coverage). You must configure these separately. @@ -112,24 +112,14 @@ attempts to build the full path by: #### Path correction example -As an example, a project with: +As an example, a C# project with: -- A full path of `test-org/test-project`. +- A full path of `test-org/test-cs-project`. - The following files relative to the project root: ```shell Auth/User.cs Lib/Utils/User.cs - src/main/java - ``` - -In the: - -- Cobertura XML, the `filename` attribute in the `class` element assumes the value is a relative - path to the project's root: - - ```xml - <class name="packet.name" filename="src/main/java" line-rate="0.0" branch-rate="0.0" complexity="5"> ``` - `sources` from Cobertura XML, the following paths in the format @@ -137,8 +127,8 @@ In the: ```xml <sources> - <source>/builds/test-org/test-project/Auth</source> - <source>/builds/test-org/test-project/Lib/Utils</source> + <source>/builds/test-org/test-cs-project/Auth</source> + <source>/builds/test-org/test-cs-project/Lib/Utils</source> </sources> ``` @@ -153,6 +143,29 @@ The parser: 100 iterations. If it reaches this limit without finding a matching path in the file tree, the class is not included in the final coverage report. +Automatic class path correction also works for a Java project with: + +- A full path of `test-org/test-java-project`. +- The following files relative to the project root: + + ```shell + src/main/java/com/gitlab/security_products/tests/App.java + ``` + +- `sources` from Cobertura XML: + + ```xml + <sources> + <source>/builds/test-org/test-java-project/src/main/java/</source> + </sources> + ``` + +- `class` element with the `filename` value of `com/gitlab/security_products/tests/App.java`: + + ```xml + <class name="com.gitlab.security_products.tests.App" filename="com/gitlab/security_products/tests/App.java" line-rate="0.0" branch-rate="0.0" complexity="6.0"> + ``` + NOTE: Automatic class path correction only works on `source` paths in the format `<CI_BUILDS_DIR>/<PROJECT_FULL_PATH>/...`. The `source` is ignored if the path does not follow this pattern. The parser assumes that the diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index 393b128c658..b9e5dd87b24 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -187,7 +187,12 @@ In pipelines triggered with a trigger token, jobs are labeled as `triggered` in ## Troubleshooting -### `404 not found` when triggering a pipeline +### `403 Forbidden` when you trigger a pipeline with a webhook + +When you trigger a pipeline with a webhook, the API might return a `{"message":"403 Forbidden"}` response. +To avoid trigger loops, do not use [pipeline events](../../user/project/integrations/webhook_events.md#pipeline-events) to trigger pipelines. + +### `404 Not Found` when triggering a pipeline A response of `{"message":"404 Not Found"}` when triggering a pipeline might be caused by using a [personal access token](../../user/profile/personal_access_tokens.md) diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 30c067db36a..cf3af8f1810 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -74,7 +74,8 @@ and [templates](examples/index.md#cicd-templates). ### Documentation for pipeline types -Some pipeline types have their own detailed usage guides that you should read +Branch pipelines are the most basic type. +Other pipeline types have their own detailed usage guides that you should read if you are using that type: - [Multi-project pipelines](pipelines/downstream_pipelines.md#multi-project-pipelines): Have your pipeline trigger @@ -269,6 +270,7 @@ This message is shown if the [merge request pipeline](pipelines/merge_request_pi [merged results pipeline](pipelines/merged_results_pipelines.md), or [merge train pipeline](pipelines/merge_trains.md) has failed or been canceled. +This does not happen when a basic branch pipeline fails. If a merge request pipeline or merged result pipeline was canceled or failed, you can: diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 24eee4253a8..64a0f1038ad 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -25,12 +25,7 @@ so quoted and unquoted variables might be parsed differently. For example, `VAR1 is interpreted as an octal value, so the value becomes `5349`, but `VAR1: "012345"` is parsed as a string with a value of `012345`. -> For more information about advanced use of GitLab CI/CD: -> -> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Get to productivity faster with these [7 advanced GitLab CI workflow hacks](https://about.gitlab.com/webcast/7cicd-hacks/) -> shared by GitLab engineers. -> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Learn how the Cloud Native Computing Foundation (CNCF) [eliminates the complexity](https://about.gitlab.com/customers/cncf/) -> of managing projects across many cloud providers with GitLab CI/CD. +For more information about advanced use of GitLab CI/CD, see [7 advanced GitLab CI workflow hacks](https://about.gitlab.com/webcast/7cicd-hacks/) shared by GitLab engineers. ## Predefined CI/CD variables @@ -212,9 +207,6 @@ To add an instance variable: - **Mask variable** Optional. If selected, the variable's **Value** is not shown in job logs. The variable is not saved if the value does not meet the [masking requirements](#mask-a-cicd-variable). -The instance variables that are available in a project are listed in the project's -**Settings > CI/CD > Variables** section. - ## CI/CD variable security Code pushed to the `.gitlab-ci.yml` file could compromise your variables. Variables could @@ -477,7 +469,8 @@ To pass a job-created environment variable to other jobs: - Values can be wrapped in quotes, but cannot contain newline characters. 1. Save the `.env` file as an [`artifacts:reports:dotenv`](../yaml/artifacts_reports.md#artifactsreportsdotenv) artifact. -1. Jobs in later stages can then [use the variable in scripts](#use-cicd-variables-in-job-scripts). +1. Jobs in later stages can then [use the variable in scripts](#use-cicd-variables-in-job-scripts), + unless [jobs are configured not to receive `dotenv` variables](#control-which-jobs-receive-dotenv-variables). For example: @@ -749,7 +742,7 @@ export CI_JOB_ID="50" export CI_COMMIT_SHA="1ecfd275763eff1d6b4844ea3168962458c9f27a" export CI_COMMIT_SHORT_SHA="1ecfd275" export CI_COMMIT_REF_NAME="main" -export CI_REPOSITORY_URL="https://gitlab-ci-token:[masked]@example.com/gitlab-org/gitlab-foss.git" +export CI_REPOSITORY_URL="https://gitlab-ci-token:[masked]@example.com/gitlab-org/gitlab.git" export CI_COMMIT_TAG="1.0.0" export CI_JOB_NAME="spec:other" export CI_JOB_STAGE="test" @@ -759,11 +752,11 @@ export CI_JOB_TOKEN="[masked]" export CI_PIPELINE_ID="1000" export CI_PIPELINE_IID="10" export CI_PAGES_DOMAIN="gitlab.io" -export CI_PAGES_URL="https://gitlab-org.gitlab.io/gitlab-foss" +export CI_PAGES_URL="https://gitlab-org.gitlab.io/gitlab" export CI_PROJECT_ID="34" -export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-foss" -export CI_PROJECT_NAME="gitlab-foss" -export CI_PROJECT_TITLE="GitLab FOSS" +export CI_PROJECT_DIR="/builds/gitlab-org/gitlab" +export CI_PROJECT_NAME="gitlab" +export CI_PROJECT_TITLE="GitLab" ... ``` @@ -851,6 +844,10 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_SERVER_HOST=gitlab.com ++ export CI_SERVER_PORT=3000 ++ CI_SERVER_PORT=3000 +++ export CI_SERVER_SHELL_SSH_HOST=gitlab.com +++ CI_SERVER_SHELL_SSH_HOST=gitlab.com +++ export CI_SERVER_SHELL_SSH_PORT=22 +++ CI_SERVER_SHELL_SSH_PORT=22 ++ export CI_SERVER_PROTOCOL=https ++ CI_SERVER_PROTOCOL=https ++ export CI_SERVER_NAME=GitLab diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 45dc1a7fe37..3de36479de7 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -27,6 +27,7 @@ as it can cause the pipeline to behave unexpectedly. | `CHAT_USER_ID` | 14.4 | all | The chat service's user ID of the user who triggered the [ChatOps](../chatops/index.md) command. | | `CI` | all | 0.4 | Available for all jobs executed in CI/CD. `true` when available. | | `CI_API_V4_URL` | 11.7 | all | The GitLab API v4 root URL. | +| `CI_API_GRAPHQL_URL` | 15.11 | all | The GitLab API GraphQL root URL. | | `CI_BUILDS_DIR` | all | 11.10 | The top-level directory where builds are executed. | | `CI_COMMIT_AUTHOR` | 13.11 | all | The author of the commit in `Name <email>` format. | | `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch or tag. Is always `0000000000000000000000000000000000000000` in merge request pipelines and for the first commit in pipelines for branches or tags. | @@ -67,9 +68,9 @@ as it can cause the pipeline to behave unexpectedly. | `CI_HAS_OPEN_REQUIREMENTS` | 13.1 | all | Only available if the pipeline's project has an open [requirement](../../user/project/requirements/index.md). `true` when available. | | `CI_JOB_ID` | 9.0 | all | The internal ID of the job, unique across all jobs in the GitLab instance. | | `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the Docker image running the job. | -| `CI_JOB_JWT` | 12.10 | all | A RS256 JSON web token to authenticate with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). | -| `CI_JOB_JWT_V1` | 14.6 | all | The same value as `CI_JOB_JWT`. | -| `CI_JOB_JWT_V2` | 14.6 | all | A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. Format is subject to change. Be aware, the `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). **Note:** The `CI_JOB_JWT_V2` variable is available for testing, but the full feature is planned to be generally available when [issue 360657](https://gitlab.com/gitlab-org/gitlab/-/issues/360657) is complete.| +| `CI_JOB_JWT` (Deprecated) | 12.10 | all | A RS256 JSON web token to authenticate with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). [Deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and scheduled to be removed in GitLab 16.0. Use [ID tokens](../yaml/index.md#id_tokens) instead. | +| `CI_JOB_JWT_V1` (Deprecated) | 14.6 | all | The same value as `CI_JOB_JWT`. [Deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and scheduled to be removed in GitLab 16.0. Use [ID tokens](../yaml/index.md#id_tokens) instead. | +| `CI_JOB_JWT_V2` (Deprecated) | 14.6 | all | A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. The `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). [Deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and scheduled to be removed in GitLab 16.0. Use [ID tokens](../yaml/index.md#id_tokens) instead. | | `CI_JOB_MANUAL` | 8.12 | all | Only available if the job was started manually. `true` when available. | | `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job. | | `CI_JOB_NAME_SLUG` | 15.4 | all | `CI_JOB_NAME_SLUG` in lowercase, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in paths. | @@ -121,6 +122,8 @@ as it can cause the pipeline to behave unexpectedly. | `CI_SERVER_NAME` | all | all | The name of CI/CD server that coordinates jobs. | | `CI_SERVER_PORT` | 12.8 | all | The port of the GitLab instance URL, without host or protocol. For example `8080`. | | `CI_SERVER_PROTOCOL` | 12.8 | all | The protocol of the GitLab instance URL, without host or port. For example `https`. | +| `CI_SERVER_SHELL_SSH_HOST` | 15.11 | all | The SSH host of the GitLab instance, used for access to Git repositories via SSH. For example `gitlab.com`. | +| `CI_SERVER_SHELL_SSH_PORT` | 15.11 | all | The SSH port of the GitLab instance, used for access to Git repositories via SSH. For example `22`. | | `CI_SERVER_REVISION` | all | all | GitLab revision that schedules jobs. | | `CI_SERVER_TLS_CA_FILE` | all | all | File containing the TLS CA certificate to verify the GitLab server when `tls-ca-file` set in [runner settings](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | | `CI_SERVER_TLS_CERT_FILE` | all | all | File containing the TLS certificate to verify the GitLab server when `tls-cert-file` set in [runner settings](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | @@ -139,6 +142,7 @@ as it can cause the pipeline to behave unexpectedly. | `GITLAB_USER_ID` | 8.12 | all | The ID of the user who started the pipeline, unless the job is a manual job. In manual jobs, the value is the ID of the user who started the job. | | `GITLAB_USER_LOGIN` | 10.0 | all | The username of the user who started the pipeline, unless the job is a manual job. In manual jobs, the value is the username of the user who started the job. | | `GITLAB_USER_NAME` | 10.0 | all | The name of the user who started the pipeline, unless the job is a manual job. In manual jobs, the value is the name of the user who started the job. | +| `KUBECONFIG` | 14.2 | all | The path to the `kubeconfig` file with contexts for every shared agent connection. Only available when a [GitLab agent is authorized to access the project](../../user/clusters/agent/ci_cd_workflow.md#authorize-the-agent). | | `TRIGGER_PAYLOAD` | 13.9 | all | The webhook payload. Only available when a pipeline is [triggered with a webhook](../triggers/index.md#use-a-webhook-payload). | ## Predefined variables for merge request pipelines diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index 8740cf100ac..c4e752551de 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -209,7 +209,7 @@ The exceptions to the [original dotenv rules](https://github.com/motdotla/dotenv - The `.env` file can't have empty lines or comments (starting with `#`). - Key values in the `env` file cannot have spaces or newline characters (`\n`), including when using single or double quotes. - Quote escaping during parsing (`key = 'value'` -> `{key: "value"}`) is not supported. -- Only UTF-8 encoding is [supported](../pipelines/job_artifacts.md#error-message-fatal-invalid-argument-when-uploading-a-dotenv-artifact-on-a-windows-runner). +- Only UTF-8 encoding is [supported](../jobs/job_artifacts_troubleshooting.md#error-message-fatal-invalid-argument-when-uploading-a-dotenv-artifact-on-a-windows-runner). ## `artifacts:reports:junit` diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index 6d34a3034f7..94148400896 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# GitLab CI/CD include examples **(FREE)** +# Use CI/CD configuration from other files **(FREE)** You can use [`include`](index.md#include) to include external YAML files in your CI/CD jobs. @@ -405,7 +405,8 @@ see this [CI/CD variable demo](https://youtu.be/4XR8gw3Pkos). > - Introduced in GitLab 14.2 [with a flag](../../administration/feature_flags.md) named `ci_include_rules`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.3. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.4. Feature flag `ci_include_rules` removed. -> - [Support for `exists` keyword added](https://gitlab.com/gitlab-org/gitlab/-/issues/341511) in GitLab 14.5. +> - Support for `exists` keyword [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/341511) in GitLab 14.5. +> - Support for `needs` job dependency [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345377) in GitLab 15.11. You can use [`rules`](index.md#rules) with `include` to conditionally include other configuration files. @@ -415,11 +416,6 @@ these keywords: - [`rules:if`](index.md#rulesif). - [`rules:exists`](index.md#rulesexists). -You cannot use [`needs:`](index.md#needs) to create a job dependency that points to -a job added with `include:local:rules`. When the configuration is validated, -GitLab returns `undefined need: <job-name>`. [Issue 345377](https://gitlab.com/gitlab-org/gitlab/-/issues/345377) -proposes improving this behavior. - ### `include` with `rules:if` Use [`rules:if`](index.md#rulesif) to conditionally include other configuration files @@ -507,6 +503,88 @@ When the pipeline runs, GitLab: include: 'configs/**/*.yml' ``` +## Define inputs for configuration added with `include` (Beta) + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/391331) in GitLab 15.11 as a Beta feature. + +FLAG: +`spec` and `with` are experimental [Open Beta features](../../policy/alpha-beta-support.md#beta) +and subject to change without notice. + +### Define input parameters with `spec:inputs` + +Use `spec:inputs` to define input parameters for CI/CD configuration intended to be added +to a pipeline with `include`. Use [`include:with`](#set-input-parameter-values-with-includewith) +to define the values to use when the pipeline runs. + +The specs must be declared at the top of the configuration file, in a header section. +Separate the header from the rest of the configuration with `---`. + +Use the interpolation format `$[[ input.input-id ]]` to reference the values outside of the header section. +The inputs are evaluated and interpolated once, when the configuration is fetched +during pipeline creation, but before the configuration is merged with the contents of the `.gitlab-ci.yml`. + +```yaml +spec: + inputs: + environment: + job-stage: +--- + +scan-website: + stage: $[[ inputs.job-stage ]] + script: ./scan-website $[[ inputs.environment ]] +``` + +When using `spec:inputs`: + +- Defined inputs are mandatory by default. +- Inputs can be made optional by specifying a `default`. Use `default: null` to have no default value. +- A string containing an interpolation block must not exceed 1 MB. +- The string inside an interpolation block must not exceed 1 KB. + +For example, a `custom_configuration.yml`: + +```yaml +spec: + inputs: + website: + user: + default: 'test-user' + flags: + default: null +--- + +# The pipeline configuration would follow... +``` + +In this example: + +- `website` is mandatory and must be defined. +- `user` is optional. If not defined, the value is `test-user`. +- `flags` is optional. If not defined, it has no value. + +### Set input parameter values with `include:with` + +Use `include:with` to set the values for the parameters when the included configuration +is added to the pipeline. + +For example, to include a `custom_configuration.yml` that has the same specs +as the [example above](#define-input-parameters-with-specinputs): + +```yaml +include: + - local: 'custom_configuration.yml' + with: + website: "My website" +``` + +In this example: + +- `website` has a value of `My website` for the included configuration. +- `user` has a value of `test-user`, because that is the default when not specified. +- `flags` has no value, because it is optional and has no default when not specified. + ## Troubleshooting ### `Maximum of 150 nested includes are allowed!` error diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index 6ba9d455278..681e1937c26 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -733,7 +733,7 @@ test_job_2: ### `artifacts` -Use `artifacts` to specify which files to save as [job artifacts](../pipelines/job_artifacts.md). +Use `artifacts` to specify which files to save as [job artifacts](../jobs/job_artifacts.md). Job artifacts are a list of files and directories that are attached to the job when it [succeeds, fails, or always](#artifactswhen). @@ -751,7 +751,7 @@ artifacts from the jobs defined in the `needs` configuration. Job artifacts are only collected for successful jobs by default, and artifacts are restored after [caches](#cache). -[Read more about artifacts](../pipelines/job_artifacts.md). +[Read more about artifacts](../jobs/job_artifacts.md). #### `artifacts:paths` @@ -790,7 +790,7 @@ This example creates an artifact with `.config` and all the files in the `binari **Related topics**: - To restrict which jobs a specific job fetches artifacts from, see [`dependencies`](#dependencies). -- [Create job artifacts](../pipelines/job_artifacts.md#create-job-artifacts). +- [Create job artifacts](../jobs/job_artifacts.md#create-job-artifacts). #### `artifacts:exclude` @@ -829,7 +829,7 @@ subdirectories of `binaries/`. **Related topics**: -- [Exclude files from job artifacts](../pipelines/job_artifacts.md#exclude-files-from-job-artifacts). +- [Exclude files from job artifacts](../jobs/job_artifacts.md#without-excluded-files). #### `artifacts:expire_in` @@ -839,12 +839,12 @@ subdirectories of `binaries/`. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276583) in GitLab 13.9, keeping latest job artifacts can be disabled instance-wide. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321323) in GitLab 13.12, the latest pipeline artifacts are kept regardless of expiry time. -Use `expire_in` to specify how long [job artifacts](../pipelines/job_artifacts.md) are stored before +Use `expire_in` to specify how long [job artifacts](../jobs/job_artifacts.md) are stored before they expire and are deleted. The `expire_in` setting does not affect: -- Artifacts from the latest job, unless keeping the latest job artifacts is: - - [Disabled at the project level](../pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). - - [Disabled instance-wide](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). +- Artifacts from the latest job, unless keeping the latest job artifacts is disabled + [at the project level](../jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). + or [instance-wide](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). - [Pipeline artifacts](../pipelines/pipeline_artifacts.md). You can't specify an expiration date for pipeline artifacts. See [When pipeline artifacts are deleted](../pipelines/pipeline_artifacts.md#when-pipeline-artifacts-are-deleted) for more information. @@ -890,7 +890,7 @@ job: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15018) in GitLab 12.5. Use the `artifacts:expose_as` keyword to -[expose job artifacts in the merge request UI](../pipelines/job_artifacts.md#expose-job-artifacts-in-the-merge-request-ui). +[expose job artifacts in the merge request UI](../jobs/job_artifacts.md#link-to-job-artifacts-in-the-merge-request-ui). **Keyword type**: Job keyword. You can use it only as part of a job or in the [`default` section](#default). @@ -916,7 +916,7 @@ test: - A maximum of 10 job artifacts per merge request can be exposed. - Glob patterns are unsupported. - If a directory is specified and there is more than one file in the directory, - the link is to the job [artifacts browser](../pipelines/job_artifacts.md#download-job-artifacts). + the link is to the job [artifacts browser](../jobs/job_artifacts.md#download-job-artifacts). - If [GitLab Pages](../../administration/pages/index.md) is enabled, GitLab automatically renders the artifacts when the artifacts is a single file with one of these extensions: - `.html` or `.htm` @@ -927,7 +927,7 @@ test: **Related topics**: -- [Expose job artifacts in the merge request UI](../pipelines/job_artifacts.md#expose-job-artifacts-in-the-merge-request-ui). +- [Expose job artifacts in the merge request UI](../jobs/job_artifacts.md#link-to-job-artifacts-in-the-merge-request-ui). #### `artifacts:name` @@ -958,7 +958,7 @@ job: **Related topics**: -- [Use CI/CD variables to define the artifacts name](../pipelines/job_artifacts.md#use-cicd-variables-to-define-the-artifacts-name). +- [Use CI/CD variables to define the artifacts name](../jobs/job_artifacts.md#with-a-dynamically-defined-name). #### `artifacts:public` @@ -1053,7 +1053,7 @@ job: **Related topics**: -- [Add untracked files to artifacts](../pipelines/job_artifacts.md#add-untracked-files-to-artifacts). +- [Add untracked files to artifacts](../jobs/job_artifacts.md#with-untracked-files). #### `artifacts:when` @@ -1512,6 +1512,7 @@ In this example: **Additional details**: +- You can find parse examples in [Code Coverage](../testing/code_coverage.md#test-coverage-examples). - If there is more than one matched line in the job output, the last line is used (the first result of reverse search). - If there are multiple matches in a single line, the last match is searched @@ -1626,7 +1627,7 @@ the [stage](#stages) precedence. - The job status does not matter. If a job fails or it's a manual job that isn't triggered, no error occurs. - If the artifacts of a dependent job are [expired](#artifactsexpire_in) or - [deleted](../pipelines/job_artifacts.md#delete-job-artifacts), then the job fails. + [deleted](../jobs/job_artifacts.md#delete-job-log-and-artifacts), then the job fails. ### `environment` @@ -1734,7 +1735,7 @@ Use the `action` keyword to specify how the job interacts with the environment. |:----------|:----------------| | `start` | Default value. Indicates that the job starts the environment. The deployment is created after the job starts. | | `prepare` | Indicates that the job is only preparing the environment. It does not trigger deployments. [Read more about preparing environments](../environments/index.md#access-an-environment-for-preparation-or-verification-purposes). | -| `stop` | Indicates that the job stops an environment. [Read more about stopping an environment](../environments/index.md#stop-an-environment). | +| `stop` | Indicates that the job stops an environment. [Read more about stopping an environment](../environments/index.md#stopping-an-environment). | | `verify` | Indicates that the job is only verifying the environment. It does not trigger deployments. [Read more about verifying environments](../environments/index.md#access-an-environment-for-preparation-or-verification-purposes). | | `access` | Indicates that the job is only accessing the environment. It does not trigger deployments. [Read more about accessing environments](../environments/index.md#access-an-environment-for-preparation-or-verification-purposes). | @@ -2556,11 +2557,10 @@ can use that variable in `needs:pipeline` to download artifacts from the parent To need a job that sometimes does not exist in the pipeline, add `optional: true` to the `needs` configuration. If not defined, `optional: false` is the default. -Jobs that use [`rules`](#rules), [`only`, or `except`](#only--except) might not always -be added to a pipeline. GitLab checks the `needs` relationships before starting a -pipeline: +Jobs that use [`rules`](#rules), [`only`, or `except`](#only--except) and that are added with [`include`](#include) +might not always be added to a pipeline. GitLab checks the `needs` relationships before starting a pipeline: -- If the needs entry has `optional: true` and the needed job is present in the pipeline, +- If the `needs` entry has `optional: true` and the needed job is present in the pipeline, the job waits for it to complete before starting. - If the needed job is not present, the job can start when all other needs requirements are met. - If the `needs` section contains only optional jobs, and none are added to the pipeline, @@ -2613,9 +2613,9 @@ In this example: #### `needs:pipeline` -You can mirror the pipeline status from an upstream pipeline to a bridge job by +You can mirror the pipeline status from an upstream pipeline to a job by using the `needs:pipeline` keyword. The latest pipeline status from the default branch is -replicated to the bridge job. +replicated to the job. **Keyword type**: Job keyword. You can use it only as part of a job. @@ -2628,7 +2628,7 @@ replicated to the bridge job. **Example of `needs:pipeline`**: ```yaml -upstream_bridge: +upstream_status: stage: test needs: pipeline: other/project @@ -3501,6 +3501,7 @@ docker build: - If the pipeline is a merge request pipeline, check `Dockerfile` for changes. - If `Dockerfile` has changed, add the job to the pipeline as a manual job, and the pipeline continues running even if the job is not triggered (`allow_failure: true`). +- A maximum of 50 patterns or file paths can be defined per `rules:changes` section. - If `Dockerfile` has not changed, do not add job to any pipeline (same as `when: never`). - [`rules:changes:paths`](#ruleschangespaths) is the same as `rules:changes` without any subkeys. @@ -3613,11 +3614,12 @@ job: - Glob patterns are interpreted with Ruby [`File.fnmatch`](https://docs.ruby-lang.org/en/2.7.0/File.html#method-c-fnmatch) with the flags `File::FNM_PATHNAME | File::FNM_DOTMATCH | File::FNM_EXTGLOB`. -- For performance reasons, GitLab matches a maximum of 10,000 `exists` patterns or - file paths. After the 10,000th check, rules with patterned globs always match. - In other words, `exists` always reports `true` if more than 10,000 checks - run. Repositories with less than 10,000 files might still be impacted if the `exists` - rules are checked more than 10,000 times. +- For performance reasons, GitLab performs a maximum of 10,000 checks against + `exists` patterns or file paths. After the 10,000th check, rules with patterned + globs always match. In other words, the `exists` rule always assumes a match in + projects with more than 10,000 files, or if there are fewer than 10,000 files but + the `exists` rules are checked more than 10,000 times. +- A maximum of 50 patterns or file paths can be defined per `rules:exists` section. - `exists` resolves to `true` if any of the listed files are found (an `OR` operation). #### `rules:allow_failure` @@ -3963,9 +3965,9 @@ If `stage` is not defined, the job uses the `test` stage by default. **Keyword type**: Job keyword. You can use it only as part of a job. -**Possible inputs**: An array including any number of stage names. Stage names can be: +**Possible inputs**: A string, which can be a: -- The [default stages](#stages). +- [Default stage](#stages). - User-defined stages. **Example of `stage`**: diff --git a/doc/cloud_seed/index.md b/doc/cloud_seed/index.md index 9ce18950506..7ac38ef2c15 100644 --- a/doc/cloud_seed/index.md +++ b/doc/cloud_seed/index.md @@ -1,5 +1,5 @@ --- -stage: Configure +stage: Deploy group: Incubation info: Cloud Seed (formerly 5mp) is a GitLab Incubation Engineering program. No technical writer assigned to this group. --- diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md index d29d9b9651f..3ce794514a3 100644 --- a/doc/development/api_graphql_styleguide.md +++ b/doc/development/api_graphql_styleguide.md @@ -40,6 +40,13 @@ GraphiQL is an interactive GraphQL API explorer where you can play around with e You can access it in any GitLab environment on `https://<your-gitlab-site.com>/-/graphql-explorer`. For example, the one for [GitLab.com](https://gitlab.com/-/graphql-explorer). +## Reviewing merge requests with GraphQL changes + +The GraphQL framework has some specific gotchas to be aware of, and domain expertise is required to ensure they are satisfied. + +If you are asked to review a merge request that modifies any GraphQL files or adds an endpoint, please have a look at +[our GraphQL review guide](graphql_guide/reviewing.md). + ## Authentication Authentication happens through the `GraphqlController`, right now this @@ -287,13 +294,13 @@ or by calling `#to_global_id` on an object that has mixed in the `GlobalID::Identification` module. Using an example from -[`Types::Notes::DiscussionType`](https://gitlab.com/gitlab-org/gitlab/-/blob/3c95bd9/app/graphql/types/notes/discussion_type.rb#L24-26): +[`Types::Notes::DiscussionType`](https://gitlab.com/gitlab-org/gitlab/-/blob/af48df44/app/graphql/types/notes/discussion_type.rb#L22-30): ```ruby -field :reply_id, GraphQL::Types::ID +field :reply_id, Types::GlobalIDType[Discussion] def reply_id - ::Gitlab::GlobalId.build(object, id: object.reply_id) + Gitlab::GlobalId.build(object, id: object.reply_id) end ``` @@ -785,7 +792,7 @@ The documentation mentions that the old Global ID style is now deprecated. ## Mark schema items as Alpha You can mark GraphQL schema items (fields, arguments, enum values, and mutations) as -[Alpha](../policy/alpha-beta-support.md#alpha-features). +[Alpha](../policy/alpha-beta-support.md#experiment). An item marked as Alpha is [exempt from the deprecation process](#breaking-change-exemptions) and can be removed at any time without notice. Mark an item as Alpha when it is @@ -1132,7 +1139,9 @@ you need to do something more custom however, remember, if you encounter an object the `current_user` does not have access to when resolving a field, then the entire field should resolve to `null`. -### Deriving resolvers (`BaseResolver.single` and `BaseResolver.last`) +### Deriving resolvers + +(including `BaseResolver.single` and `BaseResolver.last`) For some use cases, we can derive resolvers from others. The main use case for this is one resolver to find all items, and another to diff --git a/doc/development/api_styleguide.md b/doc/development/api_styleguide.md index 828ef21ba9a..34e043a30d6 100644 --- a/doc/development/api_styleguide.md +++ b/doc/development/api_styleguide.md @@ -338,12 +338,13 @@ Also see [verifying N+1 performance](#verifying-with-tests) in tests. When throwing an error with a message that is meant to be user-facing, you should use the error message utility function contained in `lib/gitlab/utils/error_message.rb`. It adds a prefix to the error message, making it distinguishable from non-user-facing error messages. -Please make sure that the Frontend is aware of the prefix usage and is using the according utils. ```ruby Gitlab::Utils::ErrorMessage.to_user_facing('Example user-facing error-message') ``` +Please make sure that the Frontend is aware of the prefix usage and is using the according utils. See [Error handling](fe_guide/style/javascript.md#error-handling) in JavaScript style guide for more information. + ## Include a changelog entry All client-facing changes **must** include a [changelog entry](changelog.md). diff --git a/doc/development/application_secrets.md b/doc/development/application_secrets.md index 526cc6c3f61..e640a94f0a6 100644 --- a/doc/development/application_secrets.md +++ b/doc/development/application_secrets.md @@ -17,7 +17,7 @@ This page is a development guide for application secrets. |`db_key_base` | The base key to encrypt the data for `attr_encrypted` columns | |`openid_connect_signing_key` | The signing key for OpenID Connect | | `encrypted_settings_key_base` | The base key to encrypt settings files with | -| `ci_jwt_signing_key` | The base key for encrypting the `CI_JOB_JWT` and `CI_JOB_JWT_V2` predefined CI/CD variables | +| `ci_jwt_signing_key` | The base key for encrypting the `CI_JOB_JWT` and `CI_JOB_JWT_V2` predefined CI/CD variables. `CI_JOB_JWT` and `CI_JOB_JWT_V2` were [deprecated in GitLab 15.9](../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and are scheduled to be removed in GitLab 16.0. | ## Where the secrets are stored diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 7a9c3572406..d1e16b7f137 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -558,7 +558,7 @@ GitLab CI/CD is the open-source continuous integration service included with Git #### GitLab Workhorse -- [Project page](https://gitlab.com/gitlab-org/gitlab-workhorse/blob/master/README.md) +- [Project page](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/workhorse/index.md) - Configuration: - [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template) - [Charts](https://docs.gitlab.com/charts/charts/gitlab/webservice/) @@ -567,7 +567,7 @@ GitLab CI/CD is the open-source continuous integration service included with Git - Process: `gitlab-workhorse` - GitLab.com: [Service Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture) -[GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) is a program designed at GitLab to help alleviate pressure from Puma. You can read more about the [historical reasons for developing](https://about.gitlab.com/blog/2016/04/12/a-brief-history-of-gitlab-workhorse/). It's designed to act as a smart reverse proxy to help speed up GitLab as a whole. +[GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/workhorse) is a program designed at GitLab to help alleviate pressure from Puma. You can read more about the [historical reasons for developing](https://about.gitlab.com/blog/2016/04/12/a-brief-history-of-gitlab-workhorse/). It's designed to act as a smart reverse proxy to help speed up GitLab as a whole. #### Grafana diff --git a/doc/development/auto_devops.md b/doc/development/auto_devops.md index b2281f9c032..1f98a37ac9d 100644 --- a/doc/development/auto_devops.md +++ b/doc/development/auto_devops.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/chatops_on_gitlabcom.md b/doc/development/chatops_on_gitlabcom.md index 378639c4a43..002470f7325 100644 --- a/doc/development/chatops_on_gitlabcom.md +++ b/doc/development/chatops_on_gitlabcom.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/code_owners/index.md b/doc/development/code_owners/index.md index fa0385777e8..d962a36b497 100644 --- a/doc/development/code_owners/index.md +++ b/doc/development/code_owners/index.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/219916) in GitLab 15.10. This document was created to help contributors understand the code design of -[Code Owners](../../user/project/code_owners.md). You should read this +[Code Owners](../../user/project/codeowners/index.md). You should read this document before making changes to the code for this feature. This document is intentionally limited to an overview of how the code is @@ -28,6 +28,15 @@ namespace. Code Owners is an EE-only feature, so the files only exist in the `./ - `Gitlab::CodeOwners`: the main module used to interact with the code owner rules. - Defined in `./ee/lib/gitlab/code_owners.rb`. +- `Gitlab::CodeOwners::File`: wraps a `CODEOWNERS` file and exposes the data through + the class' public methods. + - Defined in `./ee/lib/gitlab/code_owners/file.rb`. +- `Gitlab::CodeOwners::Section`: wraps a section heading from a + `CODEOWNERS` file and parses the different parts. + - Defined in `./ee/lib/gitlab/code_owners/section.rb`. +- `Gitlab::CodeOwners::Entry`: wraps an entry (a pattern and owners line) in a + `CODEOWNERS` file and exposes the data through the class' public methods. + - Defined in `./ee/lib/gitlab/code_owners/entry.rb`. - `Gitlab::CodeOwners::Loader`: finds the correct `CODEOWNER` file and loads the content into a `Gitlab::CodeOwners::File` instance. - Defined in `./ee/lib/gitlab/code_owners/loader.rb`. @@ -40,12 +49,6 @@ namespace. Code Owners is an EE-only feature, so the files only exist in the `./ - `Gitlab::CodeOwners::GroupsLoader`: finds the correct `CODEOWNER` file and loads the content into a `Gitlab::CodeOwners::File` instance. - Defined in `./ee/lib/gitlab/code_owners/groups_loader.rb`. -- `Gitlab::CodeOwners::File`: wraps a `CODEOWNERS` file and exposes the data through - the class' public methods. - - Defined in `./ee/lib/gitlab/code_owners/file.rb`. -- `Gitlab::CodeOwners::Entry`: wraps an entry (a pattern and owners line) in a - `CODEOWNERS` file and exposes the data through the class' public methods. - - Defined in `./ee/lib/gitlab/code_owners/entry.rb`. - `Gitlab::CodeOwners::Validator`: validates no files in the `CODEOWNERS` entries have been changed when a user pushes to a protected branch with `require_code_owner_approval` enabled. - Defined in `./ee/lib/gitlab/code_owners/validator.rb`. diff --git a/doc/development/code_review.md b/doc/development/code_review.md index ed9404187ae..5588afe0713 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -546,7 +546,9 @@ people who add commits to an MR are not authorized to approve or merge the MR an must seek a maintainer who has not contributed to the MR to approve and merge it. This policy is in place to satisfy the CHG-04 control of the GitLab -[Change Management Controls](https://about.gitlab.com/handbook/security/security-assurance/security-compliance/guidance/change-management.html). +[Change Management Controls](https://about.gitlab.com/handbook/security/change-management-policy.html). + +<!-- Or should it link to: https://about.gitlab.com/handbook/engineering/infrastructure/change-management/ ? --> To implement this policy in `gitlab-org/gitlab`, we have enabled the following settings to ensure MRs get an approval from a top-level CODEOWNERS maintainer: diff --git a/doc/development/contributing/design.md b/doc/development/contributing/design.md index 210370f9816..d68bc194266 100644 --- a/doc/development/contributing/design.md +++ b/doc/development/contributing/design.md @@ -73,7 +73,7 @@ Check visual design properties using your browser's _elements inspector_ ([Chrom guidelines. - _Optionally_ consider [dark mode](../../user/profile/preferences.md#dark-mode). [^1] - [^1]: You're not required to design for [dark mode](../../user/profile/preferences.md#dark-mode) while the feature is in [alpha](../../policy/alpha-beta-support.md#alpha-features). The [UX Foundations team](https://about.gitlab.com/direction/manage/foundations/) plans to improve the dark mode in the future. Until we integrate [Pajamas](https://design.gitlab.com/) components into the product and the underlying design strategy is in place to support dark mode, we cannot guarantee that we won't introduce bugs and debt to this mode. At your discretion, evaluate the need to create dark mode patches. + [^1]: You're not required to design for [dark mode](../../user/profile/preferences.md#dark-mode) while the feature is an [Experiment](../../policy/alpha-beta-support.md#experiment). The [UX Foundations team](https://about.gitlab.com/direction/manage/foundations/) plans to improve the dark mode in the future. Until we integrate [Pajamas](https://design.gitlab.com/) components into the product and the underlying design strategy is in place to support dark mode, we cannot guarantee that we won't introduce bugs and debt to this mode. At your discretion, evaluate the need to create dark mode patches. ### States diff --git a/doc/development/contributing/first_contribution.md b/doc/development/contributing/first_contribution.md new file mode 100644 index 00000000000..0c4b5b21171 --- /dev/null +++ b/doc/development/contributing/first_contribution.md @@ -0,0 +1,340 @@ +--- +stage: none +group: Development +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Tutorial: Make a GitLab contribution + +Anyone can contribute to the development of GitLab. + +Maybe you want to add functionality that you feel is missing. Or maybe +you noticed some UI text that you want to improve. + +This tutorial will walk you through the process of updating UI text +and related files by using the GitLab Development Kit and the GitLab community fork. +You can follow this example exactly to familiarize yourself with the process, +or you can choose other UI text to update. + +## Steps + +To make a contribution, you will: + +1. [Configure the GitLab Development Kit](#step-1-configure-the-gitlab-development-kit) +1. [Make your code updates](#step-2-change-the-code) +1. [Push your changes to the community fork](#step-3-push-your-changes-to-the-community-fork) +1. [Create a merge request](#step-4-create-a-merge-request) + +## Prerequisites + +On your local machine: + +- Ensure Git is installed. + (From the command line, type `git -v`. If you get a result, you have Git installed.) +- Install a source code editor, or decide which tool you're going to use to edit files. + +On GitLab.com: + +- Create an account. Ensure you can successfully sign in. +- Go to the [`gitlab-community/community-members` group](https://gitlab.com/gitlab-community/community-members) + and select **Request access**. This action will give you access to the GitLab + community fork, where you'll author your changes. + +## Step 1: Configure the GitLab Development Kit + +The GitLab Development Kit (GDK) is a local version of GitLab that's yours to play with. +It's just like an installation of self-managed GitLab. It includes sample projects you +can use to test functionality, and it gives you access to administrator functionality. +You can run it on your local machine, or use GitPod to run a remote version. + + + +If you've never used the GDK before, and you think you might contribute +more than once, you should install it. +If you already have a working GDK, you should +[update it to use the community fork](#an-existing-gdk-installation). + +### A new GDK installation + +Set aside about two hours to install the GDK. If all goes smoothly, it +should take about an hour to install. + +Sometimes the installation needs some tweaks to make it work, so you should +also set aside some time for troubleshooting. +It might seem like a lot of work, but after you have the GDK running, +you'll be able to contribute much more often and more easily. + +To install the GDK: + +1. Ensure you're on + [one of the supported platforms](https://gitlab.com/gitlab-org/gitlab-development-kit/-/tree/main/#supported-platforms) + (macOS, Ubuntu, etc.). +1. Choose the directory where you want to install the GDK. + In this location, a repository called `gitlab-development-kit` will be created, + and the application will be installed. +1. From the command line, go to that directory. + In this example, we will use the `development` directory. + + ```shell + cd development + ``` + +1. Run the one-line installation command: + + ```shell + curl "https://gitlab.com/gitlab-org/gitlab-development-kit/-/raw/main/support/install" | bash + ``` + +1. For the message `Where would you like to install the GDK? [./gitlab-development-kit]`, + press Enter to accept the default location. + +1. For the message `Which GitLab repo URL would you like to clone?`, enter the GitLab community fork: + + ```shell + https://gitlab.com/gitlab-community/gitlab.git + ``` + +While the installation is running, copy any messages that are displayed. +If you have any problems with the installation, you can use this output as +part of troubleshooting. + +When the installation is complete: + +1. Go to the directory where the GDK was installed: + + ```shell + cd gitlab-development-kit + ``` + +1. Start the GDK: + + ```shell + gdk start + ``` + +1. Connect to the GDK by using the URL provided. It should be something like <http://127.0.0.1:3000>. +1. Use the username `root` and the password `5iveL!fe`. You will be prompted + to reset your password the first time you sign in. + +If you have any problems, try going to the `gitlab-development-kit/gitlab` +directory and running these commands: + +```shell +yarn install && bundle install +bundle exec rails db:migrate RAILS_ENV=development +``` + +From the `gitlab-development-kit` folder, you can also try running `gdk doctor`. + +For more advanced troubleshooting, see +[the troubleshooting docs](https://gitlab.com/gitlab-org/gitlab-development-kit/-/tree/main/doc/troubleshooting). + +### An existing GDK installation + +If you have an existing GDK installation, you should update it so it's +using the community fork. + +1. Delete the existing `gitlab-development-kit/gitlab` directory. +1. Clone the community fork into that location: + + ```shell + cd gitlab-development-kit + git clone https://gitlab.com/gitlab-community/gitlab.git + ``` + +To confirm it was successful: + +1. Ensure the `gitlab-development-kit/gitlab` directory exists. +1. Go to the top `gitlab-development-kit` directory and run `gdk stop` and `gdk start`. + +If you get errors, run `gdk doctor` to troubleshoot. For more advanced troubleshooting, see +[the troubleshooting docs](https://gitlab.com/gitlab-org/gitlab-development-kit/-/tree/main/doc/troubleshooting). + +## Step 2: Change the code + +Now for the fun part. Let's edit some code. + +In this example, I found some UI text I'd like to change. +In the upper-right corner in GitLab, I selected my avatar and then **Preferences**. +I want to change this text: + + + +Other settings on the page start with the word `Customize` and skip the `This setting allows you to` part. +I'll update this phrase to match the others. + +1. Search the `gitlab-development-kit/gitlab` directory for the string `This setting allows you to customize`. + + The results show one `.haml` file, two `.md` files, one `.pot` file, and + several `.po` files. + +1. Open the `.haml` file. This file is where the UI text resides. +1. Update the string. In this case, I'll remove the words before `customize` + and start the word `customize` with a capital `C`. +1. Save the file. + +You can check that you were successful: + +- In the `gitlab-development-kit/gitlab` directory, type `git status` + to show the file you modified: + + ```shell + modified: app/views/profiles/preferences/show.html.haml + ``` + +- Refresh the web browser where you're viewing the GDK. + The changes should be displayed. Take a screenshot. + +  + +### Update the translation files + +English UI strings are localized into many languages. +These strings are saved in a `.pot` file, which you must update +any time you update UI text. + +To generate the localization file: + +1. Ensure you are in the `gitlab-development-kit/gitlab` directory. +1. Run the following command: + + ```shell + bin/rake gettext:compile + ``` + + After several minutes, a `.pot` file is generated in the `/locale` directory. + +Now, in the `gitlab-development-kit/gitlab` directory, if you type `git status` +you should have both files listed: + +```shell + modified: app/views/profiles/preferences/show.html.haml + modified: locale/gitlab.pot +``` + +For more information about localization, see [internationalization](../i18n/externalization.md). + +### Update the documentation + +Documentation for GitLab is published on <https://docs.gitlab.com>. +When you add or update a feature, you must update the docs as well. + +1. To find the documentation for a feature, the easiest thing is to search the + docs site. In this case, the setting is described on this documentation page: + + ```plaintext + https://docs.gitlab.com/ee/user/profile/preferences.html + ``` + +1. The URL shows you the location of the file in the `/doc` directory. + In this case, the location is: + + ```plaintext + doc/user/profile/preferences.md + ``` + +1. Go to this location in your local `gitlab` repository and update the `.md` file + and any related images. + +Now when you run `git status`, you should have something like: + +```plaintext + modified: app/views/profiles/preferences/show.html.haml + modified: doc/user/profile/img/profile-preferences-syntax-themes.png + modified: doc/user/profile/preferences.md + modified: locale/gitlab.pot +``` + +To view these changes in action, you can +[check out a merge request where these changes have already been made](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116472). + +## Step 3: Push your changes to the community fork + +Now you're going to push your changes to the community fork. This is the next step +in getting your changes put into the main GitLab repository. + +1. Ensure you are in the `gitlab-development-kit/gitlab` directory. +1. Create a branch. You don't want to work in the `master` branch. + Instead, you want to create a branch for your work. In this example, + we're going to call the branch `ui-updates`. + + ```shell + git checkout -b ui-updates + ``` + +1. Add the files to the staging area. + + ```shell + git add . + ``` + +1. Provide a commit message. GitLab has somewhat strict + [commit message guidelines](merge_request_workflow.md#commit-messages-guidelines). + To be safe, a general rule is to use three to five words, start with a capital letter, + and do **not** end with a period. + + ```shell + git commit -m "Updating UI text + + Standardizing the text on this page so + that each area uses consistent language. + + Changelog: changed" + ``` + + The `Changelog: changed` is because we're changing an existing feature. If we were adding a feature, we'd + use `Changelog: added`. For details, see [changelog entries](../changelog.md). + +1. Push the changes to the community fork. At the same time, set the fork as your upstream, + so that it will be in sync for any future contributions. + + ```shell + git push --set-upstream origin ui-updates + ``` + +## Step 4: Create a merge request + +Now you're ready to push changes from the community fork to the main GitLab repository! + +1. Go to [the community fork on GitLab.com](https://gitlab.com/gitlab-community/gitlab). + You should see a message like this one: + +  + + Select **Create merge request**. + If you don't see this message, on the left sidebar, select **Merge requests > New merge request**. + +1. Take a look at the branch names. You should be merging from your branch + in the community fork to the `master` branch in the GitLab repository. + +  + +1. Fill out the information and then select **Save changes**. + Don't worry if your merge request is not complete. If you don't want anyone + from GitLab to review it, you can select the **Mark as draft** checkbox. + If you're not happy with the merge request after you create it, you can + close it, no harm done. + +1. Select the **Changes** tab. It should look something like this: + +  + + The red text shows the code before you made changes. The green shows what + the code looks like now. + +1. If you're happy with this merge request and want to start the review process, type + `@gitlab-bot ready` in a comment and then select **Comment**. + +  + +Someone from GitLab will look at your request and let you know what the next steps are. + +Now, any time you want to make a contribution to GitLab, you can just +go to the `gitlab-development-kit` folder and run `gdk update`. Then make +your changes in the `gitlab` directory and push them to the fork. + +If you need help at any point in the process, type `@gitlab-bot help` in a comment +or initiate a [mentor session](https://about.gitlab.com/community/contribute/mentor-sessions/) +on [Discord](https://discord.gg/gitlab). + +Congratulations on submitting your request, and thank you for your contribution! diff --git a/doc/development/contributing/img/bot_ready.png b/doc/development/contributing/img/bot_ready.png Binary files differnew file mode 100644 index 00000000000..85116c8957b --- /dev/null +++ b/doc/development/contributing/img/bot_ready.png diff --git a/doc/development/contributing/img/changes_tab.png b/doc/development/contributing/img/changes_tab.png Binary files differnew file mode 100644 index 00000000000..2158e9dd183 --- /dev/null +++ b/doc/development/contributing/img/changes_tab.png diff --git a/doc/development/contributing/img/gdk_home.png b/doc/development/contributing/img/gdk_home.png Binary files differnew file mode 100644 index 00000000000..77505850609 --- /dev/null +++ b/doc/development/contributing/img/gdk_home.png diff --git a/doc/development/contributing/img/mr_button.png b/doc/development/contributing/img/mr_button.png Binary files differnew file mode 100644 index 00000000000..5bf08b44412 --- /dev/null +++ b/doc/development/contributing/img/mr_button.png diff --git a/doc/development/contributing/img/new_merge_request.png b/doc/development/contributing/img/new_merge_request.png Binary files differnew file mode 100644 index 00000000000..34512d06dd2 --- /dev/null +++ b/doc/development/contributing/img/new_merge_request.png diff --git a/doc/development/contributing/img/ui_text_after.png b/doc/development/contributing/img/ui_text_after.png Binary files differnew file mode 100644 index 00000000000..3a54e7ef653 --- /dev/null +++ b/doc/development/contributing/img/ui_text_after.png diff --git a/doc/development/contributing/img/ui_text_before.png b/doc/development/contributing/img/ui_text_before.png Binary files differnew file mode 100644 index 00000000000..afd75a11ac2 --- /dev/null +++ b/doc/development/contributing/img/ui_text_before.png diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md index 6a9dd34e703..ef3242b89c2 100644 --- a/doc/development/contributing/index.md +++ b/doc/development/contributing/index.md @@ -8,10 +8,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Contribute to GitLab Thank you for your interest in contributing to GitLab. This guide details how -to contribute to GitLab in a way that is easy for everyone. +to contribute to the development of GitLab in a way that is easy for everyone. -For a first-time step-by-step guide to the contribution process, see our -[Contributing to GitLab](https://about.gitlab.com/community/contribute/) page. +For a first-time step-by-step guide to the contribution process for development, see [Tutorial: Make a GitLab contribution](first_contribution.md). + +For other ways to contribute see our [Contributing to GitLab](https://about.gitlab.com/community/contribute/) page. Looking for something to work on? See the [How to contribute](#how-to-contribute) section for more information. @@ -43,27 +44,10 @@ If you would like to contribute to GitLab: slowest tests. These tests are good candidates for improving and checking if any of [best practices](../testing_guide/best_practices.md) could speed them up. -- Consult the [Contribution Flow](#contribution-flow) section to learn the process. - -### Contribution flow - -The general flow of contributing to GitLab is: -1. [Read about](https://gitlab.com/gitlab-community/meta) and [request access](https://gitlab.com/gitlab-community/meta#request-access-to-community-forks) - to the GitLab Community forks. In some cases, you will want to set up the - [GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit) to - [develop against the GitLab community fork](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/index.md#develop-in-your-own-gitlab-fork). -1. Create a feature branch and make changes in the [GitLab community fork](https://gitlab.com/gitlab-community/gitlab). -1. When you're ready, [create a new merge request](../../user/project/merge_requests/creating_merge_requests.md). -1. In the merge request's description: - - Ensure you provide complete and accurate information. - - Review the provided checklist. -1. Once you're ready, mark your MR as ready for review with `@gitlab-bot ready`. - - This will add the `~"workflow::ready for review"` label, and then automatically assign a merge request coach as reviewer. - - If you know a relevant reviewer (for example, someone that was involved a related issue), you can also - assign them directly with `@gitlab-bot ready @username`. +For a walkthrough of the contribution process, see [Tutorial: Make a GitLab contribution](first_contribution.md). -#### Review process +### Review process When you submit code to GitLab, we really want it to get merged! However, we always review submissions carefully, and this takes time. Code submissions will usually be reviewed by two @@ -95,7 +79,7 @@ Lastly, keep the following in mind when submitting merge requests: be merged, as well as some guidance. The maintainers will be open to discussion about how to change the code so it can be approved and merged in the future. -#### Getting attention on your merge request +### Getting attention on your merge request GitLab will do its best to review community contributions as quickly as possible. Specially appointed developers review community contributions daily. Look at the @@ -119,7 +103,7 @@ This [documentation](issue_workflow.md) outlines the current issue workflow: - [Technical and UX debt](../labels/index.md#technical-and-ux-debt) - [Technical debt in follow-up issues](issue_workflow.md#technical-debt-in-follow-up-issues) -#### Merge requests workflow +### Merge requests workflow This [documentation](merge_request_workflow.md) outlines the current merge request process. @@ -136,9 +120,9 @@ This [documentation](merge_request_workflow.md) outlines the current merge reque ## Getting an Enterprise Edition License If you need a license for contributing to an EE-feature, see -[relevant information](https://about.gitlab.com/handbook/marketing/community-relations/code-contributor-program/operations/#contributing-to-the-gitlab-enterprise-edition-ee). +[relevant information](https://about.gitlab.com/handbook/marketing/community-relations/contributor-success/community-contributors-workflows.html#contributing-to-the-gitlab-enterprise-edition-ee). ## Finding help - [Get help](https://about.gitlab.com/get-help/). -- Join the community-run [Discord server](https://discord.com/invite/gitlab) and find other contributors in the `#contribute` channel. +- Join the community-run [Discord server](https://discord.gg/gitlab) and find other contributors in the `#contribute` channel. diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md index f39d93a39bc..a67459e02ed 100644 --- a/doc/development/contributing/merge_request_workflow.md +++ b/doc/development/contributing/merge_request_workflow.md @@ -55,66 +55,25 @@ within the MR. ## Merge request guidelines for contributors -### Getting started - -Merge requests should be submitted to the appropriate project at GitLab.com, for example -[GitLab](https://gitlab.com/gitlab-org/gitlab/-/merge_requests), -[GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests), or -[Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests). - -To start developing GitLab locally, download the [GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit) -and see the [Development section](../../index.md) for the required guidelines. - -NOTE: -Consider placing your code behind a feature flag if you think it might affect production availability. -Not sure? Read [When to use feature flags](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/#when-to-use-feature-flags). - -If the change is non-trivial, we encourage you to -start a discussion with [a product manager or a member of the team](https://about.gitlab.com/handbook/product/categories/). -You can do -this by tagging them in an MR before submitting the code for review. Talking -to team members can be helpful when making design decisions. Communicating the -intent behind your changes can also help expedite merge request reviews. - -### Creating a merge request - -To create a merge request: - -1. [Read about](https://gitlab.com/gitlab-community/meta) and [request access](https://gitlab.com/gitlab-community/meta#request-access-to-community-forks) - to the GitLab Community forks. In some cases, you will want to set up the - [GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit) to - [develop against the GitLab community fork](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/index.md#develop-in-your-own-gitlab-fork). -1. Create a feature branch in the [GitLab community fork](https://gitlab.com/gitlab-community/gitlab) - (don't work off the [default branch](../../user/project/repository/branches/default.md)). -1. Follow the [commit messages guidelines](#commit-messages-guidelines). -1. If you have multiple commits, combine them into a few logically organized commits. -1. Push the commits to your working branch in the fork. -1. Submit a merge request (MR) against the default branch of the upstream project. -1. The MR title should describe the change you want to make. -1. The MR description should give a reason for your change. - 1. If you are contributing code, fill in the description according to the default - template already provided in the "Description" field. - 1. If you are contributing documentation, choose `Documentation` from the - "Choose a template" menu and fill in the description according to the template. - 1. Use the syntax `Solves #XXX`, `Closes #XXX`, or `Refs #XXX` to mention the issues your merge - request addresses. Referenced issues do not [close automatically](../../user/project/issues/managing_issues.md#closing-issues-automatically). - You must close them manually once the merge request is merged. -1. If you're allowed to, set a relevant milestone and [labels](issue_workflow.md). - MR labels should generally match the corresponding issue (if there is one). - The group label should reflect the group that executed or coached the work, - not necessarily the group that owns the feature. -1. Read and adhere to - [The responsibility of the merge request author](../code_review.md#the-responsibility-of-the-merge-request-author). -1. Read and follow - [Having your merge request reviewed](../code_review.md#having-your-merge-request-reviewed). -1. Make sure the merge request meets the [Definition of done](#definition-of-done). - -If you would like quick feedback on your merge request feel free to mention someone -from the [core team](https://about.gitlab.com/community/core-team/) or one of the -[merge request coaches](https://about.gitlab.com/company/team/). When having your code reviewed -and when reviewing merge requests, please keep the [code review guidelines](../code_review.md) -in mind. And if your code also makes changes to the database, or does expensive queries, -check the [database review guidelines](../database_review.md). +For a walkthrough of the contribution process, see [Tutorial: Make a GitLab contribution](first_contribution.md). + +### Best practices + +- If the change is non-trivial, we encourage you to start a discussion with + [a product manager or a member of the team](https://about.gitlab.com/handbook/product/categories/). + You can do this by tagging them in an MR before submitting the code for review. Talking + to team members can be helpful when making design decisions. Communicating the + intent behind your changes can also help expedite merge request reviews. + +- Consider placing your code behind a feature flag if you think it might affect production availability. + Not sure? Read [When to use feature flags](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/#when-to-use-feature-flags). + +- If you would like quick feedback on your merge request feel free to mention someone + from the [core team](https://about.gitlab.com/community/core-team/) or one of the + [merge request coaches](https://about.gitlab.com/company/team/). When having your code reviewed + and when reviewing merge requests, please keep the [code review guidelines](../code_review.md) + in mind. And if your code also makes changes to the database, or does expensive queries, + check the [database review guidelines](../database_review.md). ### Keep it simple @@ -381,3 +340,8 @@ issue) that are incremental improvements, such as: Tag a merge request with ~"Stuff that should Just Work" to track work in this area. + +## Related topics + +- [The responsibility of the merge request author](../code_review.md#the-responsibility-of-the-merge-request-author) +- [Having your merge request reviewed](../code_review.md#having-your-merge-request-reviewed) diff --git a/doc/development/database/adding_database_indexes.md b/doc/development/database/adding_database_indexes.md index 1e3a1de9b69..7b29b1b14de 100644 --- a/doc/development/database/adding_database_indexes.md +++ b/doc/development/database/adding_database_indexes.md @@ -38,6 +38,15 @@ when adding a new index: 1. Is the overhead of maintaining the index worth the reduction in query timings? +In some situations, an index might not be required: + +- The table is small (less than `1,000` records) and it's not expected to exponentially grow in size. +- Any existing indexes filter out enough rows. +- The reduction in query timings after the index is added is not significant. + +Additionally, wide indexes are not required to match all filter criteria of queries. We just need +to cover enough columns so that the index lookup has a small enough selectivity. + ## Re-using Queries The first step is to make sure your query re-uses as many existing indexes as @@ -183,6 +192,29 @@ for `index_exists?`, causing a required index to not be created properly. By always requiring a name for certain types of indexes, the chance of error is greatly reduced. +## Testing for existence of indexes + +The easiest way to test for existence of an index by name is to use the `index_name_exists?` method, but the `index_exists?` method can also be used with a name option. For example: + +```ruby +class MyMigration < Gitlab::Database::Migration[2.1] + INDEX_NAME = 'index_name' + + def up + # an index must be conditionally created due to schema inconsistency + unless index_exists?(:table_name, :column_name, name: INDEX_NAME) + add_index :table_name, :column_name, name: INDEX_NAME + end + end + + def down + # no op + end +end +``` + +Keep in mind that concurrent index helpers like `add_concurrent_index`, `remove_concurrent_index`, and `remove_concurrent_index_by_name` already perform existence checks internally. + ## Temporary indexes There may be times when an index is only needed temporarily. @@ -448,7 +480,7 @@ You must test the database index changes locally before creating a merge request the post-deploy migration has been executed in the production database. For more information, see [How to determine if a post-deploy migration has been executed on GitLab.com](https://gitlab.com/gitlab-org/release/docs/-/blob/master/general/post_deploy_migration/readme.md#how-to-determine-if-a-post-deploy-migration-has-been-executed-on-gitlabcom). 1. In the case of an [index removed asynchronously](#schedule-the-index-to-be-removed), wait - until the next week so that the index can be created over a weekend. + until the next week so that the index can be removed over a weekend. 1. Use Database Lab [to check if removal was successful](database_lab.md#checking-indexes). [Database Lab](database_lab.md) should report an error when trying to find the removed index. If not, the index may still exist. diff --git a/doc/development/database/avoiding_downtime_in_migrations.md b/doc/development/database/avoiding_downtime_in_migrations.md index 8e1eeee7a42..25310554c24 100644 --- a/doc/development/database/avoiding_downtime_in_migrations.md +++ b/doc/development/database/avoiding_downtime_in_migrations.md @@ -605,7 +605,7 @@ See example [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_request ### Remove the trigger and old `integer` columns (release N + 2) Using post-deployment migration and the provided `cleanup_conversion_of_integer_to_bigint` helper, -drop the database trigger and the old `integer` columns ([see an example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69714)). +drop the database trigger and the old `integer` columns ([see an example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70351)). ### Remove ignore rules (release N + 3) diff --git a/doc/development/database/batched_background_migrations.md b/doc/development/database/batched_background_migrations.md index 67dccd99a6c..7f3c5889017 100644 --- a/doc/development/database/batched_background_migrations.md +++ b/doc/development/database/batched_background_migrations.md @@ -597,6 +597,37 @@ for more details. more pressure on DB than you expect. Measure on staging, or ask someone to measure on production. 1. Know how much time is required to run the batched background migration. +1. Be careful when silently rescuing exceptions inside job classes. This may lead to + jobs being marked as successful, even in a failure scenario. + + ```ruby + # good + def perform + each_sub_batch do |sub_batch| + sub_batch.update_all(name: 'My Name') + end + end + + # acceptable + def perform + each_sub_batch do |sub_batch| + sub_batch.update_all(name: 'My Name') + rescue Exception => error + logger.error(message: error.message, class: error.class) + + raise + end + end + + # bad + def perform + each_sub_batch do |sub_batch| + sub_batch.update_all(name: 'My Name') + rescue Exception => error + logger.error(message: error.message, class: self.class.name) + end + end + ``` ## Additional tips and strategies @@ -768,7 +799,7 @@ To do that, you need to: #### 1. Find the batch `start_id` and `end_id` -You should be able to find those in [Kibana][#viewing-failure-error-logs]. +You should be able to find those in [Kibana](#viewing-failure-error-logs). #### 2. Create a regular migration @@ -809,7 +840,7 @@ index b8d1d21a0d2d2a23d9e8c8a0a17db98ed1ed40b7..912e20659a6919f771045178c6682856 +++ b/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb @@ -55,7 +55,7 @@ def unmatched_schemas end - + def allowed_schemas_for_connection - Gitlab::Database.gitlab_schemas_for_connection(connection) + Gitlab::Database.gitlab_schemas_for_connection(connection) << :gitlab_ci @@ -828,7 +859,7 @@ index 4ae3622479f0800c0553959e132143ec9051898e..d556ec7f55adae9d46a56665ce02de78 @@ -79,7 +79,7 @@ def restrict_to_dml_only(parsed) tables = self.dml_tables(parsed) schemas = self.dml_schemas(tables) - + - if (schemas - self.allowed_gitlab_schemas).any? + if (schemas - (self.allowed_gitlab_schemas << :gitlab_ci)).any? raise DMLAccessDeniedError, \ diff --git a/doc/development/database/ci_mirrored_tables.md b/doc/development/database/ci_mirrored_tables.md index bf3a744b936..1e37739bdc4 100644 --- a/doc/development/database/ci_mirrored_tables.md +++ b/doc/development/database/ci_mirrored_tables.md @@ -76,9 +76,8 @@ the source and the target tables in sync: 1. Deleting namespaces/projects. ```mermaid -graph TD - - subgraph "CI database (tables)" +graph LR + subgraph CI["CI Tables"] E[other CI tables] F{queries with joins allowed} G[ci_project_mirrors] @@ -89,17 +88,18 @@ graph TD F---H end - A---B - B---C - B---D + Main["Main Tables"]---L["⛔ ← Joins are not allowed → ⛔"] + L---CI -L["⛔ ← Joins are not allowed → ⛔"] - - subgraph "Main database (tables)" + subgraph Main["Main Tables"] A[other main tables] B{queries with joins allowed} C[projects] D[namespaces] + + A---B + B---C + B---D end ``` diff --git a/doc/development/database/index.md b/doc/development/database/index.md index 2cb8509e203..f532e054849 100644 --- a/doc/development/database/index.md +++ b/doc/development/database/index.md @@ -64,6 +64,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w - [Hash indexes](hash_indexes.md) - [Insert into tables in batches](insert_into_tables_in_batches.md) - [Iterating tables in batches](iterating_tables_in_batches.md) +- [Load balancing](load_balancing.md) - [`NOT NULL` constraints](not_null_constraints.md) - [Ordering table columns](ordering_table_columns.md) - [Pagination guidelines](pagination_guidelines.md) diff --git a/doc/development/database/iterating_tables_in_batches.md b/doc/development/database/iterating_tables_in_batches.md index 6357bed8b00..a927242e8d8 100644 --- a/doc/development/database/iterating_tables_in_batches.md +++ b/doc/development/database/iterating_tables_in_batches.md @@ -44,9 +44,13 @@ all of the arguments that `in_batches` supports. You should always use ## Iterating over non-unique columns -One should proceed with extra caution. When you iterate over an attribute that is not unique, -even with the applied max batch size, there is no guarantee that the resulting batches do not -surpass it. The following snippet demonstrates this situation when one attempt to select +You should not use the `each_batch` method with a non-unique column (in the context of the relation) as it +[may result in an infinite loop](https://gitlab.com/gitlab-org/gitlab/-/issues/285097). +Additionally, the inconsistent batch sizes cause performance issues when you +iterate over non-unique columns. Even when you apply a max batch size +when iterating over an attribute, there's no guarantee that the resulting +batches don't surpass it. The following snippet demonstrates this situation +when you attempt to select `Ci::Build` entries for users with `id` between `1` and `10,000`, the database returns `1 215 178` matching rows. @@ -465,6 +469,58 @@ Issue.each_batch(of: 1000) do |relation| end ``` +### Counting records + +For tables with a large amount of data, counting records through queries can result +in timeouts. The `EachBatch` module provides an alternative way to iteratively count +records. The downside of using `each_batch` is the extra count query which is executed +on the yielded relation object. + +The `each_batch_count` method is a more efficient approach that eliminates the need +for the extra count query. By invoking this method, the iteration process can be +paused and resumed as needed. This feature is particularly useful in situations +where error budget violations are triggered after five minutes, such as when performing +counting operations within Sidekiq workers. + +To illustrate, counting records using `EachBatch` involves invoking an additional +count query as follows: + +```ruby +count = 0 + +Issue.each_batch do |relation| + count += relation.count +end + +puts count +``` + +On the other hand, the `each_batch_count` method enables the counting process to be +performed more efficiently (counting is part of the iteration query) without invoking +an extra count query: + +```ruby +count, _last_value = Issue.each_batch_count # last value can be ignored here +``` + +Furthermore, the `each_batch_count` method allows the counting process to be paused +and resumed at any point. This capability is demonstrated in the following code snippet: + +```ruby +stop_at = Time.current + 3.minutes + +count, last_value = Issue.each_batch_count do + Time.current > stop_at # condition for stopping the counting +end + +# Continue the counting later +stop_at = Time.current + 3.minutes + +count, last_value = Issue.each_batch_count(last_count: count, last_value: last_value) do + Time.current > stop_at +end +``` + ### `EachBatch` vs `BatchCount` When adding new counters for Service Ping, the preferred way to count records is using the diff --git a/doc/development/database/load_balancing.md b/doc/development/database/load_balancing.md new file mode 100644 index 00000000000..f623ad1eab0 --- /dev/null +++ b/doc/development/database/load_balancing.md @@ -0,0 +1,59 @@ +--- +stage: Data Stores +group: Database +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Database load balancing + +With database load balancing, read-only queries can be distributed across multiple +PostgreSQL nodes to increase performance. + +This documentation provides a technical overview on how database load balancing +is implemented in GitLab Rails and Sidekiq. + +## Nomenclature + +1. **Host**: Each database host. It could be a primary or a replica. +1. **Primary**: Primary PostgreSQL host that is used for write-only and read-and-write operations. +1. **Replica**: Secondary PostgreSQL hosts that are used for read-only operations. +1. **Workload**: a Rails request or a Sidekiq job that requires database connections. + +## Components + +F few Ruby classes are involved in the load balancing process. All of them are +in the namespace `Gitlab::Database::LoadBalancing`: + +1. `Host` +1. `LoadBalancer` +1. `ConnectionProxy` +1. `Session` + +Each workload begins with a new instance of `Gitlab::Database::LoadBalancing::Session`. +The `Session` keeps track of the database operations that have been performed. It then +determines if the workload requires a connection to either the primary host or a replica host. + +When the workload requires a database connection through `ActiveRecord`, +`ConnectionProxy` first redirects the connection request to `LoadBalancer`. +`ConnectionProxy` requests either a `read` or `read_write` connection from the `LoadBalancer` +depending on a few criteria: + +1. Whether the query is a read-only or it requires write. +1. Whether the `Session` has recorded a write operation previously. +1. Whether any special blocks have been used to prefer primary or replica, such as: + - `use_primary` + - `ignore_writes` + - `use_replicas_for_read_queries` + - `fallback_to_replicas_for_ambiguous_queries` + +`LoadBalancer` then yields the requested connection from the respective database connection pool. +It yields either: + +- A `read_write` connection from the primary's connection pool. +- A `read` connection from the replicas' connection pools. + +When responding to a request for a `read` connection, `LoadBalancer` would +first attempt to load balance the connection across the replica hosts. +It looks for the next `online` replica host and yields a connection from the host's connection pool. +A replica host is considered `online` if it is up-to-date with the primary, based on +either the replication lag size or time. The thresholds for these requirements are configurable. diff --git a/doc/development/database/loose_foreign_keys.md b/doc/development/database/loose_foreign_keys.md index daa022a3de2..91a22d8c26b 100644 --- a/doc/development/database/loose_foreign_keys.md +++ b/doc/development/database/loose_foreign_keys.md @@ -64,7 +64,7 @@ The tool ensures that all aspects of swapping a foreign key are covered. This in - Creating a migration to remove a foreign key. - Updating `db/structure.sql` with the new migration. -- Updating `lib/gitlab/database/gitlab_loose_foreign_keys.yml` to add the new loose foreign key. +- Updating `config/gitlab_loose_foreign_keys.yml` to add the new loose foreign key. - Creating or updating a model's specs to ensure that the loose foreign key is properly supported. The tool is located at `scripts/decomposition/generate-loose-foreign-key`: diff --git a/doc/development/database/multiple_databases.md b/doc/development/database/multiple_databases.md index c1b30a4cbbe..6adfdc90cf2 100644 --- a/doc/development/database/multiple_databases.md +++ b/doc/development/database/multiple_databases.md @@ -545,7 +545,7 @@ end ``` Don't hesitate to reach out to the -[pods group](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/pods/) +[Pods group](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/tenant-scale/) for advice. ##### Avoid `dependent: :nullify` and `dependent: :destroy` across databases @@ -580,6 +580,24 @@ or records that point to nowhere, which might lead to bugs. As such we created ["loose foreign keys"](loose_foreign_keys.md) which is an asynchronous process of cleaning up orphaned records. +## Testing for multiple databases + +In our testing CI pipelines, we test GitLab by default with multiple databases set up, using +both `main` and `ci` databases. But in merge requests, for example when we modify some database-related code or +add the label `~"pipeline:run-single-db"` to the MR, we additionally run our tests in +[two other database modes](../pipelines/index.md#single-database-testing): +`single-db` and `single-db-ci-connection`. + +To handle situations where our tests need to run in specific database modes, we have some RSpec helpers +to limit the modes where tests can run, and skip them on any other modes. + +| Helper name | Test runs | +|---------------------------------------------| --- | +| `skip_if_shared_database(:ci)` | On **multiple databases** | +| `skip_if_database_exists(:ci)` | On **single-db** and **single-db-ci-connection** | +| `skip_if_multiple_databases_are_setup(:ci)` | Only on **single-db** | +| `skip_if_multiple_databases_not_setup(:ci)` | On **single-db-ci-connection** and **multiple databases** | + ## Locking writes on the tables that don't belong to the database schemas When the CI database is promoted and the two databases are fully split, diff --git a/doc/development/database/strings_and_the_text_data_type.md b/doc/development/database/strings_and_the_text_data_type.md index 47e89c1ce0f..5cd325bfa56 100644 --- a/doc/development/database/strings_and_the_text_data_type.md +++ b/doc/development/database/strings_and_the_text_data_type.md @@ -68,9 +68,17 @@ is held for a brief amount of time, the time `add_column` needs to complete its depending on how frequently the table is accessed. For example, acquiring an exclusive lock for a very frequently accessed table may take minutes in GitLab.com and requires the use of `with_lock_retries`. -For these reasons, it is advised to add the text limit on a separate migration than the `add_column` one. +When adding a text limit, transactions must be disabled with `disable_ddl_transaction!`. This means adding the column is not rolled back +in case the migration fails afterwards. An attempt to re-run the migration will raise an error because of the already existing column. -For example, consider a migration that adds a new text column `extended_title` to table `sprints`, +For these reasons, adding a text column to an existing table can be done by either: + +- [Add the column and limit in separate migrations.](#add-the-column-and-limit-in-separate-migrations) +- [Add the column and limit in one migration with checking if the column already exists.](#add-the-column-and-limit-in-one-migration-with-checking-if-the-column-already-exists) + +### Add the column and limit in separate migrations + +Consider a migration that adds a new text column `extended_title` to table `sprints`, `db/migrate/20200501000001_add_extended_title_to_sprints.rb`: ```ruby @@ -103,6 +111,33 @@ class AddTextLimitToSprintsExtendedTitle < Gitlab::Database::Migration[2.1] end ``` +### Add the column and limit in one migration with checking if the column already exists + +Consider a migration that adds a new text column `extended_title` to table `sprints`, +`db/migrate/20200501000001_add_extended_title_to_sprints.rb`: + +```ruby +class AddExtendedTitleToSprints < Gitlab::Database::Migration[2.1] + disable_ddl_transaction! + + def up + with_lock_retries do + add_column :sprints, :extended_title, :text, if_not_exists: true + end + + add_text_limit :sprints, :extended_title, 512 + end + + def down + remove_text_limit :sprints, :extended_title + + with_lock_retries do + remove_column :sprints, :extended_title, if_exists: true + end + end +end +``` + ## Add a text limit constraint to an existing column Adding text limits to existing database columns requires multiple steps split into at least two different releases: diff --git a/doc/development/database/table_partitioning.md b/doc/development/database/table_partitioning.md index 0d5e3c233f6..81733b126b6 100644 --- a/doc/development/database/table_partitioning.md +++ b/doc/development/database/table_partitioning.md @@ -6,6 +6,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Database table partitioning +WARNING: +If you have questions not answered below, check for and add them +to [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/398650). +Tag `@gitlab-org/database-team/triage` and we'll get back to you with an +answer as soon as possible. If you get an answer in Slack, document +it on the issue as well so we can update this document in the future. + Table partitioning is a powerful database feature that allows a table's data to be split into smaller physical tables that act as a single large table. If the application is designed to work with partitioning in mind, @@ -32,31 +39,38 @@ several releases. Due to the limitations of partitioning and the related migrations, you should understand how partitioning fits your use case before attempting to leverage this feature. -## Determining when to use partitioning +## Determine when to use partitioning While partitioning can be very useful when properly applied, it's imperative to identify if the data and workload of a table naturally fit a -partitioning scheme. There are a few details you have to understand -to decide if partitioning is a good fit for your particular -problem. - -First, a table is partitioned on a partition key, which is a column or -set of columns which determine how the data is split across the -partitions. The partition key is used by the database when reading or -writing data, to decide which partitions must be accessed. The -partition key should be a column that would be included in a `WHERE` -clause on almost all queries accessing that table. - -Second, it's necessary to understand the strategy the database uses -to split the data across the partitions. The scheme supported by the -GitLab migration helpers is date-range partitioning, where each partition -in the table contains data for a single month. In this case, the partitioning -key must be a timestamp or date column. In order for this type of +partitioning scheme. Understand a few details to decide if partitioning +is a good fit for your particular problem: + +- **Table partitioning**. A table is partitioned on a partition key, which is a + column or set of columns which determine how the data is split across the + partitions. The partition key is used by the database when reading or + writing data, to decide which partitions must be accessed. The + partition key should be a column that would be included in a `WHERE` + clause on almost all queries accessing that table. + +- **How the data is split**. What strategy does the database use + to split the data across the partitions? The available choices are `range`, + `hash`, and `list`. + +## Determine the appropriate partitioning strategy + +The available partitioning strategy choices are `range`, `hash`, and `list`. + +### Range partitioning + +The scheme best supported by the GitLab migration helpers is date-range partitioning, +where each partition in the table contains data for a single month. In this case, +the partitioning key must be a timestamp or date column. For this type of partitioning to work well, most queries must access data in a certain date range. -For a more concrete example, the `audit_events` table can be used, which -was the first table to be partitioned in the application database +For a more concrete example, consider using the `audit_events` table. +It was the first table to be partitioned in the application database (scheduled for deployment with the GitLab 13.5 release). This table tracks audit entries of security events that happen in the application. In almost all cases, users want to see audit activity that @@ -142,6 +156,31 @@ substantial. Partitioning should only be leveraged if the access patterns of the data support the partitioning strategy, otherwise performance suffers. +### Hash Partitioning + +Hash partitioning splits a logical table into a series of partitioned +tables. Each partition corresponds to the ID range that matches +a hash and remainder. For example, if partitioning `BY HASH(id)`, rows +with `hash(id) % 64 == 1` would end up in the partition +`WITH (MODULUS 64, REMAINDER 1)`. + +When hash partitioning, you must include a `WHERE hashed_column = ?` condition in +every performance-sensitive query issued by the application. If this is not possible, +hash partitioning may not be the correct fit for your use case. + +Hash partitioning has one main advantage: it is the only type of partitioning that +can enforce uniqueness on a single numeric `id` column. (While also possible with +range partitioning, it's rarely the correct choice). + +Hash partitioning has downsides: + +- The number of partitions must be known up-front. +- It's difficult to move new data to an extra partition if current partitions become too large. +- Range queries, such as `WHERE id BETWEEN ? and ?`, are unsupported. +- Lookups by other keys, such as `WHERE other_id = ?`, are unsupported. + +For this reason, it's often best to choose a large number of hash partitions to accommodate future table growth. + ## Partitioning a table (Range) Unfortunately, tables can only be partitioned at their creation, making @@ -257,6 +296,18 @@ for use by the application. This section will be updated when the migration helper is ready, for now development can be followed in the [Tracking Issue](https://gitlab.com/gitlab-org/gitlab/-/issues/241267). +## Partitioning a table (Hash) + +Hash partitioning divides data into partitions based on a hash of their ID. +It works well only if most queries against the table include a clause like `WHERE id = ?`, +so that PostgreSQL can decide which partition to look in based on the ID or ids being requested. + +Another key downside is that hash partitioning does not allow adding additional partitions after table creation. +The correct number of partitions must be chosen up-front. + +Hash partitioning is the only type of partitioning (aside from some complex uses of list partitioning) that can guarantee +uniqueness of an ID across multiple partitions at the database level. + ## Partitioning a table (List) > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96815) in GitLab 15.4. @@ -504,3 +555,48 @@ class Model < ApplicationRecord self.sequence_name = 'model_id_seq' end ``` + +If the partitioning constraint migration takes [more than 10 minutes](../migration_style_guide.md#how-long-a-migration-should-take) to finish, +it can be made to run asynchronously to avoid running the post-migration during busy hours. + +Prepend the following migration `AsyncPrepareTableConstraintsForListPartitioning` +and use `async: true` option. This change marks the partitioning constraint as `NOT VALID` +and enqueues a scheduled job to validate the existing data in the table during the weekend. + +Then the second post-migration `PrepareTableConstraintsForListPartitioning` only +marks the partitioning constraint as validated, because the existing data is already +tested during the previous weekend. + +For example: + +```ruby +class AsyncPrepareTableConstraintsForListPartitioning < Gitlab::Database::Migration[2.1] + include Gitlab::Database::PartitioningMigrationHelpers::TableManagementHelpers + + disable_ddl_transaction! + + TABLE_NAME = :table_name + PARENT_TABLE_NAME = :p_table_name + FIRST_PARTITION = 100 + PARTITION_COLUMN = :partition_id + + def up + prepare_constraint_for_list_partitioning( + table_name: TABLE_NAME, + partitioning_column: PARTITION_COLUMN, + parent_table_name: PARENT_TABLE_NAME, + initial_partitioning_value: FIRST_PARTITION, + async: true + ) + end + + def down + revert_preparing_constraint_for_list_partitioning( + table_name: TABLE_NAME, + partitioning_column: PARTITION_COLUMN, + parent_table_name: PARENT_TABLE_NAME, + initial_partitioning_value: FIRST_PARTITION + ) + end +end +``` diff --git a/doc/development/database/transaction_guidelines.md b/doc/development/database/transaction_guidelines.md index 26bb6c2ce8f..1648cf58937 100644 --- a/doc/development/database/transaction_guidelines.md +++ b/doc/development/database/transaction_guidelines.md @@ -12,7 +12,7 @@ For further reference, check PostgreSQL documentation about [transactions](https ## Database decomposition and sharding -The [Pods Group](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/pods/) plans +The [Pods group](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/tenant-scale/) plans to split the main GitLab database and move some of the database tables to other database servers. We start decomposing the `ci_*`-related database tables first. To maintain the current application diff --git a/doc/development/database_review.md b/doc/development/database_review.md index 1f653f8af94..feac0b3098d 100644 --- a/doc/development/database_review.md +++ b/doc/development/database_review.md @@ -125,7 +125,7 @@ the following preparations into account. test its execution using `CREATE INDEX CONCURRENTLY` in [Database Lab](database/database_lab.md) and add the execution time to the MR description: - Execution time largely varies between Database Lab and GitLab.com, but an elevated execution time from Database Lab can give a hint that the execution on GitLab.com is also considerably high. - - If the execution from Database Lab is longer than `1h`, the index should be moved to a [post-migration](database/post_deployment_migrations.md). + - If the execution from Database Lab is longer than `10 minutes`, the [index](database/adding_database_indexes.md) should be moved to a [post-migration](database/post_deployment_migrations.md). Keep in mind that in this case you may need to split the migration and the application changes in separate releases to ensure the index is in place when the code that needs it is deployed. - Manually trigger the [database testing](database/database_migration_pipeline.md) job (`db:gitlabcom-database-testing`) in the `test` stage. diff --git a/doc/development/diffs.md b/doc/development/diffs.md deleted file mode 100644 index c84bf57e085..00000000000 --- a/doc/development/diffs.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'merge_request_concepts/diffs/index.md' -remove_date: '2023-04-10' ---- - -This document was moved to [another location](merge_request_concepts/diffs/index.md). - -<!-- This redirect file can be deleted after <2023-04-10>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/distributed_tracing.md b/doc/development/distributed_tracing.md index bd9e7035290..823a71eb130 100644 --- a/doc/development/distributed_tracing.md +++ b/doc/development/distributed_tracing.md @@ -22,6 +22,14 @@ Distributed tracing adds minimal overhead when disabled, but imposes only small enabled and is therefore capable in any environment, including production. For this reason, it can be useful in diagnosing production issues, particularly performance problems. +Services have different levels of support for distributed tracing. Custom +instrumentation code must be added to the application layer in addition to +pre-built instrumentation for the most common libraries. + +For service-specific information, see: + +- [Using Jaeger for Gitaly local development](https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/jaeger_for_local_development.md) + ## Using Correlation IDs to investigate distributed requests The GitLab application passes correlation IDs between the various components in a request. A @@ -77,15 +85,56 @@ In this example, we have the following hypothetical values: they should be URL encoded. Multiple values should be separated by `&` characters like a URL. +GitLab Rails provides pre-implemented instrumentations for common types of +operations that offer a detailed view of the requests. However, the detailed +information comes at a cost. The resulting traces are long and can be difficult +to process, making it hard to identify bigger underlying issues. To address this +concern, some instrumentations are disabled by default. To enable those disabled +instrumentations, set the following environment variables: + +- `GITLAB_TRACING_TRACK_CACHES`: enable tracking cache operations, such as cache +read, write, or delete. +- `GITLAB_TRACING_TRACK_REDIS`: enable tracking Redis operations. Most Redis +operations are for caching, though. + ## Using Jaeger in the GitLab Development Kit The first tracing implementation that GitLab supports is Jaeger, and the -[GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit/) supports distributed tracing with -Jaeger out-of-the-box. +[GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit/) +supports distributed tracing with Jaeger out-of-the-box. GDK automatically adds +`GITLAB_TRACING` environment variables to add services. + +Configure GDK for Jaeger by editing the `gdk.yml` file and adding the following +settings: + +```yaml +tracer: + build_tags: tracer_static tracer_static_jaeger + jaeger: + enabled: true + listen_address: 127.0.0.1 + version: 1.43.0 +``` -The easiest way to access tracing from a GDK environment is through the -[performance-bar](../administration/monitoring/performance/performance_bar.md). This can be shown -by typing `p` `b` in the browser window. +After modifying the `gdk.yml` file, reconfigure your GDK by running +the `gdk reconfigure` command. This ensures that your GDK is properly configured +and ready to use. + +The above configuration sets the `tracer_static` and `tracer_static_jaeger` +build tags when rebuilding services written in Go for the first time. Any +changes made afterward require rebuilding them with those build tags. You can +either: + +- Add those build tags to the default set of build tags. +- Manually attach them to the build command. For example, Gitaly supports adding + build tag out of the box. You can run + `make all WITH_BUNDLED_GIT=YesPlease BUILD_TAGS="tracer_static tracer_static_jaeger"`. + +After reconfiguration, Jaeger dashboard is available at +`http://localhost:16686`. Another way to access tracing from a GDK environment +is through the +[performance-bar](../administration/monitoring/performance/performance_bar.md). +This can be shown by typing `p` `b` in the browser window. Once the performance bar is enabled, select **Trace** in the performance bar to go to the Jaeger UI. diff --git a/doc/development/documentation/alpha_beta.md b/doc/development/documentation/alpha_beta.md index 4b5c24d512f..d421aae3cb9 100644 --- a/doc/development/documentation/alpha_beta.md +++ b/doc/development/documentation/alpha_beta.md @@ -4,40 +4,46 @@ stage: none group: unassigned --- -# Document Alpha, Beta, LA features +# Documenting Experiment and Beta features Some features are not generally available and are instead considered -[Alpha, Beta, or Limited Availability](../../policy/alpha-beta-support.md). +[Experiment or Beta](../../policy/alpha-beta-support.md). When you document a feature in one of these three statuses: -- Add `(Alpha)`, `(Beta)`, or `(Limited Availability)` in parentheses after the page or topic title. -- Do not include `(Alpha)`, `(Beta)`, or `(Limited Availability)` in the left nav. +- Add `(Experiment)` or `(Beta)` in parentheses after the page or topic title. +- Do not include `(Experiment)` or `(Beta)` in the left nav. - Ensure the version history lists the feature's status. These features are usually behind a feature flag, which follow [these documentation guidelines](feature_flags.md). If you add details of how users should enroll, or how to contact the team with issues, -the `FLAG:` note should be above these details. For example: +the `FLAG:` note should be above these details. + +For example: ```markdown -## Great new feature (Alpha) +## Great new feature (Experiment) + +> [Introduced](link) in GitLab 15.10. This feature is an [Experiment](<link_to>/policy/alpha-beta-support.md). FLAG: On self-managed GitLab, by default this feature is not available. -To make it available, ask an administrator to enable the feature flag named example_flag. +To make it available, ask an administrator to enable the feature flag named `example_flag`. On GitLab.com, this feature is not available. This feature is not ready for production use. Use this great new feature when you need to do this new thing. -This feature is in Alpha. To join the list of users testing this feature, -do this thing. If you find a bug, [open an issue](link). +This feature is an [Experiment](<link_to>/policy/alpha-beta-support.md). To join +the list of users testing this feature, do this thing. If you find a bug, +[open an issue](link). ``` -When the feature is released, remove: +When the feature is ready for production, remove: - The text in parentheses. -- Any language about the feature not being ready for production. -- The feature flag information. +- Any language about the feature not being ready for production in the body + description. +- The feature flag information if available. -Ensure the version history is up-to-date. +Ensure the version history is up-to-date by adding a note about the production release. diff --git a/doc/development/documentation/feature_flags.md b/doc/development/documentation/feature_flags.md index 986252eedac..854faa839ef 100644 --- a/doc/development/documentation/feature_flags.md +++ b/doc/development/documentation/feature_flags.md @@ -17,7 +17,7 @@ When the state of a feature flag changes, the developer who made the change Every feature introduced to the codebase, even if it's behind a disabled feature flag, must be documented. For more information, see -[the discussion that led to this decision](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/47917#note_459984428). [Alpha, Beta, or Limited Availability](../../policy/alpha-beta-support.md) features are usually behind a feature flag, and must also be documented. For more information, see [Document Alpha, Beta, LA features](alpha_beta.md). +[the discussion that led to this decision](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/47917#note_459984428). [Experiment or Beta](../../policy/alpha-beta-support.md) features are usually behind a feature flag, and must also be documented. For more information, see [Document Experiment or Beta features](alpha_beta.md). When the feature is [implemented in multiple merge requests](../feature_flags/index.md#feature-flags-in-gitlab-development), discuss the plan with your technical writer. diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index d1f5ee8f9f3..c9f31b36e3f 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -114,21 +114,6 @@ The following metadata should be added when a page is moved to another location: - `redirect_to`: The relative path and filename (with an `.md` extension) of the location to which visitors should be redirected for a moved page. [Learn more](redirects.md). -- `disqus_identifier`: Identifier for Disqus commenting system. Used to keep - comments with a page that has been moved to a new URL. - [Learn more](redirects.md#redirections-for-pages-with-disqus-comments). - -### Comments metadata - -The [docs website](site_architecture/index.md) has comments (provided by Disqus) -enabled by default. In case you want to disable them (for example in index pages), -set it to `false`: - -```yaml ---- -comments: false ---- -``` ### Additional page metadata @@ -353,28 +338,6 @@ feedback: false The default is to leave it there. If you want to omit it from a document, you must check with a technical writer before doing so. -## Disqus - -We have integrated the docs site with Disqus (introduced by -[!151](https://gitlab.com/gitlab-org/gitlab-docs/-/merge_requests/151)), -allowing our users to post comments. - -To omit only the comments from the feedback section, use the following key in -the front matter: - -```yaml ---- -comments: false ---- -``` - -We're hiding comments only in main index pages, such as [the main documentation index](../../index.md), -since its content is too broad to comment on. Before omitting Disqus, you must -check with a technical writer. - -Note that after adding `feedback: false` to the front matter, it will omit -Disqus, therefore, don't add both keys to the same document. - The click events in the feedback section are tracked with Google Tag Manager. The conversions can be viewed on Google Analytics by navigating to **Behavior > Events > Top events > docs**. diff --git a/doc/development/documentation/redirects.md b/doc/development/documentation/redirects.md index 4cfe8be713a..068c1e84a0f 100644 --- a/doc/development/documentation/redirects.md +++ b/doc/development/documentation/redirects.md @@ -81,7 +81,6 @@ To redirect a page to another page in the same repository: - Replace both instances of `../newpath/to/file/index.md` with the new file path. - Replace both instances of `YYYY-MM-DD` with the expiration date, as explained in the template. -1. If the page has Disqus comments, follow [the steps for pages with Disqus comments](#redirections-for-pages-with-disqus-comments). 1. If the page had images that aren't used on any other pages, delete them. After your changes are committed, search for and update all links that point to the old file: @@ -158,26 +157,3 @@ If you create a new page and then rename it before it's added to a release on th Instead of following that procedure, ask a Technical Writer to manually add the redirect to [`redirects.yaml`](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/content/_data/redirects.yaml). - -## Redirections for pages with Disqus comments - -If the documentation page being relocated already has Disqus comments, -we need to preserve the Disqus thread. - -Disqus uses an identifier per page, and for <https://docs.gitlab.com>, the page identifier -is configured to be the page URL. Therefore, when we change the document location, -we need to preserve the old URL as the same Disqus identifier. - -To do that, add to the front matter the variable `disqus_identifier`, -using the old URL as value. For example, let's say we moved the document -available under `https://docs.gitlab.com/my-old-location/README.html` to a new location, -`https://docs.gitlab.com/my-new-location/index.html`. - -Into the **new document** front matter, we add the following information. You must -include the filename in the `disqus_identifier` URL, even if it's `index.html` or `README.html`. - -```yaml ---- -disqus_identifier: 'https://docs.gitlab.com/my-old-location/README.html' ---- -``` diff --git a/doc/development/documentation/site_architecture/global_nav.md b/doc/development/documentation/site_architecture/global_nav.md index ef6f3c0ae18..3e0534220a8 100644 --- a/doc/development/documentation/site_architecture/global_nav.md +++ b/doc/development/documentation/site_architecture/global_nav.md @@ -69,8 +69,7 @@ Sometimes pages for deprecated features are not in the global nav, depending on All other pages should be in the global nav. The technical writing team runs a report to determine which pages are not in the nav. -For now this report is manual, but [an issue exists](https://gitlab.com/gitlab-org/gitlab-docs/-/issues/1212) -to automate it. +The team reviews this list each month. ### Where to add diff --git a/doc/development/documentation/site_architecture/index.md b/doc/development/documentation/site_architecture/index.md index d4553614fad..849308f3086 100644 --- a/doc/development/documentation/site_architecture/index.md +++ b/doc/development/documentation/site_architecture/index.md @@ -33,7 +33,7 @@ Then you can use one of these approaches: of the documentation in the external repository. The landing page is indexed and searchable on <https://docs.gitlab.com>, but the rest of the documentation is not. For example, the [GitLab Workflow extension for VS Code](../../../user/project/repository/vscode.md). - We do not encourage the use of [pages with lists of links](../topic_types/index.md#topics-and-resources), + We do not encourage the use of [pages with lists of links](../topic_types/index.md#pages-and-topics-to-avoid), so only use this option if the recommended options are not feasible. ## Monthly release process (versions) diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md index 7f69a1f0d54..2925a0f0371 100644 --- a/doc/development/documentation/styleguide/index.md +++ b/doc/development/documentation/styleguide/index.md @@ -60,7 +60,7 @@ GitLab adds all troubleshooting information to the documentation, no matter how unlikely a user is to encounter a situation. GitLab Support maintains their own -[troubleshooting content](../../../administration/index.md#support-team-documentation) +[troubleshooting content](../../../administration/troubleshooting/index.md) in the GitLab documentation. ### The documentation includes all media types @@ -718,6 +718,15 @@ This is overridden by the [documentation-specific punctuation rules](#punctuatio Links help the docs adhere to the [single source of truth](#documentation-is-the-single-source-of-truth-ssot) principle. +However, you should avoid putting too many links on any page. Too many links can hinder readability. + +- Do not duplicate links on the same page. For example, on **Page A**, do not link to **Page B** multiple times. +- Avoid multiple links in a single paragraph. +- Avoid multiple links in a single task. +- On any one page, try not to use more than 15 links to other pages. +- Consider using [Related topics](../topic_types/index.md#related-topics) to reduce links that interrupt the flow of a task. +- Try to avoid anchor links to sections on the same page. Let users rely on the right navigation instead. + ### Links within the same repository To link to another page in the same repository, @@ -1449,7 +1458,7 @@ For tab titles, be brief and consistent. Ensure they are parallel, and start eac For example: - `Linux package (Omnibus)`, `Helm chart (Kubernetes)` (when documenting configuration edits, follow the - [configuration edits guide](#configuration-documentation-for-different-installation-methods)) + [configuration edits guide](#how-to-document-different-installation-methods)) - `15.1 and earlier`, `15.2 and later` Until we implement automated testing for broken links to tabs ([Issue 1355](https://gitlab.com/gitlab-org/gitlab-docs/-/issues/1355)), do not link directly to a single tab, even though they do have unique URL parameters. @@ -1546,24 +1555,28 @@ If the document resides outside of the `doc/` directory, use the full path instead of the relative link: `https://docs.gitlab.com/ee/administration/restart_gitlab.html`. -### Installation guide - -In [step 2 of the installation guide](../../../install/installation.md#2-ruby), -we install Ruby from source. To update the guide for a new Ruby version: +### How to document different installation methods -- Change the version throughout the code block. -- Replace the sha256sum. It's available on the - [downloads page](https://www.ruby-lang.org/en/downloads/) of the Ruby website. +GitLab supports five official installation methods. If you're referring to +words as part of sentences and titles, use the following phrases: -### Configuration documentation for different installation methods +- Linux package +- Helm chart +- GitLab Operator +- Docker +- Self-compiled -GitLab supports four installation methods: +It's OK to add the explanatory parentheses when +[using tabs](#use-tabs-to-describe-a-self-managed-configuration-procedure): - Linux package (Omnibus) - Helm chart (Kubernetes) +- GitLab Operator (Kubernetes) - Docker - Self-compiled (source) +### Use tabs to describe a self-managed configuration procedure + Configuration procedures can require users to edit configuration files, reconfigure GitLab, or restart GitLab. In this case: @@ -1576,8 +1589,8 @@ GitLab, or restart GitLab. In this case: - The final step to reconfigure or restart GitLab can be used verbatim since it's the same every time. -You can copy and paste the following snippet when describing a configuration -edit: +When describing a configuration edit, you can use and edit to your liking the +following snippet: <!-- markdownlint-disable tabs-blank-lines --> ````markdown diff --git a/doc/development/documentation/styleguide/word_list.md b/doc/development/documentation/styleguide/word_list.md index 1ea95367d01..eb0576d0c05 100644 --- a/doc/development/documentation/styleguide/word_list.md +++ b/doc/development/documentation/styleguide/word_list.md @@ -149,13 +149,6 @@ Instead of: This phrasing is more active and is from the user perspective, rather than the person who implemented the feature. [View details in the Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/a/allow-allows). -## Alpha - -Use uppercase for **Alpha**. For example: **The XYZ feature is in Alpha.** or **This Alpha release is ready to test.** - -You might also want to link to [this section](../../../policy/alpha-beta-support.md#alpha-features) -in the handbook when writing about Alpha features. - ## analytics Use lowercase for **analytics** and its variations, like **contribution analytics** and **issue analytics**. @@ -214,7 +207,7 @@ Try to avoid **below** when referring to an example or table in a documentation Use uppercase for **Beta**. For example: **The XYZ feature is in Beta.** or **This Beta release is ready to test.** -You might also want to link to [this section](../../../policy/alpha-beta-support.md#beta-features) +You might also want to link to [this section](../../../policy/alpha-beta-support.md#beta) in the handbook when writing about Beta features. ## blacklist @@ -286,8 +279,7 @@ CI/CD is always uppercase. No need to spell it out on first use. ## CI/CD minutes -Use **CI/CD minutes** instead of **CI minutes**, **pipeline minutes**, **pipeline minutes quota**, or -**CI pipeline minutes**. This decision was made in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342813). +Do not use **CI/CD minutes**. This term was renamed to [**compute credits**](#compute-credits). ## click @@ -297,7 +289,7 @@ Do not use **click**. Instead, use **select** with buttons, links, menu items, a ## cloud native When you're talking about using a Kubernetes cluster to host GitLab, you're talking about a **cloud-native version of GitLab**. -This version is different than the larger, more monolithic **Omnibus package** that is used to deploy GitLab. +This version is different than the larger, more monolithic **Linux package** that is used to deploy GitLab. You can also use **cloud-native GitLab** for short. It should be hyphenated and lowercase. @@ -305,6 +297,25 @@ You can also use **cloud-native GitLab** for short. It should be hyphenated and Use **collapse** instead of **close** when you are talking about expanding or collapsing a section in the UI. +## command line + +Use **From the command line** to introduce commands. + +Hyphenate when using as an adjective. For example, **a command-line tool**. + +## compute credits + +Use **compute credits** instead of these (or similar) terms: + +- **CI/CD minutes** +- **CI minutes** +- **pipeline minutes** +- **CI pipeline minutes** +- **pipeline minutes quota** + +This language is still being standardized in the documentation and UI beginning in March, 2023. +For more information, see [issue 5218](https://gitlab.com/gitlab-com/Product/-/issues/5218). + ## confirmation dialog Use **confirmation dialog** to describe the dialog box that asks you to confirm your action. For example: @@ -480,6 +491,13 @@ Instead of: Use **expand** instead of **open** when you are talking about expanding or collapsing a section in the UI. +## Experiment + +Use uppercase for **Experiment**. For example: **The XYZ feature is an Experiment.** or **This Experiment is ready to test.** + +You might also want to link to [this section](../../../policy/alpha-beta-support.md#experiment) +in the handbook when writing about Experiment features. + ## FAQ We want users to find information quickly, and they rarely search for the term **FAQ**. @@ -518,6 +536,17 @@ Filtering is different from [searching](#search). Do not use **foo** in product documentation. You can use it in our API and contributor documentation, but try to use a clearer and more meaningful example instead. +## fork + +A **fork** is a project that was created from a **upstream project** by using the +[forking process](../../../topics/git/terminology.md#fork). + +The **upstream project** (also known as the **source project**) and the **fork** have a **fork relationship** and are +**linked**. + +If the [**fork relationship** is removed](../../../user/project/repository/forking_workflow.md#unlink-a-fork), the +**fork** is **unlinked** from the **upstream project**. + ## future tense When possible, use present tense instead of future tense. For example, use **after you execute this command, GitLab displays the result** instead of **after you execute this command, GitLab will display the result**. ([Vale](../testing.md#vale) rule: [`FutureTense.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/FutureTense.yml)) @@ -552,6 +581,10 @@ Do not use **the `gitlab` chart**, **the GitLab Chart**, or **the cloud-native c You use the **GitLab Helm chart** to deploy **cloud-native GitLab** in a Kubernetes cluster. +If you use it in a context of describing the +[different installation methods](index.md#how-to-document-different-installation-methods). +use `Helm chart (Kubernetes)`. + ## GitLab Pages For consistency and branding, use **GitLab Pages** rather than **Pages**. @@ -561,7 +594,14 @@ you can use **Pages** thereafter. ## GitLab Runner -Use title case for **GitLab Runner**. This is the product you install. See also [runners](#runner-runners) and [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/233529). +Use title case for **GitLab Runner**. This is the product you install. For more information about the decision for this usage, +see [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/233529). + +See also: + +- [runners](#runner-runners) +- [runner managers](#runner-manager-runner-managers) +- [runner workers](#runner-worker-runner-workers) ## GitLab SaaS @@ -643,13 +683,29 @@ Do not use Latin abbreviations. Use **that is** instead. ([Vale](../testing.md#v Do not use **in order to**. Use **to** instead. ([Vale](../testing.md#vale) rule: [`Wordy.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/Wordy.yml)) +## Installation from source + +When referring to the installation method using the self-compiled code, refer to it +as **self-compiled**. + +Use: + +- For self-compiled installations... + +Instead of: + +- For installations from source... + +For more information, see the +[different installation methods](index.md#how-to-document-different-installation-methods). + ## -ing words Remove **-ing** words whenever possible. They can be difficult to translate, and more precise terms are usually available. For example: - Instead of **The files using storage are deleted**, use **The files that use storage are deleted**. -- Instead of **Delete files using the Edit button**, use **Delete files by using the Edit button**. +- Instead of **Delete files using the Edit button**, use **Use the Edit button to delete files**. - Instead of **Replicating your server is required**, use **You must replicate your server**. ## issue @@ -814,6 +870,23 @@ Use lowercase for **merge requests**. If you use **MR** as the acronym, spell it Use lowercase for **milestones**. +## Minimal Access + +When writing about the Minimal Access role: + +- Use a capital **M** and a capital **A**. +- Write it out: + - Use: if you are assigned the Minimal Access role + - Instead of: if you are a Minimal Access user + +- When the Minimal Access role is the minimum required role: + - Use: at least the Minimal Access role + - Instead of: the Minimal Access role or higher + +Do not use bold. + +Do not use **Minimal Access permissions**. A user who is assigned the Minimal Access role has a set of associated permissions. + ## n/a, N/A, not applicable When possible, use **not applicable**. Spelling out the phrase helps non-English speaking users and avoids @@ -876,6 +949,22 @@ Instead of: - Note that you can change the settings. +## Omnibus GitLab + +When referring to the installation method that uses the Linux package, refer to it +as **Linux package**. + +Use: + +- For installations that use the Linux package... + +Instead of: + +- For installations that use Omnibus GitLab... + +For more information, see the +[different installation methods](index.md#how-to-document-different-installation-methods). + ## on When documenting how to select high-level UI elements, use the word **on**. @@ -969,6 +1058,10 @@ Use lowercase for **personal access token**. Do not use **please**. For details, see the [Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/p/please). +## Premium + +Use **Premium**, in uppercase, for the subscription tier. + ## prerequisites Use **prerequisites** when documenting the steps before a task. Do not use **requirements**. @@ -1079,6 +1172,14 @@ Use lowercase for **runners**. These are the agents that run CI/CD jobs. See als When referring to runners, if you have to specify that the runners are installed on a customer's GitLab instance, use **self-managed** rather than **self-hosted**. +## runner manager, runner managers + +Use lowercase for **runner managers**. These are a type of runner that can create multiple runners for autoscaling. See also [GitLab Runner](#gitlab-runner). + +## runner worker, runner workers + +Use lowercase for **runner workers**. This is the process created by the runner on the host computing platform to run jobs. See also [GitLab Runner](#gitlab-runner). + ## (s) Do not use **(s)** to make a word optionally plural. It can slow down comprehension. For example: @@ -1239,6 +1340,13 @@ Use lowercase for **terminal**. For example: - Open a terminal. - From a terminal, run the `docker login` command. +## Terraform Module Registry + +Use title case for the GitLab Terraform Module Registry, but use lowercase `m` when +talking about non-specific modules. For example: + +- You can publish a Terraform module to your project's Terraform Module Registry. + ## text box Use **text box** instead of **field** or **box** when referring to the UI element. @@ -1314,6 +1422,10 @@ For example: See also [**enter**](#enter). +## Ultimate + +Use **Ultimate**, in uppercase, for the subscription tier. + ## units of measurement Use a space between the number and the unit of measurement. For example, **128 GB**. diff --git a/doc/development/documentation/testing.md b/doc/development/documentation/testing.md index 4f61b2424eb..4a7e206da20 100644 --- a/doc/development/documentation/testing.md +++ b/doc/development/documentation/testing.md @@ -49,17 +49,21 @@ To run tests locally, it's important to: ### Lint checks Lint checks are performed by the [`lint-doc.sh`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/lint-doc.sh) -script and can be executed as follows: +script and can be executed with the help of a Rake task as follows: -1. Go to the `gitlab` directory. +1. Go to your `gitlab` directory. 1. Run: ```shell - MD_DOC_PATH=path/to/my_doc.md scripts/lint-doc.sh + rake lint:markdown ``` -Where `MD_DOC_PATH` points to the file or directory you would like to run lint checks for. -If you omit it completely, it defaults to the `doc/` directory. +To specify a single file or directory you would like to run lint checks for, run: + +```shell +MD_DOC_PATH=path/to/my_doc.md rake lint:markdown +``` + The output should be similar to: ```plaintext @@ -77,7 +81,7 @@ The output should be similar to: This requires you to either: - Have the [required lint tools installed](#local-linters) on your computer. -- A working Docker installation, in which case an image with these tools pre-installed is used. +- A working Docker or containerd installation, to use an image with these tools pre-installed. ### Documentation link tests diff --git a/doc/development/documentation/topic_types/glossary.md b/doc/development/documentation/topic_types/glossary.md new file mode 100644 index 00000000000..4985101a391 --- /dev/null +++ b/doc/development/documentation/topic_types/glossary.md @@ -0,0 +1,70 @@ +--- +stage: none +group: Style Guide +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Glossary topic type + +A glossary provides a list of unfamiliar terms and their definitions to help users understand a specific +GitLab feature. + +Each glossary item provides a single term and its associated definition. The definition should answer the questions: + +- **What** is this? +- **Why** would you use it? + +For glossary terms: + +- Do not use jargon. +- Do not use internal terminology or acronyms. +- Ensure the correct usage is defined in the [word list](../styleguide/word_list.md). + +## Alternatives to glossaries + +Glossaries should provide short, concise term-definition pairs. + +- If a definition requires more than a brief explanation, use a concept topic instead. +- If you find yourself explaining how to use the feature, use a task topic instead. + +## Glossary example + +Glossary topics should be in this format. Use an unordered list primarily. You can use a table if you need to apply +additional categorization. + +Try to include glossary topics on pages that explain the feature, rather than as a standalone page. + +```markdown +## FeatureName glossary + +This glossary provides definitions for terms related to FeatureName. + +- **Term A**: Term A does this thing. +- **Term B**: Term B does this thing. +- **Term C**: Term C does this thing. +``` + +If you use the table format: + +```markdown +## FeatureName glossary + +This glossary provides definitions for terms related to FeatureName. + +| Term | Definition | Additional category | +|--------|-------------------------|---------------------| +| Term A | Term A does this thing. | | +| Term B | Term B does this thing. | | +| Term C | Term C does this thing. | | +``` + +## Glossary topic titles + +Use `FeatureName glossary`. + +Don't use alternatives to `glossary`. For example: + +- `Terminology` +- `Glossary of terms` +- `Glossary of common terms` +- `Definitions` diff --git a/doc/development/documentation/topic_types/index.md b/doc/development/documentation/topic_types/index.md index cfc231c268a..37ed876fc59 100644 --- a/doc/development/documentation/topic_types/index.md +++ b/doc/development/documentation/topic_types/index.md @@ -16,22 +16,52 @@ Each topic on a page should be one of the following topic types: Even if a page is short, the page usually starts with a concept and then includes a task or reference topic. -The tech writing team sometimes uses the acronym `CTRT` to refer to our topic types. +The tech writing team sometimes uses the acronym `CTRT` to refer to the topic types. The acronym refers to the first letter of each topic type. -In addition to the four primary topic types, we also have a page type for -[Tutorials](tutorial.md) and [Get started](#get-started). +## Other page and topic types + +In addition to the four primary topic types, you can use the following: + +- Page types: [Tutorials](tutorial.md) and [Get started](#get-started) +- Topic type: [Related topics](#related-topics) +- Page or topic type: [Glossaries](glossary.md) + +## Pages and topics to avoid + +You should avoid: + +- Pages that are exclusively links to other pages. The only exception are + top-level pages that aid with navigation. +- Topics that have one or two sentences only. In these cases: + - Incorporate the information in another topic. + - If the sentence links to another page, use a [Related topics](#related-topics) link instead. + +## Topic title guidelines + +In general, for topic titles: + +- Be clear and direct. Make every word count. +- Use articles and prepositions. +- Follow [capitalization](../styleguide/index.md#capitalization) guidelines. +- Do not repeat text from earlier topic titles. For example, if the page is about merge requests, + instead of `Troubleshooting merge requests`, use only `Troubleshooting`. + +See also [guidelines for heading levels in Markdown](../styleguide/index.md#heading-levels-in-markdown). ## Related topics If inline links are not sufficient, you can create a topic called **Related topics** and include an unordered list of related topics. This topic should be above the Troubleshooting section. +Links in this section should be brief and scannable. They are usually not +full sentences, and so should not end in a period. + ```markdown ## Related topics -- [Configure your pipeline](link-to-topic). -- [Trigger a pipeline manually](link-to-topic). +- [CI/CD variables](link-to-topic) +- [Environment variables](link-to-topic) ``` ## Get started @@ -57,22 +87,3 @@ consider using subsections for each distinct task. In the left nav, use `Get started` as the text. On the page itself, spell out the full name. For example, `Get started with application security`. - -## Topics and resources - -Some pages are solely a list of links to other documentation. - -We do not encourage this page type. Lists of links can get out-of-date quickly -and offer little value to users, who prefer to search to find information. - -## Topic title guidelines - -In general, for topic titles: - -- Be clear and direct. Make every word count. -- Use articles and prepositions. -- Follow [capitalization](../styleguide/index.md#capitalization) guidelines. -- Do not repeat text from earlier topic titles. For example, if the page is about merge requests, - instead of `Troubleshooting merge requests`, use only `Troubleshooting`. - -See also [guidelines for heading levels in Markdown](../styleguide/index.md#heading-levels-in-markdown). diff --git a/doc/development/documentation/topic_types/task.md b/doc/development/documentation/topic_types/task.md index f596ffc9b8b..2ddfd841ec3 100644 --- a/doc/development/documentation/topic_types/task.md +++ b/doc/development/documentation/topic_types/task.md @@ -155,4 +155,4 @@ how to write the phrase for each role. ## Related topics -- [View the format for writing task steps](../styleguide/index.md#navigation). +- [How to write task steps](../styleguide/index.md#navigation) diff --git a/doc/development/documentation/topic_types/tutorial.md b/doc/development/documentation/topic_types/tutorial.md index 1b1426a0465..b75b1e6b629 100644 --- a/doc/development/documentation/topic_types/tutorial.md +++ b/doc/development/documentation/topic_types/tutorial.md @@ -13,9 +13,17 @@ In general, you might consider using a tutorial when: of sub-steps. - The steps cover a variety of GitLab features or third-party tools. -Tutorials are learning aids that complement our core documentation. -They do not introduce new features. -Always use the primary [topic types](index.md) to document new features. +## Tutorial guidance + +- Tutorials are not [tasks](task.md). A task gives instructions for one procedure. + A tutorial combines multiple tasks to achieve a specific goal. +- Tutorials provide a working example. Ideally the reader can create the example the + tutorial describes. If they can't replicate it exactly, they should be able + to replicate something similar. +- Tutorials do not introduce new features. +- Tutorials do not need to adhere to the Single Source of Truth tenet. While it's not + ideal to duplicate content that is available elsewhere, it's worse to force the reader to + leave the page to find what they need. ## Tutorial format @@ -31,7 +39,9 @@ To create a website: 1. [Do the first task](#do-the-first-task) 1. [Do the second task](#do-the-second-task) -Prerequisites (optional): +## Prerequisites + +This topic is optional. - Thing 1 - Thing 2 diff --git a/doc/development/emails.md b/doc/development/emails.md index 2d03d8bca2f..fdcdcec814d 100644 --- a/doc/development/emails.md +++ b/doc/development/emails.md @@ -151,37 +151,10 @@ fork of [`MailRoom`](https://github.com/tpitale/mail_room/), to ensure that we can make updates quickly to the gem if necessary. We try to upstream changes as soon as possible and keep the two projects in sync. -Updating the `gitlab-mail_room` dependency in `Gemfile` is deprecated at -the moment in favor of updating the version in -[Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/config/software/mail_room.rb) -(see [example merge request](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5816)) -and Helm Chart configuration (see [example merge request](https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/854)). +To update MailRoom: -#### Rationale - -This was done because to avoid [thread deadlocks](https://github.com/ruby/net-imap/issues/14), `MailRoom` needs -an updated version of the `net-imap` gem. However, this -[version of the net-imap cannot be installed by an unprivileged user](https://github.com/ruby/net-imap/issues/14) due to -[an error installing the digest gem](https://github.com/ruby/digest/issues/14). -[This bug in the Ruby interpreter](https://bugs.ruby-lang.org/issues/17761) was fixed in Ruby -3.0.2. - -Updating the gem directly in the GitLab Rails `Gemfile` caused a [production incident](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/4053) -since Kubernetes pods do not run as privileged users. - -Note that Omnibus GitLab runs `/opt/gitlab/embedded/bin/mail_room` -instead of `bundle exec rake` to avoid loading the older version. With a -Kubernetes install, the MailRoom version has always been explicitly set -in the Helm Chart configuration rather than the `Gemfile`. - -#### Preserving backwards compatibility - -Removing the `Gemfile` would break incoming email processing for source -installs. For now, source installs are advised to upgrade manually to -the version specified in Omnibus and run `bin/mail_room` directly as -done with Omnibus. - -We can reconsider this deprecation once we upgrade to Ruby 3.0. +1. Update `Gemfile` in GitLab Rails (see [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116494)). +1. Update the Helm Chart configuration (see [example merge request](https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/854)). --- diff --git a/doc/development/experiment_guide/index.md b/doc/development/experiment_guide/index.md index fd5b0ea15e6..5e68921510e 100644 --- a/doc/development/experiment_guide/index.md +++ b/doc/development/experiment_guide/index.md @@ -29,7 +29,7 @@ the documentation on that project if you want to dig into more advanced topics o aware that the documentation there reflects what's in the main branch and may not be the same as the version being used in GitLab. -## Glossary of terms +## Glossary To ensure a shared language, you should understand these fundamental terms we use when communicating about experiments: diff --git a/doc/development/fe_guide/content_editor.md b/doc/development/fe_guide/content_editor.md index 5c7fe68fec5..25140a067ca 100644 --- a/doc/development/fe_guide/content_editor.md +++ b/doc/development/fe_guide/content_editor.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/fe_guide/customizable_dashboards.md b/doc/development/fe_guide/customizable_dashboards.md index 26c73b26126..9e45c660745 100644 --- a/doc/development/fe_guide/customizable_dashboards.md +++ b/doc/development/fe_guide/customizable_dashboards.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Customizable dashboards -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98610) in GitLab 15.5 as an [Alpha feature](../../policy/alpha-beta-support.md#alpha-features). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98610) in GitLab 15.5 as an [Experiment](../../policy/alpha-beta-support.md#experiment). Customizable dashboards provide a dashboard structure that allows users to create their own dashboards and commit the structure to a repository. diff --git a/doc/development/fe_guide/graphql.md b/doc/development/fe_guide/graphql.md index 5f0855d8f91..da3a6eff79d 100644 --- a/doc/development/fe_guide/graphql.md +++ b/doc/development/fe_guide/graphql.md @@ -1129,53 +1129,7 @@ We use [subscriptions](https://www.apollographql.com/docs/react/data/subscriptio **NOTE:** We cannot test subscriptions using GraphiQL, because they require an ActionCable client, which GraphiQL does not support at the moment. -Subscriptions don't require any additional configuration of Apollo Client instance, you can use them in the application right away. To distinguish subscriptions from queries and mutations, we recommend naming them with `.subscription.graphql` extension: - -```graphql -// ~/sidebar/queries/issuable_assignees.subscription.graphql - -subscription issuableAssigneesUpdated($issuableId: IssuableID!) { - issuableAssigneesUpdated(issuableId: $issuableId) { - ... on Issue { - assignees { - nodes { - ...User - status { - availability - } - } - } - } - } -} -``` - -When using GraphQL subscriptions in Vue application, we recommend updating existing Apollo query results with [subscribeToMore](https://apollo.vuejs.org/guide/apollo/subscriptions.html#subscribe-to-more) option: - -```javascript -import issuableAssigneesSubscription from '~/sidebar/queries/issuable_assignees.subscription.graphql' - -apollo: { - issuable: { - query() { - return assigneesQueries[this.issuableType].query; - }, - subscribeToMore: { - // Specify the subscription that will update the query - document() { - return issuableAssigneesSubscription; - }, - variables() { - return { - issuableId: convertToGraphQLId(this.issuableClass, this.issuableId), - }; - }, - }, - }, -}, -``` - -We would need also to define a field policy similarly like we do it for the [paginated queries](#defining-field-merge-policy) +Refer to the [Real-time widgets developer guide](../real_time.md) for a comprehensive introduction to subscriptions. ### Best Practices @@ -1446,6 +1400,24 @@ describe('when query times out', () => { }); ``` +Previously, we've used `{ mocks: { $apollo ...}}` on `mount` to test Apollo functionality. This approach is discouraged - proper `$apollo` mocking leaks a lot of implementation details to the tests. Consider replacing it with mocked Apollo provider + +```javascript +wrapper = mount(SomeComponent, { + mocks: { + // avoid! Mock real graphql queries and mutations instead + $apollo: { + mutate: jest.fn(), + queries: { + groups: { + loading, + }, + }, + }, + }, +}); +``` + #### Testing `@client` queries ##### Using mock resolvers diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 6403cff549f..ef506f260f7 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -11,7 +11,7 @@ across the GitLab frontend team. ## Overview -GitLab is built on top of [Ruby on Rails](https://rubyonrails.org). It uses [Haml](https://haml.info/) and a JavaScript-based frontend with [Vue.js](https://vuejs.org). +GitLab is built on top of [Ruby on Rails](https://rubyonrails.org). It uses [Haml](https://haml.info/) and a JavaScript-based frontend with [Vue.js](https://vuejs.org). If you are not sure when to use Vue on top of Haml-page, please read [this explanation](vue.md#when-to-add-vue-application). <!-- vale gitlab.Spelling = NO --> diff --git a/doc/development/fe_guide/merge_request_widget_extensions.md b/doc/development/fe_guide/merge_request_widget_extensions.md index 4242dd837f8..07029aec015 100644 --- a/doc/development/fe_guide/merge_request_widget_extensions.md +++ b/doc/development/fe_guide/merge_request_widget_extensions.md @@ -329,7 +329,6 @@ To generate these known events for a single widget: 1. `product_section` = `dev` 1. `product_stage` = `create` 1. `product_group` = `code_review` - 1. `product_category` = `code_review` 1. `introduced_by_url` = `'[your MR]'` 1. `options.events` = (the event in the command from above that generated this file, like `i_code_review_merge_request_widget_test_reports_count_view`) - This value is how the telemetry events are linked to "metrics" so this is probably one of the more important values. diff --git a/doc/development/fe_guide/performance.md b/doc/development/fe_guide/performance.md index 20609718217..432e66bee33 100644 --- a/doc/development/fe_guide/performance.md +++ b/doc/development/fe_guide/performance.md @@ -474,4 +474,4 @@ General tips: - [WebPage Test](https://www.webpagetest.org) for testing site loading time and size. - [Google PageSpeed Insights](https://pagespeed.web.dev/) grades web pages and provides feedback to improve the page. - [Profiling with Chrome DevTools](https://developer.chrome.com/docs/devtools/) -- [Browser Diet](https://browserdiet.com/) is a community-built guide that catalogues practical tips for improving web page performance. +- [Browser Diet](https://github.com/zenorocha/browser-diet) was a community-built guide that cataloged practical tips for improving web page performance. diff --git a/doc/development/fe_guide/registry_architecture.md b/doc/development/fe_guide/registry_architecture.md index d86f8416db6..cf267e80619 100644 --- a/doc/development/fe_guide/registry_architecture.md +++ b/doc/development/fe_guide/registry_architecture.md @@ -14,7 +14,7 @@ Existing registries: - Package Registry - Container Registry -- Infrastructure Registry +- Terraform Module Registry - Dependency Proxy ## Frontend architecture diff --git a/doc/development/fe_guide/source_editor.md b/doc/development/fe_guide/source_editor.md index 9f52850041d..2c115effcf9 100644 --- a/doc/development/fe_guide/source_editor.md +++ b/doc/development/fe_guide/source_editor.md @@ -17,6 +17,14 @@ GitLab features use it, including: - [Web Editor](../../user/project/repository/web_editor.md) - [Security Policies](../../user/application_security/policies/index.md) +## When to use Source Editor + +Use Source Editor only when users need to edit the file content. +If you only need to display source code, consider using the [`BlobContent`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/blob/components/blob_content.vue) component. + +If the page you're working on is already loading the Source Editor, +displaying read-only content in the Source Editor is still a valid option. + ## How to use Source Editor Source Editor is framework-agnostic and can be used in any application, including both @@ -56,7 +64,7 @@ An instance of Source Editor accepts the following configuration options: | `blobContent` | `false` | `String`: The initial content to render in the editor. | | `extensions` | `false` | `Array`: Extensions to use in this instance. | | `blobGlobalId` | `false` | `String`: An auto-generated property.<br>**Note:** This property may go away in the future. Do not pass `blobGlobalId` unless you know what you're doing.| -| Editor Options | `false` | `Object(s)`: Any property outside of the list above is treated as an Editor Option for this particular instance. Use this field to override global Editor Options on the instance level. A full [index of Editor Options](https://microsoft.github.io/monaco-editor/api/enums/monaco.editor.EditorOption.html) is available. | +| Editor Options | `false` | `Object(s)`: Any property outside of the list above is treated as an Editor Option for this particular instance. Use this field to override global Editor Options on the instance level. A full [index of Editor Options](https://microsoft.github.io/monaco-editor/docs.html#enums/editor.EditorOption.html) is available. | ## API @@ -68,7 +76,7 @@ with additional functions on the instance level: | --------------------- | ----- | ----- | | `updateModelLanguage` | `path`: String | Updates the instance's syntax highlighting to follow the extension of the passed `path`. Available only on the instance level. | | `use` | Array of objects | Array of extensions to apply to the instance. Accepts only an array of **objects**. The extensions' ES6 modules must be fetched and resolved in your views or components before they're passed to `use`. Available on the instance and global editor (all instances) levels. | -| Monaco Editor options | See [documentation](https://microsoft.github.io/monaco-editor/api/interfaces/monaco.editor.IStandaloneCodeEditor.html) | Default Monaco editor options. | +| Monaco Editor options | See [documentation](https://microsoft.github.io/monaco-editor/docs.html#interfaces/editor.IStandaloneCodeEditor.html) | Default Monaco editor options. | ## Tips diff --git a/doc/development/fe_guide/style/javascript.md b/doc/development/fe_guide/style/javascript.md index b35ffdd8669..987543642f0 100644 --- a/doc/development/fe_guide/style/javascript.md +++ b/doc/development/fe_guide/style/javascript.md @@ -2,7 +2,6 @@ stage: none group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/development/fe_guide/style_guide_js.html' --- # JavaScript style guide @@ -332,19 +331,23 @@ Only export the constants as a collection (array, or object) when there is a nee ## Error handling -When catching a server-side error you should use the error message +When catching a server-side error, you should use the error message utility function contained in `app/assets/javascripts/lib/utils/error_message.js`. -This utility parses the received error message and checks for a prefix that indicates -whether the message is meant to be user-facing or not. The utility returns -an object with the message, and a boolean indicating whether the message is meant to be user-facing or not. Please make sure that the Backend is aware of the utils usage and is adding the prefix -to the error message accordingly. +This utility accepts two parameters: the error object received from the server response and a +default error message. The utility examines the message in the error object for a prefix that +indicates whether the message is meant to be user-facing or not. If the message is intended +to be user-facing, the utility returns it as is. Otherwise, it returns the default error +message passed as a parameter. ```javascript import { parseErrorMessage } from '~/lib/utils/error_message'; onError(error) { - const { message, userFacing } = parseErrorMessage(error); - - const errorMessage = userFacing ? message : genericErrorText; + const errorMessage = parseErrorMessage(error, genericErrorText); } ``` + +To benefit from this parsing mechanism, the utility user should ensure that the server-side +code is aware of this utility's usage and prefixes the error messages where appropriate +before sending them back to the user. See +[Error handling for API](../../api_styleguide.md#error-handling) for more information. diff --git a/doc/development/fe_guide/style/scss.md b/doc/development/fe_guide/style/scss.md index aed7310e95d..e760b0adaaa 100644 --- a/doc/development/fe_guide/style/scss.md +++ b/doc/development/fe_guide/style/scss.md @@ -2,7 +2,6 @@ stage: none group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/development/fe_guide/style_guide_scss.html' --- # SCSS style guide @@ -77,7 +76,7 @@ These mixins should be used to replace _magic values_ in our code. For example a `margin-top: 8px` is a good candidate for the `@include gl-mt-3` mixin replacement. Avoid using utility mixins for [pre-defined CSS keywords](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Values_and_Units#pre-defined_keyword_values). -For example prefer `display: flex` over `@include gl-display-flex`. +For example prefer `display: flex` over `@include gl-display-flex`. Utility mixins are particularly useful for encapsulating our design system but there is no need to encapsulate simple properties. ```scss // Bad diff --git a/doc/development/fe_guide/style/vue.md b/doc/development/fe_guide/style/vue.md index e9d2a724d9d..a3ab1c1af30 100644 --- a/doc/development/fe_guide/style/vue.md +++ b/doc/development/fe_guide/style/vue.md @@ -650,6 +650,18 @@ over [`expect.objectContaining`](https://jestjs.io/docs/expect#expectobjectconta }); ``` +### Testing props validation + +1. When checking component props use `assertProps` helper. Props validation failures will be thrown as errors + +```javascript +import { assertProps } from 'helpers/assert_props' + +// ... + +expect(() => assertProps(SomeComponent, { invalidPropValue: '1', someOtherProp: 2 })).toThrow() +``` + ## The JavaScript/Vue Accord The goal of this accord is to make sure we are all on the same page. diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index 119a91e33b5..65ceb9f0220 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -16,11 +16,23 @@ What is described in the following sections can be found in these examples: - [Security products](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/app/assets/javascripts/vue_shared/security_reports) - [Registry](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/app/assets/javascripts/registry/stores) +## When to add Vue application + +Sometimes, HAML page is enough to satisfy requirements. This statement is correct primarily for the static pages or pages that have very little logic. How do we know it's worth adding a Vue application to the page? The answer is "when we need to maintain application state and synchronize the rendered page with it". + +To better explain this, let's imagine the page that has one toggle, and toggling it sends an API request. This case does not involve any state we want to maintain, we send the request and switch the toggle. However, if we add ont more toggle that should always be the opposite to the first one, we need a _state_: one toggle should be "aware" about the state of another one. When written in plain JavaScript, this logic usually involves listening to DOM event and reacting with modifying DOM. Cases like this are much easier to handle with Vue.js so we should create a Vue application here. + +### What are some flags signaling that you might need Vue application? + +- when you need to define complex conditionals based on multiple factors and update them on user interaction; +- when you have to maintain any form of application state and share it between tags/elements; +- when you expect complex logic to be added in the future - it's easier to start with basic Vue application than having to rewrite JS/HAML to Vue on the next step. + ## Vue architecture -All new features built with Vue.js must follow a [Flux architecture](https://facebook.github.io/flux/). +All new features built with Vue.js must follow a [Flux architecture](https://facebookarchive.github.io/flux/). The main goal we are trying to achieve is to have only one data flow, and only one data entry. -To achieve this goal we use [Vuex](#vuex). +To achieve this goal we use [Vuex](#vuex) or [Apollo Client](graphql.md#libraries) You can also read about this architecture in Vue documentation about [state management](https://v2.vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch) @@ -121,7 +133,7 @@ export default { ```javascript //index.js import MyComponent from './my_component.vue' -import initSimpleApp from '~/helpers/init_simple_app_helper' +import { initSimpleApp } from '~/helpers/init_simple_app_helper' initSimpleApp('#js-my-element', MyComponent) ``` @@ -391,6 +403,87 @@ You can read more about components in Vue.js site, [Component System](https://v2 Check this [page](vuex.md) for more details. +### Vue Router + +To add [Vue Router](https://router.vuejs.org/) to a page: + +1. Add a catch-all route to the Rails route file using a wildcard named `*vueroute`: + + ```ruby + # example from ee/config/routes/project.rb + + resources :iteration_cadences, path: 'cadences(/*vueroute)', action: :index + ``` + + The above example serves the `index` page from `iteration_cadences` controller to any route + matching the start of the `path`, for example `groupname/projectname/-/cadences/123/456/`. +1. Pass the base route (everything before `*vueroute`) to the frontend to use as the `base` parameter to initialize Vue Router: + + ```haml + .js-my-app{ data: { base_path: project_iteration_cadences_path(project) } } + ``` + +1. Initialize the router: + + ```javascript + Vue.use(VueRouter); + + export function createRouter(basePath) { + return new VueRouter({ + routes: createRoutes(), + mode: 'history', + base: basePath, + }); + } + ``` + +1. Add a fallback for unrecognised routes with `path: '*'`. Either: + - Add a redirect to the end of your routes array: + + ```javascript + const routes = [ + { + path: '/', + name: 'list-page', + component: ListPage, + }, + { + path: '*', + redirect: '/', + }, + ]; + ``` + + - Add a fallback component to the end of your routes array: + + ```javascript + const routes = [ + { + path: '/', + name: 'list-page', + component: ListPage, + }, + { + path: '*', + component: NotFound, + }, + ]; + ``` + +1. Optional. To also allow using the path helper for child routes, add `controller` and `action` + parameters to use the parent controller. + + ```ruby + resources :iteration_cadences, path: 'cadences(/*vueroute)', action: :index do + resources :iterations, only: [:index, :new, :edit, :show], constraints: { id: /\d+/ }, controller: :iteration_cadences, action: :index + end + ``` + + This means routes like `/cadences/123/iterations/456/edit` can be validated on the backend, + for example to check group or project membership. + It also means we can use the `_path` helper, which means we can load the page in feature specs + without manually building the `*vueroute` part of the path.. + ### Mixing Vue and jQuery - Mixing Vue and jQuery is not recommended. diff --git a/doc/development/feature_development.md b/doc/development/feature_development.md index c3cee596d8a..5828855a725 100644 --- a/doc/development/feature_development.md +++ b/doc/development/feature_development.md @@ -80,7 +80,7 @@ Consult these topics for information on contributing to specific GitLab features - [Sidekiq guidelines](sidekiq/index.md) for working with Sidekiq workers - [Working with Gitaly](gitaly.md) - [Advanced search integration docs](advanced_search.md) -- [Working with merge request diffs](diffs.md) +- [Working with merge request diffs](merge_request_concepts/diffs/index.md) - [Approval Rules](merge_request_concepts/approval_rules.md) - [Repository mirroring](repository_mirroring.md) - [Uploads development guide](uploads/index.md) @@ -130,9 +130,8 @@ See [database guidelines](database/index.md). The following integration guides are internal. Some integrations require access to administrative accounts of third-party services and are available only for GitLab team members to contribute to: -- [Jira app development](https://gitlab.com/gitlab-org/manage/integrations/team/-/blob/main/integrations/jira.md) -- [Slack app development](https://gitlab.com/gitlab-org/manage/integrations/team/-/blob/main/integrations/slack.md) -- [ZenTao development](https://gitlab.com/gitlab-org/manage/integrations/team/-/blob/main/integrations/zentao.md) +- [Jira app development](https://gitlab.com/gitlab-org/manage/integrate/team/-/blob/main/integrations/jira.md) +- [GitLab for Slack app development](https://gitlab.com/gitlab-org/manage/integrate/team/-/blob/main/integrations/slack.md) ## Testing guides diff --git a/doc/development/feature_flags/controls.md b/doc/development/feature_flags/controls.md index f8a592f98f5..0e88c68cc99 100644 --- a/doc/development/feature_flags/controls.md +++ b/doc/development/feature_flags/controls.md @@ -40,7 +40,7 @@ easier to measure the impact of both separately. The GitLab feature library (using [Flipper](https://github.com/jnunemaker/flipper), and covered in the -[Feature Flags process](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/) guide) supports rolling out changes to a percentage of +[Feature flags process](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/) guide) supports rolling out changes to a percentage of time to users. This in turn can be controlled using [GitLab ChatOps](../../ci/chatops/index.md). For an up to date list of feature flag commands please see @@ -115,13 +115,13 @@ incidents or in-progress change issues, for example: 2021-06-29 Canary deployment failing QA tests ``` -Before enabling a feature flag, verify that you are not violating any [Production Change Lock periods](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/#production-change-lock-pcl) and are in compliance with the [Feature Flags and the Change Management Process](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/#feature-flags-and-the-change-management-process). +Before enabling a feature flag, verify that you are not violating any [Production Change Lock periods](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/#production-change-lock-pcl) and are in compliance with the [Feature flags and the Change Management Process](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/#feature-flags-and-the-change-management-process). The following `/chatops` commands should be performed in the Slack `#production` channel. When you begin to enable the feature, please link to the relevant -Feature Flag Rollout Issue within a Slack thread of the first `/chatops` +feature flag rollout issue within a Slack thread of the first `/chatops` command you make so people can understand the change if they need to. To enable a feature for 25% of the time, run the following in Slack: @@ -358,7 +358,7 @@ After turning on the feature flag, you need to [monitor the relevant graphs](htt In this illustration, you can see that the Apdex score started to decline after the feature flag was enabled at `09:46`. The feature flag was then deactivated at `10:31`, and the service returned to the original value: - + ### Feature flag change logging diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md index 9b6876ac0bc..40a5d2a2abc 100644 --- a/doc/development/feature_flags/index.md +++ b/doc/development/feature_flags/index.md @@ -83,7 +83,7 @@ used for deploying unfinished code to production. Most feature flags used at GitLab are the `development` type. A `development` feature flag must have a rollout issue -created from the [Feature Flag Roll Out template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20Flag%20Roll%20Out.md). +created from the [Feature flag Roll Out template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20Flag%20Roll%20Out.md). The format for `development` feature flags is `Feature.<state>(:<dev_flag_name>)`. To enable and disable them, run on the GitLab Rails console: @@ -252,7 +252,7 @@ deleting feature flags. ## Develop with a feature flag -There are two main ways of using Feature Flags in the GitLab codebase: +There are two main ways of using feature flags in the GitLab codebase: - [Backend code (Rails)](#backend) - [Frontend code (VueJS)](#frontend) diff --git a/doc/development/features_inside_dot_gitlab.md b/doc/development/features_inside_dot_gitlab.md index dda2c05288f..f1915327c8c 100644 --- a/doc/development/features_inside_dot_gitlab.md +++ b/doc/development/features_inside_dot_gitlab.md @@ -13,9 +13,9 @@ When implementing new features, please refer to these existing features to avoid - [Issue Templates](../user/project/description_templates.md#create-an-issue-template): `.gitlab/issue_templates/`. - [Merge request Templates](../user/project/description_templates.md#create-a-merge-request-template): `.gitlab/merge_request_templates/`. - [GitLab agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/configuration_repository.md#layout): `.gitlab/agents/`. -- [CODEOWNERS](../user/project/code_owners.md#set-up-code-owners): `.gitlab/CODEOWNERS`. +- [CODEOWNERS](../user/project/codeowners/index.md#set-up-code-owners): `.gitlab/CODEOWNERS`. - [Route Maps](../ci/review_apps/index.md#route-maps): `.gitlab/route-map.yml`. - [Customize Auto DevOps Helm Values](../topics/autodevops/customize.md#customize-helm-chart-values): `.gitlab/auto-deploy-values.yaml`. - [Insights](../user/project/insights/index.md#configure-project-insights): `.gitlab/insights.yml`. -- [Service Desk Templates](../user/project/service_desk.md#using-customized-email-templates): `.gitlab/service_desk_templates/`. +- [Service Desk Templates](../user/project/service_desk.md#create-customized-email-templates): `.gitlab/service_desk_templates/`. - [Web IDE](../user/project/web_ide/index.md#web-ide-configuration-file): `.gitlab/.gitlab-webide.yml`. diff --git a/doc/development/gemfile.md b/doc/development/gemfile.md index 38c0d77bcae..d4dc66bae8f 100644 --- a/doc/development/gemfile.md +++ b/doc/development/gemfile.md @@ -65,6 +65,17 @@ This means that new dependencies should, at a minimum, meet the following criter - If the project uses a C extension, consider requesting an additional review from a C or MRI domain expert. C extensions can greatly impact GitLab stability and performance. +## Gems that require a domain expert approval + +Changes to the following gems require a domain expert review and approval by a backend team member of the group. + +For gems not listed in this table, it's still recommended but not required that you find a domain expert to review changes. + +| Gem | Requires approval by | +| ------ | ------ | +| `doorkeeper` | [Manage:Authentication and Authorization](https://about.gitlab.com/handbook/product/categories/#authentication-and-authorization-group) | +| `doorkeeper-openid_connect` | [Manage:Authentication and Authorization](https://about.gitlab.com/handbook/product/categories/#authentication-and-authorization-group) | + ## Request an Appsec review When adding a new gem to our `Gemfile` or even changing versions in diff --git a/doc/development/git_object_deduplication.md b/doc/development/git_object_deduplication.md index 6014ccbfb39..e98ebe5efe1 100644 --- a/doc/development/git_object_deduplication.md +++ b/doc/development/git_object_deduplication.md @@ -23,15 +23,17 @@ At the Git level, we achieve deduplication by using Git alternates is a mechanism that lets a repository borrow objects from another repository on the same machine. -If we want repository A to borrow from repository B, we first write a -path that resolves to `B.git/objects` in the special file -`A.git/objects/info/alternates`. This establishes the alternates link. -Next, we must perform a Git repack in A. After the repack, any objects -that are duplicated between A and B are deleted from A. Repository -A is now no longer self-contained, but it still has its own refs and -configuration. Objects in A that are not in B remain in A. For this -to work, it is of course critical that **no objects ever get deleted from -B** because A might need them. +To make repository A borrow from repository B: + +1. Establish the alternates link in the special file `A.git/objects/info/alternates` + by writing a path that resolves to `B.git/objects`. +1. In repository A, run `git repack` to remove all objects in repository A that + also exist in repository B. + +After the repack, repository A is no longer self-contained, but still contains its +own refs and configuration. Objects in A that are not in B remain in A. For this +configuration to work, **objects must not be deleted from repository B** because +repository A might need them. WARNING: Do not run `git prune` or `git gc` in object pool repositories, which are diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md index addab512f89..408f3d5fa3c 100644 --- a/doc/development/gitaly.md +++ b/doc/development/gitaly.md @@ -54,16 +54,6 @@ they are merged. - See [below](#running-tests-with-a-locally-modified-version-of-gitaly) for instructions on running GitLab tests with a modified version of Gitaly. - In GDK run `gdk install` and restart GDK using `gdk restart` to use a locally modified Gitaly version for development -### `gitaly-ruby` - -It is possible to implement and test RPCs in Gitaly using Ruby code, -in -[`gitaly-ruby`](https://gitlab.com/gitlab-org/gitaly/tree/master/ruby). -This should make it easier to contribute for developers who are less -comfortable writing Go code. - -For more information, see the [Beginner's guide to Gitaly contributions](https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/beginners_guide.md). - ## Gitaly-Related Test Failures If your test-suite is failing with Gitaly issues, as a first step, try running: @@ -249,7 +239,7 @@ Re-run steps 2-5 each time you want to try out new changes. [Return to Development documentation](index.md) -## Wrapping RPCs in Feature Flags +## Wrapping RPCs in feature flags Here are the steps to gate a new feature in Gitaly behind a feature flag. diff --git a/doc/development/github_importer.md b/doc/development/github_importer.md index 2d9ae768d33..86142efa679 100644 --- a/doc/development/github_importer.md +++ b/doc/development/github_importer.md @@ -70,7 +70,12 @@ don't need to perform this work in parallel. This worker imports all pull requests. For every pull request a job for the `Gitlab::GithubImport::ImportPullRequestWorker` worker is scheduled. -### 5. Stage::ImportPullRequestsMergedByWorker +### 5. Stage::ImportCollaboratorsWorker + +This worker imports all repository collaborators. For every collaborator, we schedule a job +for the `Gitlab::GithubImport::ImportCollaboratorWorker` worker. + +### 6. Stage::ImportPullRequestsMergedByWorker This worker imports the pull requests' _merged-by_ user information. The [_List pull requests_](https://docs.github.com/en/rest/pulls#list-pull-requests) @@ -79,21 +84,21 @@ individually to import this information. A `Gitlab::GithubImport::ImportPullRequestMergedByWorker` job is scheduled for each fetched pull request. -### 6. Stage::ImportPullRequestsReviewRequestsWorker +### 7. Stage::ImportPullRequestsReviewRequestsWorker This worker imports assigned reviewers of pull requests. For each pull request, this worker: - Fetches all assigned review requests. - Schedules a `Gitlab::GithubImport::PullRequests::ImportReviewRequestWorker` job for each fetched review request. -### 7. Stage::ImportPullRequestsReviewsWorker +### 8. Stage::ImportPullRequestsReviewsWorker This worker imports reviews of pull requests. For each pull request, this worker: - Fetches all the pages of reviews. - Schedules a `Gitlab::GithubImport::ImportPullRequestReviewWorker` job for each fetched review. -### 8. Stage::ImportIssuesAndDiffNotesWorker +### 9. Stage::ImportIssuesAndDiffNotesWorker This worker imports all issues and pull request comments. For every issue, we schedule a job for the `Gitlab::GithubImport::ImportIssueWorker` worker. For @@ -109,7 +114,7 @@ label links in the same worker removes the need for performing a separate crawl through the API data, reducing the number of API calls necessary to import a project. -### 9. Stage::ImportIssueEventsWorker +### 10. Stage::ImportIssueEventsWorker This worker imports all issues and pull request events. For every event, we schedule a job for the `Gitlab::GithubImport::ImportIssueEventWorker` worker. @@ -125,7 +130,7 @@ Therefore, both issues and pull requests have a common API for most related thin NOTE: This stage is optional and can consume significant extra import time (controlled by `Gitlab::GithubImport::Settings`). -### 10. Stage::ImportNotesWorker +### 11. Stage::ImportNotesWorker This worker imports regular comments for both issues and pull requests. For every comment, we schedule a job for the @@ -136,7 +141,7 @@ returns comments for both issues and pull requests. This means we have to wait for all issues and pull requests to be imported before we can import regular comments. -### 11. Stage::ImportAttachmentsWorker +### 12. Stage::ImportAttachmentsWorker This worker imports note attachments that are linked inside Markdown. For each entity with Markdown text in the project, we schedule a job of: @@ -155,7 +160,7 @@ Each job: NOTE: It's an optional stage that could consume significant extra import time (controlled by `Gitlab::GithubImport::Settings`). -### 12. Stage::ImportProtectedBranchesWorker +### 13. Stage::ImportProtectedBranchesWorker This worker imports protected branch rules. For every rule that exists on GitHub, we schedule a job of @@ -164,7 +169,7 @@ For every rule that exists on GitHub, we schedule a job of Each job compares the branch protection rules from GitHub and GitLab and applies the strictest of the rules to the branches in GitLab. -### 13. Stage::FinishImportWorker +### 14. Stage::FinishImportWorker This worker completes the import process by performing some housekeeping (such as flushing any caches) and by marking the import as completed. diff --git a/doc/development/gitlab_flavored_markdown/index.md b/doc/development/gitlab_flavored_markdown/index.md index 064a7ecbfa9..cde83bff32e 100644 --- a/doc/development/gitlab_flavored_markdown/index.md +++ b/doc/development/gitlab_flavored_markdown/index.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/gitlab_flavored_markdown/specification_guide/index.md b/doc/development/gitlab_flavored_markdown/specification_guide/index.md index 806ac3837bf..ae78daa3687 100644 --- a/doc/development/gitlab_flavored_markdown/specification_guide/index.md +++ b/doc/development/gitlab_flavored_markdown/specification_guide/index.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -178,7 +178,7 @@ strings in the standard In this context, it should not be confused with other similar or related meanings of _example_, such as -[RSpec examples](https://relishapp.com/rspec/rspec-core/docs/example-groups/basic-structure-describe-it). +[RSpec examples](https://rspec.info/features/3-12/rspec-core/example-groups/basic-structure/). See the section on the [`glfm_official_specification.md`](#glfm_official_specificationmd) file for more details on the backtick-delimited Markdown+HTML example syntax. @@ -297,7 +297,7 @@ The Markdown dialect used in the GitLab application has a dual requirement for r 1. Rendering to static read-only HTML format, to be displayed in various places throughout the application. 1. Rendering editable content in the - [Content Editor](https://about.gitlab.com/direction/create/editor/content_editor/), + [Content Editor](https://about.gitlab.com/direction/plan/knowledge/content_editor/), a ["What You See Is What You Get" (WYSIWYG)](https://en.wikipedia.org/wiki/WYSIWYG) editor. The Content Editor supports real-time instant switching between an editable Markdown source and an editable WYSIWYG document. diff --git a/doc/development/gitlab_shell/index.md b/doc/development/gitlab_shell/index.md index 3a1af0fc9e9..0663341f806 100644 --- a/doc/development/gitlab_shell/index.md +++ b/doc/development/gitlab_shell/index.md @@ -217,6 +217,6 @@ sequenceDiagram ## Related topics -- [LICENSE](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/LICENSE). +- [LICENSE](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/LICENSE) - [PROCESS.md](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/PROCESS.md) - [Using the GitLab Shell chart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/) diff --git a/doc/development/graphql_guide/reviewing.md b/doc/development/graphql_guide/reviewing.md new file mode 100644 index 00000000000..9c32179a89d --- /dev/null +++ b/doc/development/graphql_guide/reviewing.md @@ -0,0 +1,96 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# GraphQL API merge request checklist + +The GitLab GraphQL API has a fair degree of complexity so it's important that merge requests containing GraphQL changes be reviewed by someone familiar with GraphQL. +You can ping one via the `@gitlab-org/graphql-experts` group in a MR or in the [`#f_graphql` channel](https://gitlab.slack.com/archives/C6MLS3XEU) in Slack (available to GitLab team members only). + +GraphQL queries need to be reviewed for: + +- breaking changes +- authorization +- performance + +## Review criteria + +This is not an exhaustive list. + +### Description with sample query + +Ensure that the description includes a sample query with setup instructions. +Try running the query in [GraphiQL](../api_graphql_styleguide.md#graphiql) on your local GDK instance. + +### No breaking changes (unless after full deprecation cycle) + +Check the MR for any [breaking changes](../api_graphql_styleguide.md#breaking-changes). + +If a feature is marked as an [Experiment](../api_graphql_styleguide.md#mark-schema-items-as-alpha), you can make breaking changes immediately, with no deprecation period. + +For more information, see [deprecation and removal process](../../api/graphql/index.md#deprecation-and-removal-process). + +### Multiversion compatibility + +Ensure that multi-version compatibility is guaranteed. +This generally means frontend and backend code for the same GraphQL feature can't be shipped in the same release. + +For details, see [multiple version compatibility](../multi_version_compatibility.md). + +### Technical writing review + +Changes to the generated API docs require a technical writer review. + +### Changelog + +Public-facing changes that are not marked as an [Experiment](../api_graphql_styleguide.md#mark-schema-items-as-alpha) require a [changelog entry](../changelog.md). + +### Use the framework + +GraphQL is a framework with many moving parts. It's important that the framework is followed. + +- Do not manually invoke framework bits. For example, do not instantiate resolvers during execution and instead let the framework do that. +- You can subclass resolvers, as in `MyResolver.single` (see [deriving resolvers](../api_graphql_styleguide.md#deriving-resolvers)). +- Use the `ready?` method for more complex argument logic (see [correct use of resolver#ready](../api_graphql_styleguide.md#correct-use-of-resolverready)). +- Use the `prepare` method for more complex argument validation (see [validating arguments](../api_graphql_styleguide.md#validating-arguments)). + +For details, see [resolver guide](../api_graphql_styleguide.md#writing-resolvers). + +### Authorization + +Ensure proper authorization is followed and that `authorize :some_ability` is tested in the specs. + +For details, see [authorization guide](authorization.md). + +### Performance + +Ensure: + +- You avoid N+1s with [BatchLoader](batchloader.md) or [Lookahead](../api_graphql_styleguide.md#look-ahead) when appropriate. +- You use [laziness](../api_graphql_styleguide.md#laziness) appropriately. + +### Use appropriate types + +For example: + +- [`TimeType`](../api_graphql_styleguide.md#typestimetype) for Ruby `Time` and `DateTime` objects. +- Global IDs for `id` fields + +For details, see [types](../api_graphql_styleguide.md#types). + +### Appropriate complexity + +Query complexity is a way of quantifying how expensive a query is likely to be. Query complexity limits are defined as constants in the schema. +When a resolver or type is expensive to call we need to ensure that the query complexity reflects that. + +For details, see [max complexity](../api_graphql_styleguide.md#max-complexity), [field complexity](../api_graphql_styleguide.md#field-complexity) and [query limits](../api_graphql_styleguide.md#query-limits). + +### Testing + +- Resolver (unit) specs are deprecated in favour of request (integration) specs. +- Many aspects of our framework are outside the `resolve` method and a request spec is the only way to ensure they behave properly. +- Every GraphQL change MR should ideally have changes to API specs. + +For details, see [testing guide](../api_graphql_styleguide.md#testing). diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index 1957662317a..964bee8620e 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -19,6 +19,8 @@ are very appreciative of the work done by translators and proofreaders! - Tsegaselassie Tadesse - [GitLab](https://gitlab.com/tsega), [Crowdin](https://crowdin.com/profile/tsegaselassi) - Arabic - Proofreaders needed. +- Belarusian + - Anton Katsuba - [GitLab](https://gitlab.com/coinvariant), [Crowdin](https://crowdin.com/profile/aerialfiddle) - Bosnian - Haris Delalić - [GitLab](https://gitlab.com/haris.delalic), [Crowdin](https://crowdin.com/profile/haris.delalic) - Bulgarian @@ -126,6 +128,7 @@ are very appreciative of the work done by translators and proofreaders! - Proofreaders needed. - Spanish - Pedro Garcia - [GitLab](https://gitlab.com/pedgarrod), [Crowdin](https://crowdin.com/profile/breaking_pitt) + - David Elizondo - [GitLab](https://gitlab.com/daelmo), [Crowdin](https://crowdin.com/profile/daelmo) - Swedish - Johannes Nilsson - [GitLab](https://gitlab.com/nlssn), [Crowdin](https://crowdin.com/profile/nlssn) - Turkish diff --git a/doc/development/index.md b/doc/development/index.md index a39a83b257a..b8e67748bc9 100644 --- a/doc/development/index.md +++ b/doc/development/index.md @@ -1,5 +1,4 @@ --- -comments: false type: index, dev stage: none group: Development @@ -14,8 +13,8 @@ Learn how to contribute to the development of the GitLab product. This content is intended for GitLab team members as well as members of the wider community. - [Contribute to GitLab development](contributing/index.md) -- [Contribute to Omnibus development](https://docs.gitlab.com/omnibus/development/) -- [Contribute to GitLab Pages development](pages/index.md) - [Contribute to GitLab Runner development](https://docs.gitlab.com/runner/development/) -- [Contribute to Helm chart development](https://docs.gitlab.com/charts/development/) +- [Contribute to GitLab Pages development](pages/index.md) +- [Contribute to GitLab distribution development](distribution/index.md) +- [Contribute to the GitLab Design System](https://design.gitlab.com/get-started/contributing) - [Contribute to the GitLab documentation](documentation/index.md) diff --git a/doc/development/integrations/index.md b/doc/development/integrations/index.md index 6f42cb4d4fe..efe70e5948a 100644 --- a/doc/development/integrations/index.md +++ b/doc/development/integrations/index.md @@ -23,9 +23,11 @@ if you need clarification or spot any outdated information. - For example, `Integrations::FooBar` in `app/models/integrations/foo_bar.rb`. - For certain types of integrations, you can also build on these base classes: - `Integrations::BaseChatNotification` + - `Integrations::BaseCi` - `Integrations::BaseIssueTracker` - `Integrations::BaseMonitoring` - `Integrations::BaseSlashCommands` + - `Integrations::BaseThirdPartyWiki` - For integrations that primarily trigger HTTP calls to external services, you can also use the `Integrations::HasWebHook` concern. This reuses the [webhook functionality](../../user/project/integrations/webhooks.md) in GitLab through an associated `ServiceHook` model, and automatically records request logs @@ -37,9 +39,6 @@ if you need clarification or spot any outdated information. has_one :foo_bar_integration, class_name: 'Integrations::FooBar' ``` -1. TEMPORARY: Accommodate the current migration to [rename "services" to "integrations"](#rename-services-to-integrations): - - Add the integration's camel-cased name (`'FooBar'`) to `Gitlab::Integrations::StiType::NAMESPACED_INTEGRATIONS`. - ### Define properties Integrations can define arbitrary properties to store their configuration with the class method `Integration.prop_accessor`. @@ -237,9 +236,7 @@ module Integrations end ``` -### Expose the integration in the API - -#### REST API +### Expose the integration in the REST API To expose the integration in the [REST API](../../api/integrations.md): @@ -258,46 +255,6 @@ Sensitive fields are not exposed over the API. Sensitive fields are those fields - `token` - `webhook` -#### GraphQL API - -Integrations use the `Types::Projects::ServiceType` type by default, -which only exposes the `type` and `active` properties. - -To expose additional properties, you can write a class implementing `ServiceType`: - -```ruby -# in app/graphql/types/project/services/foo_bar_service_type.rb -module Types - module Projects - module Services - class FooBarServiceType < BaseObject - graphql_name 'FooBarService' - implements(Types::Projects::ServiceType) - authorize :read_project - - field :frobinity, - GraphQL::Types::Float, - null: true, - description: 'The level of frobinity.' - - field :foo_label, - GraphQL::Types::String, - null: true, - description: 'The foo label to apply.' - end - end - end -end -``` - -Each property you want to expose should have a field defined for it. You can also expose any public instance method of the integration. - -Contact a member of the Integrations team to discuss the best authorization. - -Reference documentation for GraphQL is automatically generated. - -You can also refer to our [GraphQL API style guide](../api_graphql_styleguide.md). - ## Availability of integrations By default, integrations are available on the project, group, and instance level. @@ -345,26 +302,8 @@ The strings should use the integration name as [namespace](../i18n/externalizati ## Ongoing migrations and refactorings -The Integrations team is in the process of some larger migrations that developers should be aware of. - -### [Rename "services" to "integrations"](https://gitlab.com/groups/gitlab-org/-/epics/2504) - -The "integrations" in GitLab were historically called "services", which frequently caused -confusion with our "service" classes in `app/services`. We sometimes also called -them "project services" because they were initially only available on projects, which is -not the case anymore. - -We decided to change the naming from "services" and "project services" to "integrations". -This refactoring is an ongoing effort, and there are still references to the old names in some places. - -Developers should be especially aware that we still use the old class names for the STI column -`integrations.type`. For example, a class `Integrations::FooBar` still stores -the old name `FooBarService` in the database. This mapping is handled via `Gitlab::Integrations::StiType` -and should be mostly transparent to the rest of the app. - -### [Consolidate integration settings](https://gitlab.com/groups/gitlab-org/-/epics/3955) - -We want to unify the way integration properties are defined. +Developers should be aware that the Integrations team is in the process of +[unifying the way integration properties are defined](https://gitlab.com/groups/gitlab-org/-/epics/3955). ## Integration examples diff --git a/doc/development/integrations/jira_connect.md b/doc/development/integrations/jira_connect.md index 4789280c09d..3d5e6aae25a 100644 --- a/doc/development/integrations/jira_connect.md +++ b/doc/development/integrations/jira_connect.md @@ -72,22 +72,6 @@ To avoid external dependencies like Gitpod and a Jira Cloud instance, use the [J 1. Go to `http://localhost:9292`. 1. Paste the token and select **Install GitLab.com Jira Cloud app**. -### Troubleshooting - -If the app install failed, you might need to delete `jira_connect_installations` from your database. - -1. Open the [database console](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/postgresql.md#access-postgresql). -1. Run `TRUNCATE TABLE jira_connect_installations CASCADE;`. - -#### Not authorized to access the file - -If you use Gitpod and you get an error about Jira not being able to access the descriptor file, you might need to make the GDK port public by following these steps: - -1. Open your GitLab workspace in Gitpod. -1. When the GDK is running, select **Ports** in the bottom-right corner. -1. On the left sidebar, select the port the GDK is listening to (typically `3000`). -1. If the port is marked as private, select the lock icon to make it public. - ## Test the GitLab OAuth authentication flow > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81126) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `jira_connect_oauth`. Disabled by default. @@ -123,3 +107,21 @@ The following steps describe setting up an environment to test the GitLab OAuth 1. Scroll down and expand the GitLab for Jira App section. 1. Go to [gitpod.io/variables](https://gitpod.io/variables). 1. Paste the Application ID into the **Jira Connect Application ID** field and select **Save changes**. + +## Troubleshooting + +### App installation fails + +If the app installation fails, you might need to delete `jira_connect_installations` from your database. + +1. Open the [database console](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/postgresql.md#access-postgresql). +1. Run `TRUNCATE TABLE jira_connect_installations CASCADE;`. + +### Not authorized to access the file + +If you use Gitpod and you get an error about Jira not being able to access the descriptor file, you might need to make the GDK port public by following these steps: + +1. Open your GitLab workspace in Gitpod. +1. When the GDK is running, select **Ports** in the bottom-right corner. +1. On the left sidebar, select the port the GDK is listening to (typically `3000`). +1. If the port is marked as private, select the lock icon to make it public. diff --git a/doc/development/integrations/secure_partner_integration.md b/doc/development/integrations/secure_partner_integration.md index 853541144fb..ab94cad3bdc 100644 --- a/doc/development/integrations/secure_partner_integration.md +++ b/doc/development/integrations/secure_partner_integration.md @@ -42,7 +42,7 @@ best place to integrate your own product and its results into GitLab. - Pipeline jobs serve a variety of purposes. Jobs can do scanning for and have implications for app security, corporate policy, or compliance. When complete, the job reports back on its status and creates a - [job artifact](../../ci/pipelines/job_artifacts.md) as a result. + [job artifact](../../ci/jobs/job_artifacts.md) as a result. - The [Merge Request Security Widget](../../ci/testing/index.md#security-reports) displays the results of the pipeline's security checks and the developer can review them. The developer can review both a summary and a detailed version @@ -84,7 +84,7 @@ and complete an integration with the Secure stage. to successfully display your own product's results with the rest of GitLab. - See detailed [technical directions](secure.md) for this step. - Read more about [job report artifacts](../../ci/yaml/index.md#artifactsreports). - - Read about [job artifacts](../../ci/pipelines/job_artifacts.md). + - Read about [job artifacts](../../ci/jobs/job_artifacts.md). - Your report artifact must be in one of our currently supported formats. For more information, see the [documentation on reports](secure.md#report). - Documentation for [SAST reports](../../user/application_security/sast/index.md#reports-json-format). @@ -95,7 +95,7 @@ and complete an integration with the Secure stage. and add the label `devops::secure`. - Once the job is completed, the data can be seen: - In the [Merge Request Security Report](../../ci/testing/index.md#security-reports) ([MR Security Report data flow](https://gitlab.com/snippets/1910005#merge-request-view)). - - While [browsing a Job Artifact](../../ci/pipelines/job_artifacts.md). + - While [browsing a Job Artifact](../../ci/jobs/job_artifacts.md). - In the [Security Dashboard](../../user/application_security/security_dashboard/index.md) ([Dashboard data flow](https://gitlab.com/snippets/1910005#project-and-group-dashboards)). 1. Optional: Provide a way to interact with results as Vulnerabilities: - Users can interact with the findings from your artifact within their workflow. They can dismiss the findings or accept them and create a backlog issue. diff --git a/doc/development/interacting_components.md b/doc/development/interacting_components.md index f4becd06245..7fb711795c1 100644 --- a/doc/development/interacting_components.md +++ b/doc/development/interacting_components.md @@ -29,5 +29,5 @@ See also [File Storage in GitLab](file_storage.md). ### Forks GitLab supports a great amount of features for [merge requests](../user/project/merge_requests/index.md). One -of them is the ability to create merge requests from and to [forks](../user/project/repository/forking_workflow.md#creating-a-fork), +of them is the ability to create merge requests from and to [forks](../user/project/repository/forking_workflow.md#create-a-fork), which should also be highly considered and tested upon development phase. diff --git a/doc/development/internal_api/index.md b/doc/development/internal_api/index.md index aa10bbeda9c..c1c0177609b 100644 --- a/doc/development/internal_api/index.md +++ b/doc/development/internal_api/index.md @@ -809,6 +809,107 @@ Example response: - CustomersDot +## Storage limit exclusions + +The namespace storage limit exclusion endpoints manage storage limit exclusions on top-level namespaces on GitLab.com. +These endpoints can only be consumed in the Admin Area of GitLab.com. + +### Retrieve storage limit exclusions + +Use a GET request to retrieve all `Namespaces::Storage::LimitExclusion` records. + +```plaintext +GET /namespaces/storage/limit_exclusions +``` + +Example request: + +```shell +curl --request GET \ + --url "https://gitlab.com/v4/namespaces/storage/limit_exclusions" \ + --header 'PRIVATE-TOKEN: <admin access token>' +``` + +Example response: + +```json +[ + { + "id": 1, + "namespace_id": 1234, + "namespace_name": "A Namespace Name", + "reason": "a reason to exclude the Namespace" + }, + { + "id": 2, + "namespace_id": 4321, + "namespace_name": "Another Namespace Name", + "reason": "another reason to exclude the Namespace" + }, +] +``` + +### Create a storage limit exclusion + +Use a POST request to create an `Namespaces::Storage::LimitExclusion`. + +```plaintext +POST /namespaces/:id/storage/limit_exclusion +``` + +| Attribute | Type | Required | Description | +|:------------|:--------|:---------|:------------| +| `reason` | string | yes | The reason to exclude the namespace. | + +Example request: + +```shell +curl --request POST \ + --url "https://gitlab.com/v4/namespaces/123/storage/limit_exclusion" \ + --header 'Content-Type: application/json' \ + --header 'PRIVATE-TOKEN: <admin access token>' \ + --data '{ + "reason": "a reason to exclude the Namespace" + }' +``` + +Example response: + +```json +{ + "id": 1, + "namespace_id": 1234, + "namespace_name": "A Namespace Name", + "reason": "a reason to exclude the Namespace" +} +``` + +### Delete a storage limit exclusion + +Use a DELETE request to delete a `Namespaces::Storage::LimitExclusion` for a namespace. + +```plaintext +DELETE /namespaces/:id/storage/limit_exclusion +``` + +Example request: + +```shell +curl --request DELETE \ + --url "https://gitlab.com/v4/namespaces/123/storage/limit_exclusion" \ + --header 'PRIVATE-TOKEN: <admin access token>' +``` + +Example response: + +```plaintext +204 +``` + +### Known consumers + +- GitLab.com Admin Area + ## CI/CD minutes provisioning The CI/CD Minutes endpoints are used by [CustomersDot](https://gitlab.com/gitlab-org/customers-gitlab-com) (`customers.gitlab.com`) diff --git a/doc/development/internal_users.md b/doc/development/internal_users.md index 67000d969af..541a7117c6a 100644 --- a/doc/development/internal_users.md +++ b/doc/development/internal_users.md @@ -44,5 +44,8 @@ Other examples of internal users: - [Ghost User](../user/profile/account/delete_account.md#associated-records) - [Support Bot](../user/project/service_desk.md#support-bot-user) - Visual Review Bot -- Resource access tokens (including [project access tokens](../user/project/settings/project_access_tokens.md)). - These are implemented as `project_bot` users with a `PersonalAccessToken`. +- Resource access tokens, including: + - [Project access tokens](../user/project/settings/project_access_tokens.md). + - [Group access tokens](../user/group/settings/group_access_tokens.md). + + These are implemented as `project_{project_id}_bot_{random_string}` i.e. `group_{group_id}_bot_{random_string}` users, with a `PersonalAccessToken`. diff --git a/doc/development/kubernetes.md b/doc/development/kubernetes.md index 99737e71b22..e537b4c4c76 100644 --- a/doc/development/kubernetes.md +++ b/doc/development/kubernetes.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/lfs.md b/doc/development/lfs.md index 8f6d54e08e3..bb327bf5d04 100644 --- a/doc/development/lfs.md +++ b/doc/development/lfs.md @@ -3,12 +3,144 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- - # Git LFS development guidelines This page contains developer-centric information for GitLab team members. For the user documentation, see [Git Large File Storage](../topics/git/lfs/index.md). +## Controllers and Services + +### Repositories::GitHttpClientController + +The methods for authentication defined here are inherited by all the other LFS controllers. + +### Repositories::LfsApiController + +#### `#batch` + +After authentication the `batch` action is the first action called by the Git LFS +client during downloads and uploads (such as pull, push, and clone). + +### Repositories::LfsStorageController + +#### `#upload_authorize` + +Provides payload to Workhorse including a path for Workhorse to save the file to. Could be remote object storage. + +#### `#upload_finalize` + +Handles requests from Workhorse that contain information on a file that workhorse already uploaded (see [this middleware](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/middleware/multipart.rb)) so that `gitlab` can either: + +- Create an `LfsObject`. +- Connect an existing `LfsObject` to a project with an `LfsObjectsProject`. + +### LfsObject and LfsObjectsProject + +- Only one `LfsObject` is created for a file with a given `oid` (a SHA256 checksum of the file) and file size. +- `LfsObjectsProject` associate `LfsObject`s with `Project`s. They determine if a file can be accessed through a project. +- These objects are also used for calculating the amount of LFS storage a given project is using. + For more information, see + [`ProjectStatistics#update_lfs_objects_size`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/project_statistics.rb#L82-84). + +### Repositories::LfsLocksApiController + +Handles the lock API for LFS. Delegates mostly to corresponding services: + +- `Lfs::LockFileService` +- `Lfs::UnlockFileService` +- `Lfs::LocksFinderService` + +These services create and delete `LfsFileLock`. + +#### `#verify` + +- This endpoint responds with a payload that allows a client to check if there are any files being pushed that have locks that belong to another user. +- A client-side `lfs.locksverify` configuration can be set so that the client aborts the push if locks exist that belong to another user. +- The existence of locks belonging to other users is also [validated on the server side](https://gitlab.com/gitlab-org/gitlab/-/blob/65f0c6e59121b62c9b0f89b810ef5186969bb4d2/lib/gitlab/checks/diff_check.rb#L69). + +## Example authentication + +```mermaid +sequenceDiagram +autonumber + alt Over HTTPS + Git client-->>Git client: user-supplied credentials + else Over SSH + Git client->>gitlab-shell: git-lfs-authenticate + activate gitlab-shell + activate GitLab Rails + gitlab-shell->>GitLab Rails: POST /api/v4/internal/lfs_authenticate + GitLab Rails-->>gitlab-shell: token with expiry + deactivate gitlab-shell + deactivate GitLab Rails + end +``` + +1. Clients can be configured to store credentials in a few different ways. + See the [Git LFS documentation on authentication](https://github.com/git-lfs/git-lfs/blob/bea0287cdd3acbc0aa9cdf67ae09b6843d3ffcf0/docs/api/authentication.md#git-credentials). +1. Running `gitlab-lfs-authenticate` on `gitlab-shell`. See the [Git LFS documentation concerning `gitlab-lfs-authenticate`](https://github.com/git-lfs/git-lfs/blob/bea0287cdd3acbc0aa9cdf67ae09b6843d3ffcf0/docs/api/server-discovery.md#ssh). +1. `gitlab-shell`makes a request to the GitLab API. +1. [Responding to shell with token](https://gitlab.com/gitlab-org/gitlab/-/blob/7a2f7a31a88b6085ea89b8ba188a4d92d5fada91/lib/api/internal/base.rb#L168) which is used in subsequent requests. See [Git LFS documentation concerning authentication](https://github.com/git-lfs/git-lfs/blob/bea0287cdd3acbc0aa9cdf67ae09b6843d3ffcf0/docs/api/authentication.md). + +## Example clone + +```mermaid +sequenceDiagram + Note right of Git client: Typical Git clone things happen first + Note right of Git client: Authentication for LFS comes next + activate GitLab Rails + autonumber + Git client->>GitLab Rails: POST project/namespace/info/lfs/objects/batch + GitLab Rails-->>Git client: payload with objects + deactivate GitLab Rails + loop each object in payload + Git client->>GitLab Rails: GET project/namespace/gitlab-lfs/objects/:oid/ (<- This URL is from the payload) + GitLab Rails->>Workhorse: SendfileUpload + Workhorse-->> Git client: Binary data + end +``` + +1. Git LFS requests the ability to download files with authorization header from authorization. +1. `gitlab` responds with the list of objects and where to find them. See + [LfsApiController#batch](https://gitlab.com/gitlab-org/gitlab/-/blob/7a2f7a31a88b6085ea89b8ba188a4d92d5fada91/app/controllers/repositories/lfs_api_controller.rb#L25). +1. Git LFS makes a request for each file for the `href` in the previous response. See + [how downloads are handled with the basic transfer mode](https://github.com/git-lfs/git-lfs/blob/bea0287cdd3acbc0aa9cdf67ae09b6843d3ffcf0/docs/api/basic-transfers.md#downloads). +1. `gitlab` redirects to the remote URL if remote object storage is enabled. See + [SendFileUpload](https://gitlab.com/gitlab-org/gitlab/-/blob/7a2f7a31a88b6085ea89b8ba188a4d92d5fada91/app/controllers/concerns/send_file_upload.rb#L4). + +## Example push + +```mermaid +sequenceDiagram + Note right of Git client: Typical Git push things happen first. + Note right of Git client: Suthentication for LFS comes next. + autonumber + activate GitLab Rails + Git client ->> GitLab Rails: POST project/namespace/info/lfs/objects/batch + GitLab Rails-->>Git client: payload with objects + deactivate GitLab Rails + loop each object in payload + Git client->>Workhorse: PUT project/namespace/gitlab-lfs/objects/:oid/:size (URL is from payload) + Workhorse->>GitLab Rails: PUT project/namespace/gitlab-lfs/objects/:oid/:size/authorize + GitLab Rails-->>Workhorse: response with where path to upload + Workhorse->>Workhorse: Upload + Workhorse->>GitLab Rails: PUT project/namespace/gitlab-lfs/objects/:oid/:size/finalize + end +``` + +1. Git LFS requests the ability to upload files. +1. `gitlab` responds with the list of objects and uploads to find them. See + [LfsApiController#batch](https://gitlab.com/gitlab-org/gitlab/-/blob/7a2f7a31a88b6085ea89b8ba188a4d92d5fada91/app/controllers/repositories/lfs_api_controller.rb#L27). +1. Git LFS makes a request for each file for the `href` in the previous response. See + [how uploads are handled with the basic transfer mode](https://github.com/git-lfs/git-lfs/blob/bea0287cdd3acbc0aa9cdf67ae09b6843d3ffcf0/docs/api/basic-transfers.md#uploads). +1. `gitlab` responds with a payload including a path for Workhorse to save the file to. + Could be remote object storage. See + [LfsStorageController#upload_authorize](https://gitlab.com/gitlab-org/gitlab/-/blob/96250de93a410e278ef659a3d38b056f12024636/app/controllers/repositories/lfs_storage_controller.rb#L42). +1. Workhorse does the work of saving the file. +1. Workhorse makes a request to `gitlab` with information on the uploaded file so + that `gitlab` can create an `LfsObject`. See + [LfsStorageController#upload_finalize](https://gitlab.com/gitlab-org/gitlab/-/blob/96250de93a410e278ef659a3d38b056f12024636/app/controllers/repositories/lfs_storage_controller.rb#L51). + ## Deep Dive In April 2019, Francisco Javier López hosted a Deep Dive (GitLab team members only: `https://gitlab.com/gitlab-org/create-stage/-/issues/1`) diff --git a/doc/development/logging.md b/doc/development/logging.md index 6840445a0a5..084bad70b90 100644 --- a/doc/development/logging.md +++ b/doc/development/logging.md @@ -202,7 +202,7 @@ suffix and `duration` within its name (for example, `view_duration_s`). ## Multi-destination Logging -GitLab is transitioning from unstructured/plaintext logs to structured/JSON logs. During this transition period some logs are recorded in multiple formats through multi-destination logging. +GitLab transitioned from structured to JSON logs. However, through multi-destination logging, the logs can be recorded in multiple formats. ### How to use multi-destination logging diff --git a/doc/development/merge_request_concepts/diffs/frontend.md b/doc/development/merge_request_concepts/diffs/frontend.md index 6bd6d80af94..ff163050e1f 100644 --- a/doc/development/merge_request_concepts/diffs/frontend.md +++ b/doc/development/merge_request_concepts/diffs/frontend.md @@ -84,40 +84,9 @@ The most important part of the metadata response is the diff file names. This da app to render the file browser inside of the diffs app, without waiting for all batch diffs requests to complete. -When the metadata response is received, the diff file data gets sent to a web worker. The web worker -exists to allow for this data, which for larger merge requests could be huge, to be processed off -the main thread. Processing this data involves getting the data into the correct structure +When the metadata response is received, the diff file data is processed into the correct structure that the frontend requires to render the file browser in either tree view or list view. -```mermaid -graph TD - A[fetchDiffFilesMeta] - B[Create web worker] - C[Fetch data] - D[SET_DIFF_METADATA] - E[Post worker data] - F[Worker message event listener] - K[SET_TREE_DATA] - - G[TreeWorker] - H[onMessage] - I[generateTreeList] - J[postMessage sortTree] - - A -->B - E -->F - B -->C - C -->D - D -->E - - G -->H - H -->I - I -->J - J -->F - - F -->K -``` - The structure for this file object is: ```javascript @@ -128,6 +97,11 @@ The structure for this file object is: "type": "", "tree": [], "changed": true, + "diffLoaded": false, + "filePaths": { + "old": file.old_path, + "new": file.new_path + }, "tempFile": false, "deleted": false, "fileHash": "", diff --git a/doc/development/merge_request_concepts/index.md b/doc/development/merge_request_concepts/index.md index 8e9b586d5b0..8bec5e0c0b0 100644 --- a/doc/development/merge_request_concepts/index.md +++ b/doc/development/merge_request_concepts/index.md @@ -61,7 +61,7 @@ Additionally, approval settings provide configuration options to define how thos Examples of approval rules and settings include: - [Merge request approval rules](../../user/project/merge_requests/approvals/rules.md) -- [Code owner approvals](../../user/project/code_owners.md) +- [Code owner approvals](../../user/project/codeowners/index.md) - [Security approvals](../../user/application_security/index.md#security-approvals-in-merge-requests) - [Prevent editing approval rules](../../user/project/merge_requests/approvals/settings.md#prevent-editing-approval-rules-in-merge-requests) - [Remove all approvals when commits are added](../../user/project/merge_requests/approvals/settings.md#remove-all-approvals-when-commits-are-added-to-the-source-branch) diff --git a/doc/development/merge_request_concepts/performance.md b/doc/development/merge_request_concepts/performance.md index 3b2a097ea2d..174307fc6a7 100644 --- a/doc/development/merge_request_concepts/performance.md +++ b/doc/development/merge_request_concepts/performance.md @@ -158,7 +158,7 @@ query. This in turn makes it much harder for this code to overload a database. ## Use read replicas when possible -In a DB cluster we have many read replicas and one primary. A classic use of scaling the DB is to have read-only actions be performed by the replicas. We use [load balancing](../../administration/postgresql/database_load_balancing.md) to distribute this load. This allows for the replicas to grow as the pressure on the DB grows. +In a DB cluster we have many read replicas and one primary. A classic use of scaling the DB is to have read-only actions be performed by the replicas. We use [load balancing](../database/load_balancing.md) to distribute this load. This allows for the replicas to grow as the pressure on the DB grows. By default, queries use read-only replicas, but due to [primary sticking](../../administration/postgresql/database_load_balancing.md#primary-sticking), GitLab uses the diff --git a/doc/development/merge_request_diffs.md b/doc/development/merge_request_diffs.md deleted file mode 100644 index 9ec7e6cae8b..00000000000 --- a/doc/development/merge_request_diffs.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'merge_request_concepts/diffs/development.md' -remove_date: '2023-04-10' ---- - -This document was moved to [another location](merge_request_concepts/diffs/development.md). - -<!-- This redirect file can be deleted after <2023-04-10>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md deleted file mode 100644 index 1af81a8af9f..00000000000 --- a/doc/development/merge_request_performance_guidelines.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'merge_request_concepts/performance.md' -remove_date: '2023-04-10' ---- - -This document was moved to [another location](merge_request_concepts/performance.md). - -<!-- This redirect file can be deleted after <2023-04-10>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 9f2ad1f7769..47580f91af6 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -156,7 +156,7 @@ regenerate a clean `db/structure.sql` for the migrations you're adding. This script applies all migrations found in `db/migrate` or `db/post_migrate`, so if there are any migrations you don't want to commit to the schema, rename or remove them. If your branch is not -targeting `main` you can set the `TARGET` environment variable. +targeting the default Git branch, you can set the `TARGET` environment variable. ```shell # Regenerate schema against `main` @@ -644,89 +644,14 @@ for more details. ## Adding indexes -Before adding an index, consider if this one is necessary. There are situations in which an index -might not be required, like: - -- The table is small (less than `1,000` records) and it's not expected to exponentially grow in size. -- Any existing indexes filter out enough rows. -- The reduction in query timings after the index is added is not significant. - -Additionally, wide indexes are not required to match all filter criteria of queries, we just need -to cover enough columns so that the index lookup has a small enough selectivity. Please review our -[Adding Database indexes](database/adding_database_indexes.md) guide for more details. - -When adding an index to a non-empty table make sure to use the method -`add_concurrent_index` instead of the regular `add_index` method. -The `add_concurrent_index` method automatically creates concurrent indexes -when using PostgreSQL, removing the need for downtime. - -To use this method, you must disable single-transactions mode -by calling the method `disable_ddl_transaction!` in the body of your migration -class like so: - -```ruby -class MyMigration < Gitlab::Database::Migration[2.1] - disable_ddl_transaction! - - INDEX_NAME = 'index_name' - - def up - add_concurrent_index :table, :column, name: INDEX_NAME - end - - def down - remove_concurrent_index :table, :column, name: INDEX_NAME - end -end -``` - -You must explicitly name indexes that are created with more complex -definitions beyond table name, column names, and uniqueness constraint. -Consult the [Adding Database Indexes](database/adding_database_indexes.md#requirements-for-naming-indexes) -guide for more details. - -If you need to add a unique index, please keep in mind there is the possibility -of existing duplicates being present in the database. This means that should -always _first_ add a migration that removes any duplicates, before adding the -unique index. - -For a small table (such as an empty one or one with less than `1,000` records), -it is recommended to use `add_index` in a single-transaction migration, combining it with other -operations that don't require `disable_ddl_transaction!`. +Before adding an index, consider if one is necessary. The [Adding Database indexes](database/adding_database_indexes.md) guide contains more details to help you decide if an index is necessary and provides best practices for adding indexes. ## Testing for existence of indexes -If a migration requires conditional logic based on the absence or -presence of an index, you must test for existence of that index using -its name. This helps avoids problems with how Rails compares index definitions, -which can lead to unexpected results. For more details, review the -[Adding Database Indexes](database/adding_database_indexes.md#why-explicit-names-are-required) -guide. - -The easiest way to test for existence of an index by name is to use the -`index_name_exists?` method, but the `index_exists?` method can also -be used with a name option. For example: - -```ruby -class MyMigration < Gitlab::Database::Migration[2.1] - INDEX_NAME = 'index_name' - - def up - # an index must be conditionally created due to schema inconsistency - unless index_exists?(:table_name, :column_name, name: INDEX_NAME) - add_index :table_name, :column_name, name: INDEX_NAME - end - end - - def down - # no op - end -end -``` +If a migration requires conditional logic based on the absence or presence of an index, you must test for existence of that index using its name. This helps avoids problems with how Rails compares index definitions, which can lead to unexpected results. -Keep in mind that concurrent index helpers like `add_concurrent_index`, -`remove_concurrent_index`, and `remove_concurrent_index_by_name` already -perform existence checks internally. +For more details, review the [Adding Database Indexes](database/adding_database_indexes.md#testing-for-existence-of-indexes) +guide. ## Adding foreign-key constraints diff --git a/doc/development/navigation_sidebar.md b/doc/development/navigation_sidebar.md index 495f30a796c..90b87e9518d 100644 --- a/doc/development/navigation_sidebar.md +++ b/doc/development/navigation_sidebar.md @@ -20,6 +20,10 @@ To enable the new navigation sidebar: - Enable the `super_sidebar_nav` feature flag. - Select your avatar, then turn on the **New navigation** toggle. +## Adding items to the sidebar + +Before adding an item to the sidebar, ensure you follow [this process](https://about.gitlab.com/handbook/product/ux/navigation/#how-to-propose-a-change-that-impacts-navigation). + ## Adding page-specific Vue content Pages can render arbitrary content into the sidebar using the `SidebarPortal` @@ -36,3 +40,16 @@ subclassing `::Sidebars::Panel`. NOTE: Do not use the `SidebarPortalTarget` component. It is internal to the sidebar. + +## Snowplow Tracking + +All clicks on the nav items should be automatically tracked in Snowplow, but may require additional input. +We use `data-tracking` attributes on all the elements in the nav to send the data up to Snowplow. +You can test that they're working by [setting up snowplow on your GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/snowplow_micro.md). + +| Field | Data attribute | Notes | Example | +| -- | -- | -- | -- | +| Category | `data-tracking-category` | The page that the user was on when the item was clicked. | `groups:show` | +| Action | `data-tracking-action` | The action taken. In most cases this is `click_link` or `click_menu_item` | `click_link` | +| Label | `data-tracking-label` | A descriptor for what was clicked on. This is inferred by the ID of the item in most cases, but falls back to `item_without_id`. This is one to look out for. | `group_issue_list` | +| Property | `data-tracking-property` | This describes where in the nav the link was clicked. If it's in the main nav panel, then it needs to describe which panel. | `nav_panel_group` | diff --git a/doc/development/organization/index.md b/doc/development/organization/index.md index 6a643603170..3a23e50caf9 100644 --- a/doc/development/organization/index.md +++ b/doc/development/organization/index.md @@ -1,5 +1,4 @@ --- -comments: false type: index, dev stage: Data Stores group: Tenant Scale @@ -39,87 +38,14 @@ A solution for this problem is to consolidate groups and projects into a single entity, `namespace`. The work on this solution is split into several phases and is tracked in [epic 6473](https://gitlab.com/groups/gitlab-org/-/epics/6473). -### Phase 1 - -- [Phase 1 epic](https://gitlab.com/groups/gitlab-org/-/epics/6697). -- **Goals**: - 1. Ensure every project receives a corresponding record in the `namespaces` - table with `type='Project'`. - 1. For user namespaces, the type changes from `NULL` to `User`. - -We should make sure that projects, and the project namespace, are equivalent: - -- **Create project:** use Rails callbacks to ensure a new project namespace is - created for each project. Project namespace records should contain `created_at` and - `updated_at` attributes equal to the project's `created_at`/`updated_at` attributes. -- **Update project:** use the `after_save` callback in Rails to ensure some - attributes are kept in sync between project and project namespaces. - Read [`project#after_save`](https://gitlab.com/gitlab-org/gitlab/blob/6d26634e864d7b748dda0e283eb2477362263bc3/app/models/project.rb#L101-L101) - for more information. -- **Delete project:** use FKs cascade delete or Rails callbacks to ensure when a `Project` - or its `ProjectNamespace` is removed, its corresponding `ProjectNamespace` or `Project` - is also removed. -- **Transfer project to a different group:** make sure that when a project is transferred, - its corresponding project namespace is transferred to the same group. -- **Transfer group:** make sure when transferring a group that all of its sub-projects, - either direct or through descendant groups, have their corresponding project - namespaces transferred correctly as well. -- **Export or import project** - - **Export project** continues to export only the project, and not its project namespace, - in this phase. The project namespace does not contain any specific information - to export at this point. Eventually we want the project namespace to be exported as well. - - **Import project** creates a new project, so the project namespace is created through - Rails `after_save` callback on the project model. -- **Export or import group:** when importing or exporting a `Group`, projects are not - included in the operation. If that feature is changed to include `Project` when its group is - imported or exported, the logic must include their corresponding project namespaces - in the import or export. - -After ensuring these points, run a database migration to create a `ProjectNamespace` -record for every `Project`. Project namespace records created during the migration -should have `created_at` and `updated_at` attributes set to the migration runtime. -The project namespaces' `created_at` and `updated_at` attributes would not match -their corresponding project's `created_at` and `updated_at` attributes. We want -the different dates to help audit any of the created project namespaces, in case we need it. -After this work completes, we must migrate data as described in -[Backfill `ProjectNamespace` for every Project](https://gitlab.com/gitlab-org/gitlab/-/issues/337100). - -### Phase 2 - -- [Phase 2 epic](https://gitlab.com/groups/gitlab-org/-/epics/6768). -- **Goal**: Link `ProjectNamespace` to other entities on the database level. - -In this phase: - -- Communicate the changes company-wide at the engineering level. We want to make - engineers aware of the upcoming changes, even though teams are not expected to - collaborate actively until phase 3. -- Raise awareness to avoid regressions, and conflicting or duplicate work that - can be dealt with before phase 3. - -### Phase 3 - -- [Phase 3 epic](https://gitlab.com/groups/gitlab-org/-/epics/6585). -- **Goal**: Achieve feature parity between the namespace types. -Problems to solve as part of this phase: - -- Routes handling through `ProjectNamespace` rather than `Project`. -- Memberships handling. -- Policies handling. -- Import and export. -- Other interactions between project namespace and project models. - -Phase 3 is when the active migration of features from `Project` to `ProjectNamespace`, -or directly to `Namespace`, happens. - -### How to plan features that interact with Group and ProjectNamespace +## How to plan features that interact with Group and ProjectNamespace As of now, every Project in the system has a record in the `namespaces` table. This makes it possible to use common interface to create features that are shared between Groups and Projects. Shared behavior can be added using a concerns mechanism. Because the `Namespace` model is responsible for `UserNamespace` methods as well, it is discouraged to use the `Namespace` model for shared behavior for Projects and Groups. -#### Resource-based features +### Resource-based features To migrate resource-based features, existing functionality will need to be supported. This can be achieved in two Phases. @@ -141,7 +67,7 @@ To migrate resource-based features, existing functionality will need to be suppo Introducing new functionality is very much dependent on every single team and feature. -#### Settings-related features +### Settings-related features Right now, cascading settings are available for `NamespaceSettings`. By creating `ProjectNamespace`, we can use this framework to make sure that some settings are applicable on the project level as well. @@ -150,7 +76,7 @@ When working on settings, we need to make sure that: - They are not used in `join` queries or modify those queries. - Updating settings is taken into consideration. -- If we want to move from project to project namespace, we follow a similar database process to the one described in [Phase 1](#phase-1). +- If we want to move from project to project namespace, we follow a similar database process to the one described in Phase 1. ## Related topics diff --git a/doc/development/pages/index.md b/doc/development/pages/index.md index e71d7df642c..4769dbf427d 100644 --- a/doc/development/pages/index.md +++ b/doc/development/pages/index.md @@ -1,7 +1,7 @@ --- type: reference, dev -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: "GitLab's development guidelines for GitLab Pages" --- @@ -255,7 +255,7 @@ incidents and downtime. To add a new feature flag to GitLab Pages: 1. Create the feature flag in [`internal/feature/feature.go`](https://gitlab.com/gitlab-org/gitlab-pages/-/blob/master/internal/feature/feature.go), which must be **off** by default. -1. Create an issue to track the feature flag using the `Feature Flag` template. +1. Create an issue to track the feature flag using the `Feature flag` template. 1. Add the `~"feature flag"` label to any merge requests that handle feature flags. For GitLab Pages, the feature flags are controlled by environment variables at a global level. diff --git a/doc/development/pipelines/index.md b/doc/development/pipelines/index.md index 51cdd12a8c1..44fd20df40d 100644 --- a/doc/development/pipelines/index.md +++ b/doc/development/pipelines/index.md @@ -47,22 +47,49 @@ In summary: - RSpec tests are dependent on the backend code. - Jest tests are dependent on both frontend and backend code, the latter through the frontend fixtures. +### Predictive Tests Dashboards + +- <https://app.periscopedata.com/app/gitlab/1116767/Test-Intelligence-Accuracy> +- <https://app.periscopedata.com/app/gitlab/899368/EP---Predictive-testing-analysis> + +### The `detect-tests` CI job + +Most CI/CD pipelines for `gitlab-org/gitlab` will run a [`detect-tests` CI job](https://gitlab.com/gitlab-org/gitlab/-/blob/0c6058def8f182b4a2410db5d08a9550b951b2d8/.gitlab/ci/setup.gitlab-ci.yml#L101-146) in the `prepare` stage to detect which backend/frontend tests should be run based on the files that changed in the given MR. + +The `detect-tests` job will create many files that will contain the backend/frontend tests that should be run. Those files will be read in subsequent jobs in the pipeline, and only those tests will be executed. + ### RSpec predictive jobs #### Determining predictive RSpec test files in a merge request -To identify the RSpec tests that are likely to fail in a merge request, we use the [`test_file_finder` gem](https://gitlab.com/gitlab-org/ci-cd/test_file_finder), with two strategies: +To identify the RSpec tests that are likely to fail in a merge request, we use *static mappings* and *dynamic mappings*. + +##### Static mappings -- dynamic mapping from test coverage tracing (generated via the [`Crystalball` gem](https://github.com/toptal/crystalball)) - ([see where it's used](https://gitlab.com/gitlab-org/gitlab/-/blob/47d507c93779675d73a05002e2ec9c3c467cd698/tooling/bin/find_tests#L15)) -- static mapping maintained in the [`tests.yml` file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/tests.yml) for special cases that cannot - be mapped via coverage tracing ([see where it's used](https://gitlab.com/gitlab-org/gitlab/-/blob/47d507c93779675d73a05002e2ec9c3c467cd698/tooling/bin/find_tests#L12)) +We use the [`test_file_finder` gem](https://gitlab.com/gitlab-org/ci-cd/test_file_finder), with a static mapping maintained in the [`tests.yml` file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/tests.yml) for special cases that cannot + be mapped via coverage tracing ([see where it's used](https://gitlab.com/gitlab-org/gitlab/-/blob/5ab06422826c0d69c615655982a6f969a7f3c6ea/tooling/lib/tooling/find_tests.rb#L17)). The test mappings contain a map of each source files to a list of test files which is dependent of the source file. -In the `detect-tests` job, we use this mapping to identify the predictive tests needed for the current merge request. +##### Dynamic mappings + +First, we use the [`test_file_finder` gem](https://gitlab.com/gitlab-org/ci-cd/test_file_finder), with a dynamic mapping strategy from test coverage tracing (generated via the [`Crystalball` gem](https://github.com/toptal/crystalball)) + ([see where it's used](https://gitlab.com/gitlab-org/gitlab/-/blob/master/tooling/lib/tooling/find_tests.rb#L20)). -Later on in [the `rspec fail-fast` job](#fail-fast-job-in-merge-request-pipelines), we run the predictive tests for the current merge request. +In addition to `test_file_finder`, we have added several advanced mappings to detect even more tests to run: + +- [`FindChanges`](https://gitlab.com/gitlab-org/gitlab/-/blob/28943cbd8b6d7e9a350d00e5ea5bb52123ee14a4/tooling/lib/tooling/find_changes.rb) ([!74003](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74003)) + - Automatically detect Jest tests to run upon backend changes (via frontend fixtures) +- [`PartialToViewsMappings`](https://gitlab.com/gitlab-org/gitlab/-/blob/28943cbd8b6d7e9a350d00e5ea5bb52123ee14a4/tooling/lib/tooling/mappings/partial_to_views_mappings.rb) ([#395016](https://gitlab.com/gitlab-org/gitlab/-/issues/395016)) + - Run view specs when Rails partials included in those views are changed in an MR +- [`JsToSystemSpecsMappings`](https://gitlab.com/gitlab-org/gitlab/-/blob/28943cbd8b6d7e9a350d00e5ea5bb52123ee14a4/tooling/lib/tooling/mappings/js_to_system_specs_mappings.rb) ([#386754](https://gitlab.com/gitlab-org/gitlab/-/issues/386754)) + - Run certain system specs if a JavaScript file was changed in an MR +- [`GraphqlBaseTypeMappings`](https://gitlab.com/gitlab-org/gitlab/-/blob/28943cbd8b6d7e9a350d00e5ea5bb52123ee14a4/tooling/lib/tooling/mappings/graphql_base_type_mappings.rb) ([#386756](https://gitlab.com/gitlab-org/gitlab/-/issues/386756)) + - If a GraphQL type class changed, we should try to identify the other GraphQL types that potentially include this type, and run their specs. +- [`ViewToSystemSpecsMappings`](https://gitlab.com/gitlab-org/gitlab/-/blob/28943cbd8b6d7e9a350d00e5ea5bb52123ee14a4/tooling/lib/tooling/mappings/view_to_system_specs_mappings.rb) ([#395017](https://gitlab.com/gitlab-org/gitlab/-/issues/395017)) + - When a view gets changed, we try to find feature specs that would test that area of the code. +- [`ViewToJsMappings`](https://gitlab.com/gitlab-org/gitlab/-/blob/8d7dfb7c043adf931128088b9ffab3b4a39af6f5/tooling/lib/tooling/mappings/view_to_js_mappings.rb) ([#386719](https://gitlab.com/gitlab-org/gitlab/-/issues/386719)) + - If a JS file is changed, we should try to identify the system specs that are covering this JS component. #### Exceptional cases @@ -74,6 +101,10 @@ In addition, there are a few circumstances where we would always run the full RS - when the merge request is created in a security mirror - when any CI configuration file is changed (for example, `.gitlab-ci.yml` or `.gitlab/ci/**/*`) +#### Have you encountered a problem with backend predictive tests? + +If so, please have a look at [the Engineering Productivity RUNBOOK on predictive tests](https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/predictive-tests.md) for instructions on how to act upon predictive tests issues. Additionally, if you identified any test selection gaps, please let `@gl-quality/eng-prod` know so that we can take the necessary steps to optimize test selections. + ### Jest predictive jobs #### Determining predictive Jest test files in a merge request @@ -95,6 +126,10 @@ In addition, there are a few circumstances where we would always run the full Je The `rules` definitions for full Jest tests are defined at `.frontend:rules:jest` in [`rules.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/42321b18b946c64d2f6f788c38844499a5ae9141/.gitlab/ci/rules.gitlab-ci.yml#L938-955). +#### Have you encountered a problem with frontend predictive tests? + +If so, please have a look at [the Engineering Productivity RUNBOOK on predictive tests](https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/predictive-tests.md) for instructions on how to act upon predictive tests issues. + ### Fork pipelines We run only the predictive RSpec & Jest jobs for fork pipelines, unless the `pipeline:run-all-rspec` @@ -104,8 +139,7 @@ See the [experiment issue](https://gitlab.com/gitlab-org/quality/quality-enginee ## Fail-fast job in merge request pipelines -To provide faster feedback when a merge request breaks existing tests, we are experimenting with a -fail-fast mechanism. +To provide faster feedback when a merge request breaks existing tests, we implemented a fail-fast mechanism. An `rspec fail-fast` job is added in parallel to all other `rspec` jobs in a merge request pipeline. This job runs the tests that are directly related to the changes @@ -150,8 +184,8 @@ This number can be overridden by setting a CI/CD variable named `RSPEC_FAIL_FAST ## Re-run previously failed tests in merge request pipelines -In order to reduce the feedback time after resolving failed tests for a merge request, the `rspec rspec-pg12-rerun-previous-failed-tests` -and `rspec rspec-ee-pg12-rerun-previous-failed-tests` jobs run the failed tests from the previous MR pipeline. +In order to reduce the feedback time after resolving failed tests for a merge request, the `rspec rspec-pg13-rerun-previous-failed-tests` +and `rspec rspec-ee-pg13-rerun-previous-failed-tests` jobs run the failed tests from the previous MR pipeline. This was introduced on August 25th 2021, with <https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69053>. @@ -159,7 +193,7 @@ This was introduced on August 25th 2021, with <https://gitlab.com/gitlab-org/git 1. The `detect-previous-failed-tests` job (`prepare` stage) detects the test files associated with failed RSpec jobs from the previous MR pipeline. -1. The `rspec rspec-pg12-rerun-previous-failed-tests` and `rspec rspec-ee-pg12-rerun-previous-failed-tests` jobs +1. The `rspec rspec-pg13-rerun-previous-failed-tests` and `rspec rspec-ee-pg13-rerun-previous-failed-tests` jobs will run the test files gathered by the `detect-previous-failed-tests` job. ```mermaid @@ -169,13 +203,27 @@ graph LR end subgraph "test stage"; - B["rspec rspec-pg12-rerun-previous-failed-tests"]; - C["rspec rspec-ee-pg12-rerun-previous-failed-tests"]; + B["rspec rspec-pg13-rerun-previous-failed-tests"]; + C["rspec rspec-ee-pg13-rerun-previous-failed-tests"]; end A --"artifact: list of test files"--> B & C ``` +## Merge Trains + +### Why do we need to have a “stable” master branch to enable merge trains? + +If the master branch is unstable (i.e. CI/CD pipelines for the master branch are failing frequently), all of the merge requests pipelines that were added AFTER a faulty merge request pipeline would have to be **cancelled** and **added back to the train**, which would create a lot of delays if the merge train is long. + +### How stable does the master branch have to be for us to enable merge trains? + +We don't have a specific number, but we need to have better numbers for flaky tests failures and infrastructure failures (see the [Master Broken Incidents RCA Dashboard](https://app.periscopedata.com/app/gitlab/1082465/Master-Broken-Incidents-Root-Cause-Analysis)). + +### Could we gradually move to merge trains in our CI/CD configuration? + +There was a proposal from a contributor, but the approach is not without some downsides: [see the original proposal and discussion](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/195#note_1117151994). + ## Faster feedback for some merge requests ### Broken Master Fixes @@ -465,10 +513,9 @@ Exceptions to this general guideline should be motivated and documented. ### Ruby versions testing -We're running Ruby 3.0 for the merge requests and the default branch. However, -we're still running Ruby 2.7 for GitLab.com and there are older versions that -we need to maintain, so we also run our test suite against Ruby 2.7 on a -dedicated 2-hourly scheduled pipelines. +We're running Ruby 3.0 on GitLab.com, as well as for merge requests and the default branch. +However, there are older versions for which we need to support Ruby 2.7, so we also run our +test suite against Ruby 2.7 on a dedicated 2-hourly scheduled pipelines. For merge requests, you can add the `pipeline:run-in-ruby2` label to switch the Ruby version used for running the whole test suite to 2.7. When you do @@ -483,20 +530,22 @@ This should let us: ### PostgreSQL versions testing -Our test suite runs against PG12 as GitLab.com runs on PG12 and -[Omnibus defaults to PG12 for new installs and upgrades](../../administration/package_information/postgresql_versions.md). +Our test suite runs against PG13 as GitLab.com runs on PG13 and +[Omnibus defaults to PG13 for new installs and upgrades](../../administration/package_information/postgresql_versions.md). We do run our test suite against PG13 on nightly scheduled pipelines. +We also run our test suite against PG13 upon specific database library changes in MRs and `main` pipelines (with the `rspec db-library-code pg13` job). + #### Current versions testing -| Where? | PostgreSQL version | Ruby version | -|------------------------------------------------------------------------------------------------|--------------------------|-----------------------| -| Merge requests | 12 (default version) | 3.0 (default version) | -| `master` branch commits | 12 (default version) | 3.0 (default version) | -| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour) | 12 (default version) | 3.0 (default version) | -| `maintenance` scheduled pipelines for the `ruby2` branch (every odd-numbered hour), see below. | 12 (default version) | 2.7 | -| `nightly` scheduled pipelines for the `master` branch | 12 (default version), 13 | 3.0 (default version) | +| Where? | PostgreSQL version | Ruby version | +|------------------------------------------------------------------------------------------------|-------------------------------------------------|-----------------------| +| Merge requests | 13 (default version), 12 for DB library changes | 3.0 (default version) | +| `master` branch commits | 13 (default version), 12 for DB library changes | 3.0 (default version) | +| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour) | 13 (default version), 12 for DB library changes | 3.0 (default version) | +| `maintenance` scheduled pipelines for the `ruby2` branch (every odd-numbered hour), see below. | 13 (default version), 12 for DB library changes | 2.7 | +| `nightly` scheduled pipelines for the `master` branch | 13 (default version), 12, 14 | 3.0 (default version) | There are 2 pipeline schedules used for testing Ruby 2.7. One is triggering a pipeline in `ruby2-sync` branch, which updates the `ruby2` branch with latest @@ -511,15 +560,6 @@ The `gitlab` job in the `ruby2-sync` branch uses a `gitlab-org/gitlab` project token with `write_repository` scope and `Maintainer` role with no expiration. The token is stored in the `RUBY2_SYNC_TOKEN` variable in `gitlab-org/gitlab`. -#### Long-term plan - -We follow the [PostgreSQL versions shipped with Omnibus GitLab](../../administration/package_information/postgresql_versions.md): - -| PostgreSQL version | 14.1 (July 2021) | 14.2 (August 2021) | 14.3 (September 2021) | 14.4 (October 2021) | 14.5 (November 2021) | 14.6 (December 2021) | -| -------------------| ---------------------- | ---------------------- | ---------------------- | ---------------------- | ---------------------- | ---------------------- | -| PG12 | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | -| PG13 | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | - ### Redis versions testing Our test suite runs against Redis 6 as GitLab.com runs on Redis 6 and @@ -541,6 +581,13 @@ By default, all tests run with [multiple databases](../database/multiple_databas We also run tests with a single database in nightly scheduled pipelines, and in merge requests that touch database-related files. +Single database tests run in two modes: + +1. **Single database with one connection**. Where GitLab connects to all the tables using one connection pool. +This runs through all the jobs that end with `-single-db` +1. **Single database with two connections**. Where GitLab connects to `gitlab_main`, `gitlab_ci` database tables +using different database connections. This runs through all the jobs that end with `-single-db-ci-connection`. + If you want to force tests to run with a single database, you can add the `pipeline:run-single-db` label to the merge request. ## Monitoring @@ -561,7 +608,7 @@ In general, pipelines for an MR fall into one of the following types (from short - [Documentation pipeline](#documentation-pipeline): For MRs that touch documentation. - [Backend pipeline](#backend-pipeline): For MRs that touch backend code. -- [Frontend pipeline](#frontend-pipeline): For MRs that touch frontend code. +- [Review app pipeline](#review-app-pipeline): For MRs that touch frontend code. - [End-to-end pipeline](#end-to-end-pipeline): For MRs that touch code in the `qa/` folder. A "pipeline type" is an abstract term that mostly describes the "critical path" (for example, the chain of jobs for which the sum @@ -598,10 +645,10 @@ graph LR graph RL; classDef criticalPath fill:#f66; - 1-3["compile-test-assets (6 minutes)"]; + 1-3["compile-test-assets (5.5 minutes)"]; class 1-3 criticalPath; click 1-3 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914317&udv=0" - 1-6["setup-test-env (4 minutes)"]; + 1-6["setup-test-env (3.6 minutes)"]; click 1-6 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914315&udv=0" 1-14["retrieve-tests-metadata"]; click 1-14 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356697&udv=0" @@ -613,19 +660,19 @@ graph RL; click 2_5-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations" 2_5-1 --> 1-3 & 1-6 & 1-14 & 1-15; - 3_2-1["rspec:coverage (5.35 minutes)"]; + 3_2-1["rspec:coverage (5 minutes)"]; class 3_2-1 criticalPath; click 3_2-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=7248745&udv=0" 3_2-1 -.->|"(don't use needs<br/>because of limitations)"| 2_5-1; - 4_3-1["rspec:undercoverage (3.5 minutes)"]; + 4_3-1["rspec:undercoverage (1.3 minutes)"]; class 4_3-1 criticalPath; click 4_3-1 "https://app.periscopedata.com/app/gitlab/652085/EP---Jobs-Durations?widget=13446492&udv=1005715" 4_3-1 --> 3_2-1; ``` -### Frontend pipeline +### Review app pipeline [Reference pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipelines/431913287). @@ -635,15 +682,15 @@ graph RL; 1-2["build-qa-image (2 minutes)"]; click 1-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914325&udv=0" - 1-5["compile-production-assets (16 minutes)"]; + 1-5["compile-production-assets (12 minutes)"]; class 1-5 criticalPath; click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0" - 2_3-1["build-assets-image (1.3 minutes)"]; + 2_3-1["build-assets-image (1.1 minutes)"]; class 2_3-1 criticalPath; 2_3-1 --> 1-5 - 2_6-1["start-review-app-pipeline (49 minutes)"]; + 2_6-1["start-review-app-pipeline (52 minutes)"]; class 2_6-1 criticalPath; click 2_6-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations" 2_6-1 --> 2_3-1 & 1-2; @@ -659,17 +706,17 @@ graph RL; 1-2["build-qa-image (2 minutes)"]; click 1-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914325&udv=0" - 1-5["compile-production-assets (16 minutes)"]; + 1-5["compile-production-assets (12 minutes)"]; class 1-5 criticalPath; click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0" 1-15["detect-tests"]; click 1-15 "https://app.periscopedata.com/app/gitlab/652085/EP---Jobs-Durations?widget=10113603&udv=1005715" - 2_3-1["build-assets-image (1.3 minutes)"]; + 2_3-1["build-assets-image (1.1 minutes)"]; class 2_3-1 criticalPath; 2_3-1 --> 1-5 - 2_4-1["e2e:package-and-test (102 minutes)"]; + 2_4-1["e2e:package-and-test-ee (103 minutes)"]; class 2_4-1 criticalPath; click 2_4-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914305&udv=0" 2_4-1 --> 1-2 & 2_3-1 & 1-15; diff --git a/doc/development/pipelines/internals.md b/doc/development/pipelines/internals.md index bd96f2f2872..678297eb3e5 100644 --- a/doc/development/pipelines/internals.md +++ b/doc/development/pipelines/internals.md @@ -136,10 +136,10 @@ that are scoped to a single [configuration keyword](../../ci/yaml/index.md#job-k | `.qa-cache` | Allows a job to use a default `cache` definition suitable for QA tasks. | | `.yarn-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that do a `yarn install`. | | `.assets-compile-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that compile assets. | -| `.use-pg12` | Allows a job to use the `postgres` 12 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). | -| `.use-pg12-ee` | Same as `.use-pg12` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). | | `.use-pg13` | Allows a job to use the `postgres` 13 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). | | `.use-pg13-ee` | Same as `.use-pg13` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). | +| `.use-pg14` | Allows a job to use the `postgres` 14 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). | +| `.use-pg14-ee` | Same as `.use-pg14` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). | | `.use-kaniko` | Allows a job to use the `kaniko` tool to build Docker images. | | `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` CI/CD variable. | | `.use-docker-in-docker` | Allows a job to use Docker in Docker. | @@ -187,7 +187,6 @@ and included in `rules` definitions via [YAML anchors](../../ci/yaml/yaml_optimi | `if-dot-com-gitlab-org-and-security-merge-request` | Limit jobs creation to merge requests for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | | | `if-dot-com-gitlab-org-and-security-tag` | Limit jobs creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | | | `if-dot-com-ee-schedule` | Limits jobs to scheduled pipelines for the `gitlab-org/gitlab` project on GitLab.com. | | -| `if-security-pipeline-merge-result` | Matches if the pipeline is for a security merge request triggered by `@gitlab-release-tools-bot`. | | <!-- vale gitlab.Substitutions = YES --> diff --git a/doc/development/product_qualified_lead_guide/index.md b/doc/development/product_qualified_lead_guide/index.md index 41237dcdff4..fb8ec478840 100644 --- a/doc/development/product_qualified_lead_guide/index.md +++ b/doc/development/product_qualified_lead_guide/index.md @@ -117,7 +117,7 @@ expect(wrapper.findComponent(HandRaiseLeadButton).exists()).toBe(true); The flow of a PQL lead is as follows: 1. A user triggers a [`HandRaiseLeadButton` component](#embed-a-hand-raise-lead-form) on `gitlab.com`. -1. The `HandRaiseLeadButton` submits any information to the following API endpoint: `/-/trials/create_hand_raise_lead`. +1. The `HandRaiseLeadButton` submits any information to the following API endpoint: `/-/subscriptions/hand_raise_leads`. 1. That endpoint reposts the form to the CustomersDot `trials/create_hand_raise_lead` endpoint. 1. CustomersDot records the form data to the `leads` table and posts the form to [Workato](https://about.gitlab.com/handbook/marketing/marketing-operations/workato/). 1. Workato sends the form to Marketo. @@ -170,7 +170,7 @@ sequenceDiagram ```mermaid sequenceDiagram HandRaiseForm Vue Component->>TrialsController#create_hand_raise_lead: GitLab.com frontend sends [lead] to backend - TrialsController#create_hand_raise_lead->>CreateHandRaiseLeadService: [lead] + Subscriptions::HandRaiseLeadsController#create->>CreateHandRaiseLeadService: [lead] CreateHandRaiseLeadService->>SubscriptionPortalClient: [lead] SubscriptionPortalClient->>CustomersDot|TrialsController#create_hand_raise_lead: GitLab.com sends [lead] to CustomersDot ``` diff --git a/doc/development/project_templates.md b/doc/development/project_templates.md index 31537b21527..cc53ef77c62 100644 --- a/doc/development/project_templates.md +++ b/doc/development/project_templates.md @@ -8,110 +8,63 @@ info: "To determine the technical writer assigned to the Stage/Group associated ## Adding a new built-in project template -This page provides instructions about how to contribute a -[built-in project template](../user/project/index.md#create-a-project-from-a-built-in-template). - -To contribute a built-in project template, you must complete the following tasks: - -1. [Create a project template for GitLab review](#create-a-project-template-for-review) -1. [Add the template SVG icon to GitLab SVGs](#add-the-template-svg-icon-to-gitlab-svgs) -1. [Create a merge request with vendor details](#create-a-merge-request-with-vendor-details) - -You can contribute the following types of project templates: - -- Enterprise: For users with GitLab Premium and above. -- Non-enterprise: For users with GitLab Free and above. - -### Prerequisites - -To add or update an existing template, you must have the following tools -installed: - -- `wget` -- `tar` - -### Create a project template for review - -1. In your selected namespace, create a public project. -1. Add the project content you want to use in the template. Do not include unnecessary assets or dependencies. For an example, -[see this project](https://gitlab.com/gitlab-org/project-templates/dotnetcore). -1. When the project is ready for review, [create an issue](https://gitlab.com/gitlab-org/gitlab/-/issues) with a link to your project. - In your issue, mention the Create:Source Code [Backend Engineering Manager and Product Manager](https://about.gitlab.com/handbook/product/categories/#source-code-group) - for the Templates feature. - -### Add the template SVG icon to GitLab SVGs - -If the project template has an SVG icon, you must add it to the -[GitLab SVGs project](https://gitlab.com/gitlab-org/gitlab-svgs/-/blob/main/README.md#adding-icons-or-illustrations) -before you can create a merge request with vendor details. - -### Create a merge request with vendor details - -Before GitLab can implement the project template, you must [create a merge request](../user/project/merge_requests/creating_merge_requests.md) in [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) that includes vendor details about the project. - -1. [Export the project](../user/project/settings/import_export.md#export-a-project-and-its-data) - and save the file as `<name>.tar.gz`, where `<name>` is the short name of the project. - Move this file to the root directory of `gitlab-org/gitlab`. -1. In `gitlab-org/gitlab`, create and check out a new branch. -1. Edit the following files to include the project template: - - For **non-Enterprise** project templates: - - In `lib/gitlab/project_template.rb`, add details about the template - in the `localized_templates_table` method. In the following example, - the short name of the project is `hugo`: - - ```ruby - ProjectTemplate.new('hugo', 'Pages/Hugo', _('Everything you need to create a GitLab Pages site using Hugo'), 'https://gitlab.com/pages/hugo', 'illustrations/logos/hugo.svg'), - ``` - - If the project doesn't have an SVG icon, exclude `, 'illustrations/logos/hugo.svg'`. - - - In `spec/support/helpers/project_template_test_helper.rb`, append the short name - of the template in the `all_templates` method. - - In `app/assets/javascripts/projects/default_project_templates.js`, - add details of the template. For example: - - ```javascript - hugo: { - text: s__('ProjectTemplates|Pages/Hugo'), - icon: '.template-option .icon-hugo', - }, - ``` - - If the project doesn't have an SVG icon, use `.icon-gitlab_logo` - instead. - - For **Enterprise** project templates: - - In `ee/lib/ee/gitlab/project_template.rb`, in the `localized_ee_templates_table` method, add details about the template. For example: - - ```ruby - ::Gitlab::ProjectTemplate.new('hipaa_audit_protocol', 'HIPAA Audit Protocol', _('A project containing issues for each audit inquiry in the HIPAA Audit Protocol published by the U.S. Department of Health & Human Services'), 'https://gitlab.com/gitlab-org/project-templates/hipaa-audit-protocol', 'illustrations/logos/asklepian.svg') - ``` - - - In `ee/spec/lib/gitlab/project_template_spec.rb`, add the short name - of the template in the `.all` test. - - In `ee/app/assets/javascripts/projects/default_project_templates.js`, - add the template details. For example: - - ```javascript - hipaa_audit_protocol: { - text: s__('ProjectTemplates|HIPAA Audit Protocol'), - icon: '.template-option .icon-hipaa_audit_protocol', - }, - ``` - -1. Run the following Rake task, where `<path>/<name>` is the - name you gave the template in `lib/gitlab/project_template.rb`: +If you'd like to contribute a new built-in project template to be distributed with GitLab, please do the following: + +1. Create a new public project with the project content you'd like to contribute in a namespace of your choosing. You can view a working example [here](https://gitlab.com/gitlab-org/project-templates/dotnetcore). + - Projects should be as simple as possible and free of any unnecessary assets or dependencies. +1. When the project is ready for review, please create a new issue in [GitLab](https://gitlab.com/gitlab-org/gitlab/issues) with a link to your project. + - In your issue, `@` mention the relevant Backend Engineering Manager and Product Manager for the [Create:Source Code group](https://about.gitlab.com/handbook/product/categories/#source-code-group). + +To make the project template available when creating a new project, the vendoring process will have to be completed: + +1. Create a working template ([example](https://gitlab.com/gitlab-org/project-templates/dotnetcore)) + - 2 types of built-in templates are available within GitLab: + - **Normal templates**: Available in GitLab Core, Starter and above (this is the most common type of built-in template). + - To contribute a normal template: + - Add details of the template in the `localized_templates_table` method in `gitlab/lib/gitlab/project_template.rb`, + - Add details of the template in `spec/lib/gitlab/project_template_spec.rb`, in the test for the `all` method, and + - Add details of the template in `gitlab/app/assets/javascripts/projects/default_project_templates.js`. + - See MR [!25318](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25318) for an example + - **Enterprise templates**: Introduced in GitLab 12.10, that are available only in GitLab Gold & Ultimate. + - To contribute an Enterprise template: + - Add details of the template in the `localized_ee_templates_table` method in `gitlab/ee/lib/ee/gitlab/project_template.rb`, + - Add details of the template in `gitlab/ee/spec/lib/gitlab/project_template_spec.rb`, in the `enterprise_templates` method, and + - Add details of the template in `gitlab/ee/app/assets/javascripts/projects/default_project_templates.js`. + - See MR [!28187](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28187) for an example. + +1. Run the following in the `gitlab` project, where `$name` is the name you gave the template in `gitlab/project_template.rb`: ```shell - bin/rake gitlab:update_project_templates\[<path>/<name>\] + bin/rake gitlab:update_project_templates[$name] ``` -1. Regenerate `gitlab.pot`: - - ```shell - bin/rake gettext:regenerate - ``` - -1. After you run the scripts, there is one new file in `vendor/project_templates/` and four changed files. Commit all changes and push your branch to update the merge request. For an example, see this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25318). +1. Run the `bundle_repo` script. Make sure to pass the correct arguments, or the script may damage the folder structure. +1. Add exported project (`$name.tar.gz`) to `gitlab/vendor/project_templates` and remove the resulting build folders `tar-base` and `project`. +1. Run `bin/rake gettext:regenerate` in the `gitlab` project and commit new `.pot` file. +1. Add a changelog entry in the commit message (for example, `Changelog: added`). + For more information, see [Changelog entries](changelog.md). +1. Add an icon to [`gitlab-svgs`](https://gitlab.com/gitlab-org/gitlab-svgs), as shown in + [this example](https://gitlab.com/gitlab-org/gitlab-svgs/merge_requests/195). If a logo + is not available for the project, use the default 'Tanuki' logo instead. +1. Run `yarn run svgs` on `gitlab-svgs` project and commit result. +1. Forward changes in `gitlab-svgs` project to master. This involves: + - Merging your MR in `gitlab-svgs` + - [The bot](https://gitlab.com/gitlab-org/frontend/renovate-gitlab-bot/) + will pick the new release up and create an MR in `gitlab-org/gitlab`. +1. Once the bot-created MR created above is merged, you can rebase your template MR onto the updated `master` to pick up the new svgs. +1. Test everything is working. + +### Contributing an improvement to an existing template + +Existing templates are available in the [project-templates](https://gitlab.com/gitlab-org/project-templates) +group. + +To contribute a change, please open a merge request in the relevant project +and mention `@gitlab-org/manage/import/backend` when you are ready for a review. + +Then, if your merge request gets accepted, either open an issue on +`gitlab` to ask for it to get updated, or open a merge request updating +the vendored template using [these instructions](rake_tasks.md#update-project-templates). ### Test your built-in project with the GitLab Development Kit @@ -124,17 +77,6 @@ Complete the following steps to test the project template in your own GitLab Dev bin/rake gitlab:update_project_templates\[<path>/<name>\] ``` -## Contribute an improvement to an existing template - -To update an existing built-in project template, changes are usually made to the existing template, found in the [project-templates](https://gitlab.com/gitlab-org/project-templates) group. A merge request is made directly against the template and the Create:Source Code [Backend Engineering Manager and Product Manager](https://about.gitlab.com/handbook/product/categories/#source-code-group) pinged for review. - -Sometimes it is necessary to completely replace the template files. In this case the process would be: - -1. Create a merge request in the relevant project of the `project-templates` and `pages` group and mention `@gitlab-org/manage/import/backend` when you are ready for a review. -1. If your merge request is accepted, either: - - [Create an issue](https://gitlab.com/gitlab-org/gitlab/-/issues) to ask for the template to get updated. - - [Create a merge request with vendor details](#create-a-merge-request-with-vendor-details) to update the template. - ## For GitLab team members Please ensure the merge request has been reviewed by the Security Counterpart before merging. diff --git a/doc/development/projections.md b/doc/development/projections.md index 7c727fc0901..caa54e7b912 100644 --- a/doc/development/projections.md +++ b/doc/development/projections.md @@ -28,8 +28,6 @@ You can find a basic list of projection options in - [Alternate File](https://marketplace.visualstudio.com/items?itemName=will-wow.vscode-alternate-file) - [projectionist](https://github.com/jarsen/projectionist) - [`jumpto`](https://github.com/gmdayley/jumpto) -- Atom - - [projectionist-atom](https://atom.io/packages/projectionist-atom) - Command-line - [projectionist](https://github.com/glittershark/projectionist) @@ -41,4 +39,4 @@ This started as a [plugin for vim by tpope](https://github.com/tpope/vim-projectionist) It has since become editor-agnostic and ported to most modern editors. -<!-- vale gitlab.Spelling = YES -->
\ No newline at end of file +<!-- vale gitlab.Spelling = YES --> diff --git a/doc/development/prometheus_metrics.md b/doc/development/prometheus_metrics.md index 590ac547c2d..f72805e0fe9 100644 --- a/doc/development/prometheus_metrics.md +++ b/doc/development/prometheus_metrics.md @@ -78,3 +78,15 @@ For example, a histogram with 10 buckets and a label with 100 values would gener entries in the export endpoint. 1. Trigger the relevant page or code that records the new metric. 1. Check that the new metric appears at `/-/metrics`. + +For metrics that are not bounded to a specific context (`request`, `process`, `machine`, `namespace`, etc), +generate them from a cron-based Sidekiq job: + +- For Geo related metrics, check `Geo::MetricsUpdateService`. +- For other "global" / instance-wide metrics, check: `Metrics::GlobalMetricsUpdateService`. + +When exporting data from Sidekiq in an installation with more than one Sidekiq instance, +you are not guaranteed that the same exporter will always be queried. + +You can read more and understand the caveats in [issue 406583](https://gitlab.com/gitlab-org/gitlab/-/issues/406583), +where we also discuss a possible solution using a push-gateway. diff --git a/doc/development/real_time.md b/doc/development/real_time.md index 99d09170646..017e308fc03 100644 --- a/doc/development/real_time.md +++ b/doc/development/real_time.md @@ -1,29 +1,347 @@ --- -stage: Plan -group: Project Management +stage: Data Stores +group: Application Performance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Real-time features +# Build and deploy real-time view components -This guide contains instructions on how to safely roll out new real-time -features. +GitLab provides an interactive user experience through individual view components that accept +user input and reflect state changes back to the user. For example, on the Merge Request +page, users can approve, leave comments, interact with the CI/CD pipeline, and more. -Real-time features are implemented using GraphQL Subscriptions. -[Developer documentation](api_graphql_styleguide.md#subscriptions) is available. +However, GitLab often does not reflect state updates in a timely manner. +This means parts of the page display stale data that only update after users reload the page. + +To address this, GitLab has introduced technology and programming APIs that allow view components +to receive state updates in real-time over a WebSocket. + +The following documentation tells you how to build and deploy view components that +receive updates in real-time from the GitLab Ruby on Rails server. + +NOTE: +Action Cable and GraphQL subscriptions are a work-in-progress and under active development. +Developers must evaluate their use case to check if these are the right tools to use. +If you are not sure, ask for help in the [`#f_real-time` internal Slack channel](https://gitlab.slack.com/archives/CUX9Z2N66). + +## Build real-time view components + +Prerequisites: + +Read the: + +- [GraphQL development guide](fe_guide/graphql.md). +- [Vue development guide](fe_guide/vue.md). + +To build a real-time view component on GitLab, you must: + +- Integrate a Vue component with Apollo subscriptions in the GitLab frontend. +- Add and trigger GraphQL subscriptions from the GitLab Ruby on Rails backend. + +### Integrate a Vue component with Apollo subscriptions + +NOTE: +Our current real-time stack assumes that client code is built using Vue as the rendering layer and +Apollo as the state and networking layer. If you are working with a part of +the GitLab frontend that has not been migrated to Vue + Apollo yet, complete that task first. + +Consider a hypothetical `IssueView` Vue component that observes and renders GitLab `Issue` data. +For simplicity, we assume here that all it does is render an issue's title and description: + +```javascript +import issueQuery from '~/issues/queries/issue_view.query.graqhql'; + +export default { + props: { + issueId: { + type: Number, + required: false, + default: null, + }, + }, + apollo: { + // Name of the Apollo query object. Must match the field name bound by `data`. + issue: { + // Query used for the initial fetch. + query: issueQuery, + // Bind arguments used for the initial fetch query. + variables() { + return { + iid: this.issueId, + }; + }, + // Map response data to view properties. + update(data) { + return data.project?.issue || {}; + }, + }, + }, + // Reactive Vue component data. Apollo updates these when queries return or subscriptions fire. + data() { + return { + issue: {}, // It is good practice to return initial state here while the view is loading. + }; + }, +}; + +// The <template> code is omitted for brevity as it is not relevant to this discussion. +``` + +The query should: + +- Be defined at `app/assets/javascripts/issues/queries/issue_view.query.graqhql`. +- Contain the following GraphQL operation: + + ```plaintext + query gitlabIssue($iid: String!) { + # We hard-code the path here only for illustration. Don't do this in practice. + project(fullPath: "gitlab-org/gitlab") { + issue(iid: $iid) { + title + description + } + } + } + ``` + +So far this view component only defines the initial fetch query to populate itself with data. +This is an ordinary GraphQL `query` operation sent as an HTTP POST request, initiated by the view. +Any subsequent updates on the server would make this view stale. For it to receive updates from the server, you must: + +1. Add a GraphQL subscription definition. +1. Define an Apollo subscription hook. + +#### Add a GraphQL subscription definition + +A subscription defines a GraphQL query as well, but it is wrapped inside a GraphQL `subscription` operation. +This query is initiated by the backend and its results pushed over a WebSocket into the view component. + +Similar to the initial fetch query, you must: + +- Define the subscription file at `app/assets/javascripts/issues/queries/issue_updated.subscription.graqhql`. +- Include the following GraphQL operation in the file: + + ```plaintext + subscription issueUpdatedSubscription($iid: String!) { + issueUpdated($issueId: IssueID!) { + issue(issueId: $issueId) { + title + description + } + } + } + ``` + +When adding new subscriptions, use the following naming guidelines: + +- End the subscription's operation name with `Subscription`, or `SubscriptionEE` if it's exclusive to GitLab EE. + For example, `issueUpdatedSubscription`, or `issueUpdatedSubscriptionEE`. +- Use a "has happened" action verb in the subscription's event name. For example, `issueUpdated`. + +While subscription definitions look similar to ordinary queries, there are some key differences that are important to understand: + +- The `query`: + - Originates from the frontend. + - Uses an internal ID (`iid`, numeric), which is how entities are usually referenced in URLs. + Because the internal ID is relative to the enclosing namespace (in this example, the `project`), you must nest the query under the `fullPath`. +- The `subscription`: + - Is a request from the frontend to the backend to receive future updates. + - Consists of: + - The operation name describing the subscription itself (`issueUpdatedSubscription` in this example). + - A nested event query (`issueUpdated` in this example). The nested event query: + - Executes when running the [GraphQL trigger](#trigger-graphql-subscriptions) of the same name, + so the event name used in the subscription must match the trigger field used in the backend. + - Uses a global ID string instead of a numeric internal ID, which is the preferred way to identify resources in GraphQL. + For more information, see [GraphQL global IDs](fe_guide/graphql.md#global-ids). + +#### Define an Apollo subscription hook + +After defining the subscription, add it to the view component using Apollo's `subscribeToMore` property: + +```javascript +import issueQuery from '~/issues/queries/issue_view.query.graqhql'; +import issueUpdatedSubscription from '~/issues/queries/issue_updated.subscription.graqhql'; + +export default { + // As before. + // ... + apollo: { + issue: { + // As before. + // ... + // This Apollo hook enables real-time pushes. + subscribeToMore: { + // Subscription operation that returns future updates. + document: issueUpdatedSubscription, + // Bind arguments used for the subscription operation. + variables() { + return { + iid: this.issueId, + }; + }, + // Implement this to return true|false if subscriptions should be disabled. + // Useful when using feature-flags. + skip() { + return this.shouldSkipRealTimeUpdates; + }, + }, + }, + }, + // As before. + // ... + computed: { + shouldSkipRealTimeUpdates() { + return false; // Might check a feature flag here. + }, + }, +}; +``` + +Now you can enable the view component to receive updates over a WebSocket connection through Apollo. +Next, we cover how events are triggered from the backend to initiate a push update to the frontend. + +### Trigger GraphQL subscriptions + +Writing a view component that can receive updates from a WebSocket is only half the story. +In the GitLab Rails application, we need to perform the following steps: + +1. Implement a `GraphQL::Schema::Subscription` class. This class: + - Is used by `graphql-ruby` to resolve the `subscription` operation sent by the frontend. + - Defines the arguments a subscription takes and the payload returned to the caller, if any. + - Runs any necessary business logic to ensure that the caller is authorized to create this subscription. +1. Add a new `field` to the `Types::SubscriptionType` class. This field maps the event name used + [when integrating the Vue component](#integrate-a-vue-component-with-apollo-subscriptions) to the + `GraphQL::Schema::Subscription` class. +1. Add a method matching the event name to `GraphqlTriggers` that runs the corresponding GraphQL trigger. +1. Use a service or Active Record model class to execute the new trigger as part of your domain logic. + +#### Implement the subscription + +If you subscribe to a an event that is already implemented as a `GraphQL::Schema::Subscription`, this step is optional. +Otherwise, create a new class under `app/graphql/subscriptions/` +that implements the new subscription. For the example of an `issueUpdated` event happening in response to an `Issue` being updated, +the subscription implementation is as follows: + +```ruby +module Subscriptions + class IssueUpdated < BaseSubscription + include Gitlab::Graphql::Laziness + + payload_type Types::IssueType + + argument :issue_id, Types::GlobalIDType[Issue], + required: true, + description: 'ID of the issue.' + + def authorized?(issue_id:) + issue = force(GitlabSchema.find_by_gid(issue_id)) + + unauthorized! unless issue && Ability.allowed?(current_user, :read_issue, issue) + + true + end + end +end +``` + +When creating this new class: + +- Make sure every subscription type inherits from `Subscriptions::BaseSubscription`. +- Use an appropriate `payload_type` to indicate what data subscribed queries may access, + or define the individual `field`s you want to expose. +- You may also define custom `subscribe` and `update` hooks that are called each time a client subscribes or + an event fires. Refer to the [official documentation](https://graphql-ruby.org/subscriptions/subscription_classes) + for how to use these methods. +- Implement `authorized?` to perform any necessary permission checks. These checks execute for each call + to `subscribe` or `update`. + +Read more about GraphQL subscription classes [in the official documentation](https://graphql-ruby.org/subscriptions/subscription_classes). + +#### Hook up the subscription + +Skip this step if you did not implement a new subscription class. + +After you implement a new subscription class, you must map that class to a `field` on the `SubscriptionType` before +it can execute. Open the `Types::SubscriptionType` class and add the new field: + +```ruby +module Types + class SubscriptionType < ::Types::BaseObject + graphql_name 'Subscription' + + # Existing fields + # ... + + field :issue_updated, + subscription: Subscriptions::IssueUpdated, null: true, + description: 'Triggered when an issue is updated.' + end +end +``` + +NOTE: +If you are connecting an EE subscription, update `EE::Types::SubscriptionType` instead. + +Make sure the `:issue_updated` argument matches the name used in the `subscription` request sent by the frontend in camel-case (`issueUpdated`), or `graphql-ruby` does not know which subscribers to inform. The event can now trigger. + +#### Add the new trigger + +Skip this step if you can reuse an existing trigger. + +We use a facade around `GitlabSchema.subscriptions.trigger` to make it simpler to trigger an event. +Add the new trigger to `GraphqlTriggers`: + +```ruby +module GraphqlTriggers + # Existing triggers + # ... + + def self.issue_updated(issue) + GitlabSchema.subscriptions.trigger(:issue_updated, { issue_id: issue.to_gid }, issue) + end +end +``` + +NOTE: +If the trigger is for an EE subscription, update `EE::GraphqlTriggers` instead. + +- The first argument, `:issue_updated`, must match the `field` name used in the previous + step. +- The argument hash specifies the issue for which we publish the event. + GraphQL uses this hash to identify the topic it should publish the event to. + +The final step is to call into this trigger function. + +#### Execute the trigger + +The implementation of this step depends on what exactly it is you are building. In the example +of the issue's fields changing, we could extend `Issues::UpdateService` to call `GraphqlTriggers.issue_updated`. + +The real-time view component is now functional. Updates to an issue should now propagate immediately into the GitLab UI. + +## Deploy real-time view components WebSockets are a relatively new technology at GitLab, and supporting them at scale introduces some challenges. For that reason, new features should be rolled out using the instructions below. -## Reuse an existing WebSocket connection +### Shipping a real-time component + +You can work on the frontend and backend at the same time, because updates over WebSockets +are difficult to simulate without the necessary backend code in place. + +However, it is safer to send changes in separate Merge Requests and deploy the backend changes first. +This ensures that when the frontend starts subscribing to events, the backend is already prepared +to service them. + +### Reuse an existing WebSocket connection Features reusing an existing connection incur minimal risk. Feature flag rollout is recommended to give more control to self-hosting customers. However, it is not necessary to roll out in percentages, or to estimate new connections for GitLab.com. -## Introduce a new WebSocket connection +### Introduce a new WebSocket connection Any change that introduces a WebSocket connection to part of the GitLab application incurs some scalability risk, both to nodes responsible for maintaining open @@ -70,7 +388,7 @@ of the feature flag ensures that effects can be observed on the 1. Copy in a member of the Plan and Scalability teams to estimate a percentage-based roll-out plan. -## Backward compatibility +### Backward compatibility For the duration of the feature flag roll-out and indefinitely thereafter, real-time features must be backward-compatible, or at least degrade @@ -80,18 +398,200 @@ needs to be done before Action Cable can be enabled by default. Making real-time a requirement represents a breaking change, so the next opportunity to do this is version 15.0. -## Enable Real-Time by default - -Mounting the Action Cable library adds minimal memory footprint. However, -serving WebSocket requests introduces additional memory requirements. For this -reason, enabling Action Cable by default requires additional work; perhaps -to reduce overall memory usage, including a known issue with Workhorse, but at -least to revise Reference Architectures. - -## Real-time infrastructure on GitLab.com +### Real-time infrastructure on GitLab.com On GitLab.com, WebSocket connections are served from dedicated infrastructure, entirely separate from the regular Web fleet and deployed with Kubernetes. This limits risk to nodes handling requests but not to shared services. For more information on the WebSockets Kubernetes deployment see [this epic](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/355). + +## The GitLab real-time stack in depth + +Because a push initiated by the server needs to propagate over the network and trigger a view update +in the client without any user interaction whatsoever, real-time features can only be understood +by looking at the entire stack including frontend and backend. + +NOTE: +For historic reasons, the controller routes that service updates in response to clients polling +for changes are called `realtime_changes`. They use conditional GET requests and are unrelated +to the real-time behavior covered in this guide. + +Any real-time update pushed into a client originates from the GitLab Rails application. We use the following +technologies to initiate and service these updates: + +In the GitLab Rails backend: + +- Redis PubSub to handle subscription state. +- Action Cable to handle WebSocket connections and data transport. +- `graphql-ruby` to implement GraphQL subscriptions and triggers. + +In the GitLab frontend: + +- Apollo Client to handle GraphQL requests, routing and caching. +- Vue.js to define and render view components that update in real-time. + +The following figure illustrates how data propagates between these layers. + +```mermaid +sequenceDiagram + participant V as Vue Component + participant AP as Apollo Client + participant P as Rails/GraphQL + participant AC as Action Cable/GraphQL + participant R as Redis PubSub + AP-->>V: injected + AP->>P: HTTP GET /-/cable + AC-->>P: Hijack TCP connection + AC->>+R: SUBSCRIBE(client) + R-->>-AC: channel subscription + AC-->>AP: HTTP 101: Switching Protocols + par + V->>AP: query(gql) + Note over AP,P: Fetch initial data for this view + AP->>+P: HTTP POST /api/graphql (initial query) + P-->>-AP: initial query response + AP->>AP: cache and/or transform response + AP->>V: trigger update + V->>V: re-render + and + Note over AP,AC: Subscribe to future updates for this view + V->>AP: subscribeToMore(event, gql) + AP->>+AC: WS: subscribe(event, query) + AC->>+R: SUBSCRIBE(event) + R-->>-AC: event subscription + AC-->>-AP: confirm_subscription + end + Note over V,R: time passes + P->>+AC: trigger event + AC->>+R: PUBLISH(event) + R-->>-AC: subscriptions + loop For each subscriber + AC->>AC: run GQL query + AC->>+R: PUBLISH(client, query_result) + R-->>-AC: callback + AC->>-AP: WS: push query result + end + AP->>AP: cache and/or transform response + AP->>V: trigger update + V->>V: re-render +``` + +In the subsequent sections we explain each element of this stack in detail. + +### Action Cable and WebSockets + +[Action Cable](https://guides.rubyonrails.org/action_cable_overview.html) is a library that adds +[WebSocket](https://www.rfc-editor.org/rfc/rfc6455) support to Ruby on Rails. +WebSockets were developed as an HTTP-friendly solution to enhance existing HTTP-based servers and +applications with bidirectional communication over a single TCP connection. +A client first sends an ordinary HTTP request to the server, asking it to upgrade the connection +to a WebSocket instead. When successful, the same TCP connection can then be used by both client +and server to send and receive data in either direction. + +Because the WebSocket protocol does not prescribe how the transmitted data is encoded or structured, +we need libraries like Action Cable that take care of these concerns. Action Cable: + +- Handles the initial connection upgrade from HTTP to the WebSocket protocol. Subsequent requests using + the `ws://` scheme are then handled by the Action Cable server and not Action Pack. +- Defines how data transmitted over the WebSocket is encoded. Action Cable specifies this to be JSON. This allows the + application to provide data as a Ruby Hash and Action Cable (de)serializes it from and to JSON. +- Provides callback hooks to handle clients connecting or disconnecting and client authentication. +- Provides `ActionCable::Channel` as a developer abstraction to implement publish/subscribe and remote procedure calls. + +Action Cable supports different implementations to track which client is subscribed to which +`ActionCable::Channel`. At GitLab we use the Redis adapter, which uses +[Redis PubSub](https://redis.io/docs/manual/pubsub/) channels as a distributed message bus. +Shared storage is necessary because different clients might connect to the same Action Cable channel +from different Puma instances. + +NOTE: +Do not confuse Action Cable channels with Redis PubSub channels. An Action Cable `Channel` object is a +programming abstraction to classify and handle the various kinds of data going over the WebSocket connection. +In Action Cable, the underlying PubSub channel is referred to as a broadcasting instead and the association +between a client and a broadcasting is called a subscription. In particular, there can be many broadcastings +(PubSub channels) and subscriptions for each Action Cable `Channel`. + +Because Action Cable allows us to express different kinds of behavior through its `Channel` API, and because +updates to any `Channel` can use the same WebSocket connection, we only require a single WebSocket connection +to be established for each GitLab page to enhance a view component on that page with real-time behavior. + +To implement real-time updates on a GitLab page, we do not write individual `Channel` implementations. +Instead, we provide the `GraphqlChannel` to which all pages that require push-based updates on GitLab +subscribe. + +### GraphQL subscriptions: Backend + +GitLab supports [GraphQL](https://graphql.org) for clients to request structured data from the server +using GraphQL queries. Refer to the [GitLab GraphQL overview](../api/graphql/index.md) to learn about why we adopted GraphQL. +GraphQL support in the GitLab backend is provided by the [`graphql-ruby`](https://graphql-ruby.org) gem. + +Ordinarily, GraphQL queries are client-initiated HTTP POST requests that follow the standard request-response cycle. +For real-time functionality, we use GraphQL subscriptions instead, which are an implementation of the publish/subscribe pattern. +In this approach the client first sends a subscription request to the `GraphqlChannel` with the: + +- Name of the subscription `field` (the event name). +- GraphQL query to run when this event triggers. + +This information is used by the server to create a `topic` that represents this event stream. The topic is a unique name +derived from the subscription arguments and event name and is used to identify all subscribers +that need to be informed if the event triggers. More than one client can subscribe to the +same topic. For example, `issuableAssigneesUpdated:issuableId:<hashed_id>` might serve as the topic +that clients subscribe to if they wish to be updated whenever the assignees for the issue with the +given ID change. + +The backend is responsible for triggering a subscription, typically in response to a domain +event such as "issue added to epic" or "user assigned to issue". At GitLab, this could be a service object +or an ActiveRecord model object. +A trigger is executed by calling into [`GitlabSchema.subscriptions.trigger`](https://gitlab.com/gitlab-org/gitlab/-/blob/5e3c334116178eec5f50fc5fee2ec0b3841a2504/app/graphql/graphql_triggers.rb) with the respective event name and arguments, +from which `graphql-ruby` derives the topic. It then finds all subscribers for this topic, executes the query for +each subscriber, and pushes the result back to all topic subscribers. + +Because we use Action Cable as the underlying transport for GraphQL subscriptions, topics are implemented +as Action Cable broadcastings, which as mentioned above represent Redis PubSub channels. +This means that for each subscriber, two PubSub channels are used: + +- One `graphql-event:<namespace>:<topic>` channel per each topic. This channel is used to track which client is subscribed + to which event and is shared among all potential clients. The use of a `namespace` is optional and it can be blank. +- One `graphql-subscription:<subscription-id>` channel per each client. This channel is used to transmit the query result + back to the respective client and hence cannot be shared between different clients. + +The next section describes how the GitLab frontend uses GraphQL subscriptions to implement real-time updates. + +### GraphQL subscriptions: Frontend + +Because the GitLab frontend executes JavaScript, not Ruby, we need a different GraphQL implementation +to send GraphQL queries, mutations, and subscriptions from the client to the server. +We use [Apollo](https://www.apollographql.com) to do this. + +Apollo is a comprehensive implementation of GraphQL in JavaScript and is split into `apollo-server` and `apollo-client` +as well as additional utility modules. Because we run a Ruby backend, we use `apollo-client` instead of `apollo-server`. + +It simplifies: + +- Networking, connection management and request routing. +- Client-side state management and response caching. +- Integrating GraphQL with view components using a bridge module. + +NOTE: +When reading the Apollo Client documentation, it assumes that React.js is used for view rendering. We do not use React.js +at GitLab. We use Vue.js, which integrates with Apollo using the [Vue.js adapter](https://apollo.vuejs.org/). + +Apollo provides functions and hooks with which you define how: + +- Views send queries, mutations or subscriptions. +- Responses should be dealt with. +- Response data is cached. + +The entry point is `ApolloClient`, which is a GraphQL client object that: + +- Is shared between all view components on a single page. +- All view components use internally to communicate with the server. + +To decide how different types of requests should be routed, Apollo uses the `ApolloLink` abstraction. Specifically, +it splits real-time server subscriptions from other GraphQL requests using the `ActionCableLink`. This: + +- Establishes the WebSocket connection to Action Cable. +- Maps server pushes to an `Observable` event stream in the client that views can subscribe to in order to update themselves. + +For more information about Apollo and Vue.js, see the [GitLab GraphQL development guide](fe_guide/graphql.md). diff --git a/doc/development/redis/new_redis_instance.md b/doc/development/redis/new_redis_instance.md index c1f69f4d103..00cc102b427 100644 --- a/doc/development/redis/new_redis_instance.md +++ b/doc/development/redis/new_redis_instance.md @@ -119,7 +119,7 @@ Migration Requirements: - No downtime. - No loss of stored data until the TTL for storing data expires. -- Partial rollout using Feature Flags or ENV vars or combinations of both. +- Partial rollout using feature flags or ENV vars or combinations of both. - Monitoring of the switch. - Prometheus metrics in place. - Easy rollback without downtime in case the new instance or logic does not behave as expected. diff --git a/doc/development/scalability.md b/doc/development/scalability.md index de9c57c2f2a..733e94cb5a7 100644 --- a/doc/development/scalability.md +++ b/doc/development/scalability.md @@ -123,7 +123,7 @@ the read replicas. [Omnibus ships with Patroni](../administration/postgresql/rep #### Load-balancing -GitLab EE has [application support for load balancing using read replicas](../administration/postgresql/database_load_balancing.md). This load balancer does +GitLab EE has [application support for load balancing using read replicas](database/load_balancing.md). This load balancer does some actions that aren't traditionally available in standard load balancers. For example, the application considers a replica only if its replication lag is low (for example, WAL data behind by less than 100 MB). diff --git a/doc/development/sec/index.md b/doc/development/sec/index.md index f74d31a1fb3..b887d13c267 100644 --- a/doc/development/sec/index.md +++ b/doc/development/sec/index.md @@ -52,7 +52,7 @@ The Analyzers are mainly written in Go. Some 3rd party integrators also make additional Scanners available by following our [integration documentation](../integrations/secure.md), which leverages the same architecture. -The results of the scans are exported as JSON reports that must comply with the [Secure report format](../../user/application_security/terminology/index.md#secure-report-format) and are uploaded as [CI/CD Job Report artifacts](../../ci/pipelines/job_artifacts.md) to make them available for processing after the pipelines completes. +The results of the scans are exported as JSON reports that must comply with the [Secure report format](../../user/application_security/terminology/index.md#secure-report-format) and are uploaded as [CI/CD Job Report artifacts](../../ci/jobs/job_artifacts.md) to make them available for processing after the pipelines completes. ### Processing, visualization, and management diff --git a/doc/development/sec/security_report_ingestion_overview.md b/doc/development/sec/security_report_ingestion_overview.md index 688986e0eb1..aca33990b0f 100644 --- a/doc/development/sec/security_report_ingestion_overview.md +++ b/doc/development/sec/security_report_ingestion_overview.md @@ -7,15 +7,56 @@ type: concepts # Security report ingestion overview -## Definitions +WARNING: +The `Vulnerability::Feedback` model is currently undergoing deprecation and should be actively avoided in all further development. It is currently maintained with feature parity to enable revert should any issues arise, but is intended to be removed in 16.0. Any interactions relating to the Feedback model are superseded by the `StateTransition`, `IssueLink`, and `MergeRequestLink` models. You can find out more on [in this epic](https://gitlab.com/groups/gitlab-org/-/epics/5629). -- **Vulnerability Finding** – an instance of `Vulnerabilities::Finding` class. This class was previously called `Vulnerabilities::Occurrence`; after renaming the class, we kept the associated table name `vulnerability_occurrences` due to the effort involved in renaming large tables. -- **Vulnerability** – an instance of `Vulnerability` class. They are created based on information available in `Vulnerabilities::Finding` class. Every `Vulnerability` **must have** a corresponding `Vulnerabilities::Finding` object to be valid, however this is not enforced at the database level. -- **Security Finding** – an instance of `Security::Finding` class. They store **partial** finding data to improve performance of the pipeline security report. We are working on extending this class to store almost all required information so we can stop relying on job artifacts. -- **Feedback** – an instance of `Vulnerabilities::Feedback` class. They are created to keep track of users' interactions with Vulnerability Findings before they are promoted to a Vulnerability. We are in the process of removing this model via [Deprecate and remove Vulnerabilities::Feedback epic](https://gitlab.com/groups/gitlab-org/-/epics/5629). -- **Issue Link** – an instance of `Vulnerabilities::IssueLink` class. They are used to link `Vulnerability` objects to `Issue` objects. +## Commonly used terms -## Vulnerability creation from security reports +### Feedback + +An instance of `Vulnerabilities::Feedback` class. They are created to keep track of users' interactions with Vulnerability Findings before they are promoted to a Vulnerability. This model is deprecated and due to be removed by GitLab 16.0 as part of the [Deprecate and remove Vulnerabilities::Feedback epic](https://gitlab.com/groups/gitlab-org/-/epics/5629). + +### Issue Link + +An instance of `Vulnerabilities::IssueLink` class. They are used to link `Vulnerability` records to `Issue` records. + +### Merge Request Link + +An instance of `Vulnerabilities::MergeRequestLink` class. They are used to link `Vulnerability` records to `MergeRequest` records. + +### Security Finding + +An instance of `Security::Finding` class. These serve as a meta-data store of a specific vulnerability detected in a specific `Security::Scan`. They currently store **partial** finding data to improve performance of the pipeline security report. This class has been extended to store almost all required scan information so we can stop relying on job artifacts and is [due to be used in favor of `Vulnerability::Findings` soon.](https://gitlab.com/gitlab-org/gitlab/-/issues/393394) + +### Security Scan + +An instance of the `Security::Scan` class. Security scans are representative of a `Ci::Build` which output a `Job Artifact` which has been output as a security scan result, which GitLab acknowledges and ingests the findings of as `Security::Finding` records. + +### State Transition + +An instance of the `Vulnerabilities::StateTransition` class. This model represents a state change of a respecitve Vulnerability record, for example the dismissal of a vulnerability which has been determined to be safe. + +### Vulnerability + +An instance of `Vulnerability` class. A `Vulnerability` is representative of a `Vulnerability::Finding` which has been detected in the default branch of the project, or if the `present_on_default_branch` flag is false, is representative of a finding which has been interacted with in some way outside of the default branch, such as if it is dismissed (`State Transition`), or linked to an `Issue` or `Merge Request`. They are created based on information available in `Vulnerabilities::Finding` class. Every `Vulnerability` **must have** a corresponding `Vulnerabilities::Finding` object to be valid, however this is not enforced at the database level. + +### Finding + +An instance of `Vulnerabilities::Finding` class. A `Vulnerability::Finding` is a database only representation of a security finding which has been merged into the default branch of a project, as the same `Vulnerability` may be present in multiple places within a project. This class was previously called `Vulnerabilities::Occurrence`; after renaming the class, we kept the associated table name `vulnerability_occurrences` due to the effort involved in renaming large tables. + +### Identifier + +An instance of the `Vulnerabilities::Identifier` class. Each vulnerability is given a unique identifier that can be derived from it's finding, enabling multiple Findings of the same `Vulnerability` to be correlated accordingly. + +### Vulnerability Read + +An instance of the `Vulnerabilities::Read` class. This is a denormalised record of `Vulnerability` and `Vulnerability::Finding` data to improve performance of filtered querying of vulnerability data to the front end. + +### Remediation + +An instance of the `Vulnerabilities::Remediation` class. A remediation is representative of a known solution to a detected `Vulnerability`. These enable GitLab to recommend a change to resolve a specific `Vulnerability`. + +## Vulnerability creation from Security Reports Assumptions: @@ -24,51 +65,53 @@ Assumptions: - No Vulnerabilities are present in the database - All pipelines perform security scans -1. Code is pushed to a branch that's **not** the default branch. +### Scan runs in a pipeline for a non-default branch + +1. Code is pushed to the branch. 1. GitLab CI runs a new pipeline for that branch. 1. Pipeline status transitions to any of [`::Ci::Pipeline.completed_statuses`](https://gitlab.com/gitlab-org/gitlab/-/blob/354261b2fe4fc5b86d1408467beadd90e466ce0a/app/models/concerns/ci/has_status.rb#L12). 1. `Security::StoreScansWorker` is called and it schedules `Security::StoreScansService`. -1. `Security::StoreScansService` calls `Security::StoreGroupedScansService`. +1. `Security::StoreScansService` calls `Security::StoreGroupedScansService` and schedules `ScanSecurityReportSecretsWorker`. 1. `Security::StoreGroupedScansService` calls `Security::StoreScanService`. 1. `Security::StoreScanService` calls `Security::StoreFindingsService`. -1. At this point we have `Security::Finding` objects **only**. +1. `ScanSecurityReportSecretsWorker` calls `Security::TokenRevocationService` to automatically revoke any leaked keys that were detected. +1. At this point we **only** have `Security::Finding` records as these findings are not present in the default branch of the project. -At this point, the following things can happen to the `Security::Finding`: +At this point, the following things can happen to the `Security::Finding` which would result in its promotion to a `Vulnerability::Finding` with a respective `Vulnerability` record: -- Dismissal -- Issue creation -- Promotion to a Vulnerability +### Scan runs in a pipeline for the default branch -If the pipeline ran on the default branch then the following, additional steps are done: +If the pipeline ran on the default branch then the following steps, in addition to the steps in [Scan runs in a pipeline for a non-default branch](#scan-runs-in-a-pipeline-for-a-non-default-branch), are executed: -1. `Security::StoreScansService` gets called and schedules `Security::StoreSecurityReportsWorker`. -1. `Security::StoreSecurityReportsWorker` executes `Security::Ingestion::IngestReportsService`. +1. `Security::StoreScansService` gets called and schedules `StoreSecurityReportsWorker`. +1. `StoreSecurityReportsWorker` executes `Security::Ingestion::IngestReportsService`. 1. `Security::Ingestion::IngestReportsService` takes all reports from a given Pipeline and calls `Security::Ingestion::IngestReportService` and then calls `Security::Ingestion::MarkAsResolvedService`. 1. `Security::Ingestion::IngestReportService` calls `Security::Ingestion::IngestReportSliceService` which executes a number of tasks for a report slice. ### Dismissal -If you select `Dismiss vulnerability`, a Feedback is created. You can also dismiss it with a comment. +If you change the state of a vulnerability, such as selecting `Dismiss vulnerability` the following things currently happen: -#### After Feedback removal +- A `Feedback` record of `dismissal` type is created to record the current state. +- If they do not already exist, a `Vulnerability Finding` and a `Vulnerability` with `present_on_default_branch: false` attribute get created, to which a `State Transition` reflecting the state change is related. -If there is only a Security Finding, a Vulnerability Finding and a Vulnerability get created. At the same time we create a `Vulnerabilities::StateTransition` record to indicate the Vulnerability was dismissed. +You can optionally add a comment to the state change which is recorded on both the `Feedback` and the `State Transition`. -### Issue creation +### Issue or Merge Request creation -If you select `Create issue`, a Vulnerabilities::Feedback record is created as well. The Feedback has a different `feedback_type` and an `issue_id` that's not `NULL`. +If you select `Create issue` or `Create merge request` the following things currently happen: -NOTE: -Vulnerabilities::Feedback are in the process of being [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/5629). This will later create a `Vulnerabilities::IssueLink` record. +- A `Vulnerabilities::Feedback` record is created. The Feedback will have a `feedback_type` of `issue` or `merge request` and an `issue_id` or `merge_request_id` that's not `NULL` respective to the attachment. +- If they do not already exist, a `Vulnerability Finding` and a `Vulnerability` with `present_on_default_branch: false` attribute get created, to which a `Issue Link` or `Merge Request Link` will be related respective to the action taken. -#### After Feedback removal +## Vulnerabilities in the Default Branch -If there's only a Security Finding, a Vulnerability Finding and a Vulnerability gets created. At the same time, we create an Issue and a Issue Link. +Security Findings detected in scan run on the default branch are saved as `Vulnerabilities` with the `present_on_default_branch: true` attribute and respective `Vulnerability Finding` records. `Vulnerability` records that already exist from interactions outside of the default branch will be updated to `present_on_default_branch: true` -## Promotion to a Vulnerability +`Vulnerabilities` which have already been interacted with will retain all existing `State Transitions`, `Merge Request Links` and `Issue Links`, as well as a corresponding `Vulnerability Feedback`. -If the branch with a Security Finding gets merged into the default branch, all Security Findings get promoted into Vulnerabilities. Promotion is the process of creating Vulnerability Findings and Vulnerability records from those Security Findings. +## Vulnerability Read Creation -If there's a dismissal Feedback present for that Security Finding, the created Vulnerability is marked as dismissed. +`Vulnerability::Read` records are created via postgres database trigger upon the creation of a `Vulnerability::Finding` record and as such are part of our ingestion process, though they have no impact on it bar it's denormalization performance benefits on the report pages. -If there's an issue Feedback present for that Security Finding, we also create an Issue Link for that Vulnerability. +This style of creation was intended to be fast and seamless, but has proven difficult to debug and maintain and may be [migrated to the application layer later](https://gitlab.com/gitlab-org/gitlab/-/issues/393912). diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md index 4f644dd018e..f1342d24fb4 100644 --- a/doc/development/secure_coding_guidelines.md +++ b/doc/development/secure_coding_guidelines.md @@ -1290,6 +1290,7 @@ This sensitive data must be handled carefully to avoid leaks which could lead to - Credentials must be encrypted while at rest (database or file) with `attr_encrypted`. See [issue #26243](https://gitlab.com/gitlab-org/gitlab/-/issues/26243) before using `attr_encrypted`. - Store the encryption keys separately from the encrypted credentials with proper access control. For instance, store the keys in a vault, KMS, or file. Here is an [example](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/user.rb#L70-74) use of `attr_encrypted` for encryption with keys stored in separate access controlled file. - When the intention is to only compare secrets, store only the salted hash of the secret instead of the encrypted value. +- Salted hashes should be used to store any sensitive value where the plaintext value itself does not need to be retrieved. - Never commit credentials to repositories. - The [Gitleaks Git hook](https://gitlab.com/gitlab-com/gl-security/security-research/gitleaks-endpoint-installer) is recommended for preventing credentials from being committed. - Never log credentials under any circumstance. Issue [#353857](https://gitlab.com/gitlab-org/gitlab/-/issues/353857) is an example of credential leaks through log file. @@ -1306,6 +1307,32 @@ This sensitive data must be handled carefully to avoid leaks which could lead to In the event of credential leak through an MR, issue, or any other medium, [reach out to SIRT team](https://about.gitlab.com/handbook/security/security-operations/sirt/#-engaging-sirt). +### Examples + +Encrypting a token with `attr_encrypted` so that the plaintext can be retrieved and used later: + +```ruby +module AlertManagement + class HttpIntegration < ApplicationRecord + + attr_encrypted :token, + mode: :per_attribute_iv, + key: Settings.attr_encrypted_db_key_base_32, + algorithm: 'aes-256-gcm' +``` + +Hashing a sensitive value with `CryptoHelper` so that it can be compared in future, but the plaintext is irretrievable: + +```ruby +class WebHookLog < ApplicationRecord + before_save :set_url_hash, if: -> { interpolated_url.present? } + + def set_url_hash + self.url_hash = Gitlab::CryptoHelper.sha256(interpolated_url) + end +end +``` + ## Serialization Serialization of active record models can leak sensitive attributes if they are not protected. diff --git a/doc/development/service_ping/implement.md b/doc/development/service_ping/implement.md index 5bfb81c1d00..54352a43010 100644 --- a/doc/development/service_ping/implement.md +++ b/doc/development/service_ping/implement.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/service_ping/index.md b/doc/development/service_ping/index.md index e938de9e253..9d3f3d37dca 100644 --- a/doc/development/service_ping/index.md +++ b/doc/development/service_ping/index.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/service_ping/metrics_dictionary.md b/doc/development/service_ping/metrics_dictionary.md index 90ce776af19..f36a97bcf6b 100644 --- a/doc/development/service_ping/metrics_dictionary.md +++ b/doc/development/service_ping/metrics_dictionary.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -37,7 +37,6 @@ Each metric is defined in a separate YAML file consisting of a number of fields: | `product_section` | yes | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml). | | `product_stage` | yes | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the metric. | | `product_group` | yes | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the metric. | -| `product_category` | no | The [product category](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/categories.yml) for the metric. | | `value_type` | yes | `string`; one of [`string`, `number`, `boolean`, `object`](https://json-schema.org/understanding-json-schema/reference/type.html). | | `status` | yes | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `removed`, `broken`. | | `time_frame` | yes | `string`; may be set to a value like `7d`, `28d`, `all`, `none`. | @@ -218,7 +217,6 @@ instance unique identifier. ```yaml key_path: uuid description: GitLab instance unique identifier -product_category: collection product_section: analytics product_stage: analytics product_group: product_intelligence diff --git a/doc/development/service_ping/metrics_instrumentation.md b/doc/development/service_ping/metrics_instrumentation.md index 14ec4b8c9a6..7441a2d1bd4 100644 --- a/doc/development/service_ping/metrics_instrumentation.md +++ b/doc/development/service_ping/metrics_instrumentation.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/service_ping/metrics_lifecycle.md b/doc/development/service_ping/metrics_lifecycle.md index 7fcbafe50f7..3c51eefc4b4 100644 --- a/doc/development/service_ping/metrics_lifecycle.md +++ b/doc/development/service_ping/metrics_lifecycle.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/service_ping/performance_indicator_metrics.md b/doc/development/service_ping/performance_indicator_metrics.md index 4c1c61aa05b..0ca663ce09a 100644 --- a/doc/development/service_ping/performance_indicator_metrics.md +++ b/doc/development/service_ping/performance_indicator_metrics.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/service_ping/review_guidelines.md b/doc/development/service_ping/review_guidelines.md index 9813c9e0b12..f260d9700a3 100644 --- a/doc/development/service_ping/review_guidelines.md +++ b/doc/development/service_ping/review_guidelines.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -44,7 +44,7 @@ are regular backend changes. - Assign an [engineer](https://gitlab.com/groups/gitlab-org/analytics-section/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) from the Product Intelligence team for a review. - Set the correct attributes in the metric's YAML definition: - - `product_section`, `product_stage`, `product_group`, `product_category` + - `product_section`, `product_stage`, `product_group` - Provide a clear description of the metric. - Add a changelog [according to guidelines](../changelog.md). @@ -59,12 +59,11 @@ are regular backend changes. metrics that are based on Database. - Add `~Data Warehouse::Impact Check` for any database metric that has a query change. Changes in queries can affect [data operations](https://about.gitlab.com/handbook/business-technology/data-team/how-we-work/triage/#gitlabcom-db-structure-changes). - For tracking using Redis HLL (HyperLogLog): - - Check the Redis slot. - Check if a [feature flag is needed](implement.md#recommendations). - For a metric's YAML definition: - Check the metric's `description`. - Check the metric's `key_path`. - - Check the `product_section`, `product_stage`, `product_group`, and `product_category` fields. + - Check the `product_section`, `product_stage`, and `product_group` fields. Read the [stages file](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml). - Check the file location. Consider the time frame, and if the file should be under `ee`. - Check the tiers. diff --git a/doc/development/service_ping/troubleshooting.md b/doc/development/service_ping/troubleshooting.md index 3b7cd092d97..1b896efb726 100644 --- a/doc/development/service_ping/troubleshooting.md +++ b/doc/development/service_ping/troubleshooting.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/service_ping/usage_data.md b/doc/development/service_ping/usage_data.md index 1c7a212ed64..b3bdaedd60a 100644 --- a/doc/development/service_ping/usage_data.md +++ b/doc/development/service_ping/usage_data.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/sidekiq/worker_attributes.md b/doc/development/sidekiq/worker_attributes.md index a3bfe5f27cc..1e3104c5e86 100644 --- a/doc/development/sidekiq/worker_attributes.md +++ b/doc/development/sidekiq/worker_attributes.md @@ -242,7 +242,7 @@ can put unsustainable load on the primary database server. We therefore added th By configuring a worker's `data_consistency` field, we can then allow the scheduler to target read replicas under several strategies outlined below. -## Trading immediacy for reduced primary load +### Trading immediacy for reduced primary load We require Sidekiq workers to make an explicit decision around whether they need to use the primary database node for all reads and writes, or whether reads can be served from replicas. This is @@ -259,7 +259,8 @@ that mostly or exclusively perform writes, or workers that read their own writes into data consistency issues should a stale record be read back from a replica. **Try to avoid these scenarios, since `:always` should be considered the exception, not the rule.** -To allow for reads to be served from replicas, we added two additional consistency modes: `:sticky` and `:delayed`. +To allow for reads to be served from replicas, we added two additional consistency modes: `:sticky` and `:delayed`. A RuboCop rule +reminds the developer when `:always` data consistency mode is used. If workers require the primary database, you can disable the rule in-line. When you declare either `:sticky` or `:delayed` consistency, workers become eligible for database load-balancing. @@ -268,18 +269,17 @@ In both cases, if the replica is not up-to-date and the time from scheduling the the jobs sleep up to the minimum delay interval (0.8 seconds). This gives the replication process time to finish. The difference is in what happens when there is still replication lag after the delay: `sticky` workers switch over to the primary right away, whereas `delayed` workers fail fast and are retried once. -If they still encounter replication lag, they also switch to the primary instead. -**If your worker never performs any writes, it is strongly advised to apply one of these consistency settings, -since it never needs to rely on the primary database node.** +If the workers still encounter replication lag, they switch to the primary instead. **If your worker never performs any writes, +it is strongly advised to apply `:sticky` or `:delayed` consistency settings, since the worker never needs to rely on the primary database node.** The table below shows the `data_consistency` attribute and its values, ordered by the degree to which they prefer read replicas and wait for replicas to catch up: -| **Data Consistency** | **Description** | -|--------------|-----------------------------| -| `:always` | The job is required to use the primary database (default). It should be used for workers that primarily perform writes, have strict requirements around data consistency when reading their own writes, or are cron jobs. | -| `:sticky` | The job prefers replicas, but switches to the primary for writes or when encountering replication lag. It should be used for jobs that require to be executed as fast as possible but can sustain a small initial queuing delay. | -| `:delayed` | The job prefers replicas, but switches to the primary for writes. When encountering replication lag before the job starts, the job is retried once. If the replica is still not up to date on the next retry, it switches to the primary. It should be used for jobs where delaying execution further typically does not matter, such as cache expiration or web hooks execution. | +| **Data consistency** | **Description** | **Guideline** | +|--------------|-----------------------------|----------| +| `:always` | The job is required to use the primary database (default). | It should be used for workers that primarily perform writes, have strict requirements around data consistency when reading their own writes, or are cron jobs. | +| `:sticky` | The job prefers replicas, but switches to the primary for writes or when encountering replication lag. | It should be used for jobs that require to be executed as fast as possible but can sustain a small initial queuing delay. | +| `:delayed` | The job prefers replicas, but switches to the primary for writes. When encountering replication lag before the job starts, the job is retried once. If the replica is still not up to date on the next retry, it switches to the primary. | It should be used for jobs where delaying execution further typically does not matter, such as cache expiration or web hooks execution. | In all cases workers read either from a replica that is fully caught up, or from the primary node, so data consistency is always ensured. diff --git a/doc/development/snowplow/event_dictionary_guide.md b/doc/development/snowplow/event_dictionary_guide.md index dc2214a40ed..8825a38627a 100644 --- a/doc/development/snowplow/event_dictionary_guide.md +++ b/doc/development/snowplow/event_dictionary_guide.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/snowplow/implementation.md b/doc/development/snowplow/implementation.md index ae90b45b1f0..05137a4410f 100644 --- a/doc/development/snowplow/implementation.md +++ b/doc/development/snowplow/implementation.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/snowplow/index.md b/doc/development/snowplow/index.md index 276b5913890..4ccb90c22a6 100644 --- a/doc/development/snowplow/index.md +++ b/doc/development/snowplow/index.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/snowplow/infrastructure.md b/doc/development/snowplow/infrastructure.md index ae416f40c98..ac146542630 100644 --- a/doc/development/snowplow/infrastructure.md +++ b/doc/development/snowplow/infrastructure.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/snowplow/review_guidelines.md b/doc/development/snowplow/review_guidelines.md index da7f2bc2781..2cf13385179 100644 --- a/doc/development/snowplow/review_guidelines.md +++ b/doc/development/snowplow/review_guidelines.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/snowplow/schemas.md b/doc/development/snowplow/schemas.md index da58cd5f2e5..0ef6ed04aa3 100644 --- a/doc/development/snowplow/schemas.md +++ b/doc/development/snowplow/schemas.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/snowplow/troubleshooting.md b/doc/development/snowplow/troubleshooting.md index 47df3e43d57..92267dfcb0c 100644 --- a/doc/development/snowplow/troubleshooting.md +++ b/doc/development/snowplow/troubleshooting.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/stage_group_observability/dashboards/error_budget_detail.md b/doc/development/stage_group_observability/dashboards/error_budget_detail.md index a8f932b78c0..c45c31c5bd1 100644 --- a/doc/development/stage_group_observability/dashboards/error_budget_detail.md +++ b/doc/development/stage_group_observability/dashboards/error_budget_detail.md @@ -100,28 +100,3 @@ In the graphs, there is a single line per service. In the previous example image Sidekiq is not included in this dashboard. We're tracking this in [epic 700](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/700). - -### SLI detail - - - -The SLI details row shows a breakdown of a specific SLI based on the -labels present on the source metrics. - -For example, in the previous image, the `rails_requests` SLI has an `endpoint_id` label. -We can show how much a certain endpoint was requested (RPS), and how much it contributed to the error -budget spend. - -For Apdex we show the **Apdex Attribution** panel. The more prominent -color is the one that contributed most to the spend. To see the -top spending endpoint over the entire range, sort by the average. - -For error ratio we show an error rate. To see which label contributed most to the spend, sort by the -average. - -We don't have endpoint information available for Rails errors. This work is being planned in -[epic 663](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/663). - -The number of series to be loaded in the SLI details graphs is very -high when compared to the other aggregations. Because of this, it's not possible to -load more than a few days' worth of data. diff --git a/doc/development/stage_group_observability/dashboards/img/error_budget_detail_sli_detail.png b/doc/development/stage_group_observability/dashboards/img/error_budget_detail_sli_detail.png Binary files differdeleted file mode 100644 index 99530886ae9..00000000000 --- a/doc/development/stage_group_observability/dashboards/img/error_budget_detail_sli_detail.png +++ /dev/null diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index a9d17472a9f..3a2b40a6cd5 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -313,7 +313,28 @@ NOTE: `stub_method` does not support method existence and method arity checks. WARNING: -`stub_method` is supposed to be used in factories only. It's strongly discouraged to be used elsewhere. Please consider using [RSpec's mocks](https://relishapp.com/rspec/rspec-mocks/v/3-10/docs/basics) if available. +`stub_method` is supposed to be used in factories only. It's strongly discouraged to be used elsewhere. Please consider using [RSpec mocks](https://rspec.info/features/3-12/rspec-mocks/) if available. + +#### Stubbing member access level + +To stub [member access level](../../user/permissions.md#roles) for factory stubs like `Project` or `Group` use +[`stub_member_access_level`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/stub_member_access_level.rb): + +```ruby +let(:project) { build_stubbed(:project) } +let(:maintainer) { build_stubbed(:user) } +let(:policy) { ProjectPolicy.new(maintainer, project) } + +it 'allows admin_project ability' do + stub_member_access_level(project, maintainer: maintainer) + + expect(policy).to be_allowed(:admin_project) +end +``` + +NOTE: +Refrain from using this stub helper if the test code relies on persisting +`project_authorizations` or `Member` records. Use `Project#add_member` or `Group#add_member` instead. #### Identify slow tests @@ -434,7 +455,7 @@ results are available, and not just the first failure. - When using `evaluate_script("$('.js-foo').testSomething()")` (or `execute_script`) which acts on a given element, use a Capybara matcher beforehand (such as `find('.js-foo')`) to ensure the element actually exists. - Use `focus: true` to isolate parts of the specs you want to run. -- Use [`:aggregate_failures`](https://relishapp.com/rspec/rspec-core/docs/expectation-framework-integration/aggregating-failures) when there is more than one expectation in a test. +- Use [`:aggregate_failures`](https://rspec.info/features/3-12/rspec-core/expectation-framework-integration/aggregating-failures/) when there is more than one expectation in a test. - For [empty test description blocks](https://github.com/rubocop-hq/rspec-style-guide#it-and-specify), use `specify` rather than `it do` if the test is self-explanatory. - Use `non_existing_record_id`/`non_existing_record_iid`/`non_existing_record_access_level` when you need an ID/IID/access level that doesn't actually exists. Using 123, 1234, @@ -451,6 +472,10 @@ You can use `if: Gitlab.ee?` or `unless: Gitlab.ee?` on context/spec blocks to e Example: [SchemaValidator reads a different path depending on the license](https://gitlab.com/gitlab-org/gitlab/-/blob/7cdcf9819cfa02c701d6fa9f18c1e7a8972884ed/spec/lib/gitlab/ci/parsers/security/validators/schema_validator_spec.rb#L571) +### Tests depending on SaaS + +You can use the `:saas` RSpec metadata tag helper on context/spec blocks to test code that only runs on GitLab.com. This helper sets `Gitlab.config.gitlab['url']` to `Gitlab::Saas.com_url`. + ### Coverage [`simplecov`](https://github.com/colszowka/simplecov) is used to generate code test coverage reports. @@ -867,8 +892,6 @@ it 'is overdue' do travel_to(3.days.from_now) do expect(issue).to be_overdue end - - travel_back # Returns the current time back to its original state end ``` @@ -1017,7 +1040,7 @@ a single test triggers the rate limit, the `:disable_rate_limit` can be used ins #### Stubbing File methods In the situations where you need to -[stub](https://relishapp.com/rspec/rspec-mocks/v/3-9/docs/basics/allowing-messages) +[stub](https://rspec.info/features/3-12/rspec-mocks/basics/allowing-messages/) methods such as `File.read`, make sure to: 1. Stub `File.read` for only the file path you are interested in. diff --git a/doc/development/testing_guide/end_to_end/beginners_guide.md b/doc/development/testing_guide/end_to_end/beginners_guide.md index 07a9bcb2f5c..78e6af4a347 100644 --- a/doc/development/testing_guide/end_to_end/beginners_guide.md +++ b/doc/development/testing_guide/end_to_end/beginners_guide.md @@ -34,9 +34,8 @@ For more information, see [End-to-end testing Best Practices](best_practices.md) ## Determine if end-to-end tests are needed -Check the code coverage of a specific feature before writing end-to-end tests, -for both [GitLab Community Edition](https://gitlab-org.gitlab.io/gitlab-foss/coverage-ruby/#_AllFiles) -and [GitLab Enterprise Edition](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/#_AllFiles) projects. +Check the code coverage of a specific feature before writing end-to-end tests +for the [GitLab](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/#_AllFiles) project. Does sufficient test coverage exist at the unit, feature, or integration levels? If you answered *yes*, then you *don't* need an end-to-end test. @@ -53,9 +52,8 @@ For information about the distribution of tests per level in GitLab, see the feature and the lower-level tests. WARNING: -Check both [GitLab Community Edition](https://gitlab-org.gitlab.io/gitlab-foss/coverage-ruby/#_AllFiles) and -[GitLab Enterprise Edition](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/#_AllFiles) coverage projects -for previously-written tests for this feature. For analyzing the code coverage, +Check the [GitLab](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/#_AllFiles) coverage project +for previously written tests for this feature. To analyze code coverage, you must understand which application files implement specific features. In this tutorial we're writing a login end-to-end test, even though it has been @@ -225,7 +223,7 @@ end 1. Check if the user avatar *does not* appear in the top navigation. Behind the scenes, `be_signed_in` is a -[predicate matcher](https://relishapp.com/rspec/rspec-expectations/v/3-8/docs/built-in-matchers/predicate-matchers) +[predicate matcher](https://rspec.info/features/3-12/rspec-expectations/built-in-matchers/predicates/) that [implements checking the user avatar](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/page/main/menu.rb#L92). ## De-duplicate your code diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md index fd184c13e5e..c1f06bb9a66 100644 --- a/doc/development/testing_guide/end_to_end/best_practices.md +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -413,7 +413,7 @@ except(page).to have_no_text('hidden') ``` Unfortunately, that's not automatically the case for the predicate methods that we add to our -[page objects](page_objects.md). We need to [create our own negatable matchers](https://relishapp.com/rspec/rspec-expectations/v/3-9/docs/custom-matchers/define-a-custom-matcher#matcher-with-separate-logic-for-expect().to-and-expect().not-to). +[page objects](page_objects.md). We need to [create our own negatable matchers](https://rspec.info/features/3-12/rspec-expectations/custom-matchers/define-matcher/). The initial example uses the `have_job` matcher which is derived from the [`has_job?` predicate method of the `Page::Project::Pipeline::Show` page object](https://gitlab.com/gitlab-org/gitlab/-/blob/87864b3047c23b4308f59c27a3757045944af447/qa/qa/page/project/pipeline/show.rb#L53). diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md index c9e60c06732..a42d5e3df1d 100644 --- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md +++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # RSpec metadata for end-to-end tests -This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec-core/docs/metadata/user-defined-metadata) +This is a partial list of the [RSpec metadata](https://rspec.info/features/3-12/rspec-core/metadata/user-defined/) (a.k.a. tags) that are used in our end-to-end tests. <!-- Please keep the tags in alphabetical order --> diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md index 0d6d7a161c7..bf4d750423d 100644 --- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md +++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md @@ -428,6 +428,8 @@ For instructions on how to run these tests using the `gitlab-qa` gem, please ref Tests that are tagged with `:mobile` can be run against specified mobile devices using cloud emulator/simulator services. +These tests run in the [nightly pipeline](https://gitlab.com/gitlab-org/quality/nightly/-/pipelines) in the `ce:remote_mobile_safari` job. + ### How to run mobile tests with Sauce Labs Running directly against an environment like staging is not recommended because Sauce Labs test logs expose credentials. Therefore, it is best practice and the default to use a tunnel. diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md index c626a4fa81c..a7a5e7a97d5 100644 --- a/doc/development/testing_guide/flaky_tests.md +++ b/doc/development/testing_guide/flaky_tests.md @@ -130,6 +130,7 @@ Adding a delay in API or controller could help reproducing the issue. **Examples:** - [Example 1](https://gitlab.com/gitlab-org/gitlab/-/issues/118612): A test that breaks after some time passed. +- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/issues/403332): A test that breaks in the last day of the month. ### Unstable infrastructure @@ -150,7 +151,7 @@ usually a good idea. ## Quarantined tests -When a test frequently fails in `master`, +When we have a flaky test in `master`, quarantine the test after the first failure and create [a ~"failure::flaky-test" issue](https://about.gitlab.com/handbook/engineering/workflow/#broken-master). If the test cannot be fixed in a timely fashion, there is an impact on the @@ -158,6 +159,7 @@ productivity of all the developers, so it should be quarantined. There are two w ### RSpec +Add the corresponding category and group labels to issue using [`feature_category` metadata](../feature_categorization/index.md#rspec-examples). For RSpec tests, you can use the `:quarantine` metadata with the issue URL. ```ruby @@ -300,6 +302,12 @@ If a spec hangs, it might be caused by a [bug in Rails](https://github.com/rails - <https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81112> - <https://gitlab.com/gitlab-org/gitlab/-/issues/337039> +## Suggestions + +### Split the test file + +It could help to split the large RSpec files in multiple files in order to narrow down the context and identify the problematic tests. + ## Resources - [Flaky Tests: Are You Sure You Want to Rerun Them?](https://semaphoreci.com/blog/2017/04/20/flaky-tests.html) diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 6134e0f9959..a3b580a43fa 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -567,6 +567,53 @@ Example }); ``` +### Testing local-only Apollo queries and mutations + +To add a new query or mutation before it is added to the backend, we can use the `@client` directive. For example: + +```graphql +mutation setActiveBoardItemEE($boardItem: LocalBoardItem, $isIssue: Boolean = true) { + setActiveBoardItem(boardItem: $boardItem) @client { + ...Issue @include(if: $isIssue) + ...EpicDetailed @skip(if: $isIssue) + } +} +``` + +When writing test cases for such calls, we can use resolvers to make sure they are called with the correct parameters. + +For example, when creating the wrapper, we should make sure the resolver is mapped to the query or mutation. +The mutation we are mocking here is `setActiveBoardItem`: + +```javascript +const mockSetActiveBoardItemResolver = jest.fn(); +const mockApollo = createMockApollo([], { + Mutation: { + setActiveBoardItem: mockSetActiveBoardItemResolver, + }, +}); +``` + +In the following code, we must pass four arguments. The second one must be the collection of input variables of the query or mutation mocked. +To test that the mutation is called with the correct parameters: + +```javascript +it('calls setActiveBoardItemMutation on close', async () => { + wrapper.findComponent(GlDrawer).vm.$emit('close'); + + await waitForPromises(); + + expect(mockSetActiveBoardItemResolver).toHaveBeenCalledWith( + {}, + { + boardItem: null, + }, + expect.anything(), + expect.anything(), + ); +}); +``` + ### Jest best practices > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34209) in GitLab 13.2. @@ -819,10 +866,10 @@ often using fixtures to validate correct integration with the backend code. ### Use fixtures -To import a JSON fixture, `import` it using the `test_fixtures` alias. +To import a JSON or HTML fixture, `import` it using the `test_fixtures` alias. ```javascript -import responseBody from 'test_fixtures/some/fixture.json' // loads spec/frontend/fixtures/some/fixture.json +import responseBody from 'test_fixtures/some/fixture.json' // loads tmp/tests/frontend/fixtures-ee/some/fixture.json it('makes a request', () => { axiosMock.onGet(endpoint).reply(200, responseBody); @@ -833,23 +880,6 @@ it('makes a request', () => { }); ``` -For other fixtures, Jest uses `spec/frontend/__helpers__/fixtures.js` to import them in tests. - -The following are examples of tests that work for Jest: - -```javascript -it('uses some HTML element', () => { - loadHTMLFixture('some/page.html'); // loads spec/frontend/fixtures/some/page.html and adds it to the DOM - - const element = document.getElementById('#my-id'); - - // ... - - // Jest does not clean up the DOM automatically - resetHTMLFixture(); -}); -``` - ### Generate fixtures You can find code to generate test fixtures in: diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index 7958d6db706..1e5ee9f3003 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -47,8 +47,8 @@ Maintainers can elect to use the [process for merging during broken `master`](ht ## Performance Metrics -On every [pipeline](https://gitlab.com/gitlab-org/gitlab/pipelines/125315730) in the `qa` stage, the -`review-performance` job is automatically started: this job does basic +On every Review App child pipeline in the `qa` stage, the +`browser_performance` job is automatically started: this job does basic browser performance testing using a [Sitespeed.io Container](../../ci/testing/browser_performance_testing.md). @@ -227,7 +227,7 @@ Review apps are automatically stopped 2 days after the last deployment thanks to the [Environment auto-stop](../../ci/environments/index.md#stop-an-environment-after-a-certain-time-period) feature. If you need your review app to stay up for a longer time, you can -[pin its environment](../../ci/environments/index.md#override-a-deployments-scheduled-stop-time) or retry the +[pin its environment](../../ci/environments/index.md#override-a-environments-scheduled-stop-date-and-time) or retry the `review-deploy` job to update the "latest deployed at" time. The `review-cleanup` job that automatically runs in scheduled diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 187d30c12be..480a53bbefe 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -55,6 +55,7 @@ records should use stubs/doubles as much as possible. | `lib/` | `spec/lib/` | RSpec | | | `lib/tasks/` | `spec/tasks/` | RSpec | | | `rubocop/` | `spec/rubocop/` | RSpec | | +| `spec/support/` | `spec/support_specs/` | RSpec | | ### Frontend unit tests @@ -65,7 +66,7 @@ that is not directly perceivable by a user. graph RL plain[Plain JavaScript]; Vue[Vue Components]; - feature-flags[Feature Flags]; + feature-flags[Feature flags]; license-checks[License Checks]; plain---Vuex; @@ -149,7 +150,7 @@ Component tests cover the state of a single component that is perceivable by a u graph RL plain[Plain JavaScript]; Vue[Vue Components]; - feature-flags[Feature Flags]; + feature-flags[Feature flags]; license-checks[License Checks]; plain---Vuex; @@ -243,7 +244,7 @@ Their abstraction level is comparable to how a user would interact with the UI. graph RL plain[Plain JavaScript]; Vue[Vue Components]; - feature-flags[Feature Flags]; + feature-flags[Feature flags]; license-checks[License Checks]; plain---Vuex; @@ -366,13 +367,12 @@ See also: - The [RSpec testing guidelines](../testing_guide/best_practices.md#rspec). - System / Feature tests in the [Testing Best Practices](best_practices.md#system--feature-tests). -- [Issue #26159](https://gitlab.com/gitlab-org/gitlab/-/issues/26159) which aims at combining those guidelines with this page. ```mermaid graph RL plain[Plain JavaScript]; Vue[Vue Components]; - feature-flags[Feature Flags]; + feature-flags[Feature flags]; license-checks[License Checks]; plain---Vuex; diff --git a/doc/development/value_stream_analytics.md b/doc/development/value_stream_analytics.md index 634c337793f..5f500640841 100644 --- a/doc/development/value_stream_analytics.md +++ b/doc/development/value_stream_analytics.md @@ -221,6 +221,7 @@ graph LR; MergeRequestLastBuildFinished --> MergeRequestLabelRemoved; MergeRequestLabelAdded --> MergeRequestLabelAdded; MergeRequestLabelAdded --> MergeRequestLabelRemoved; + MergeRequestLabelAdded --> MergeRequestMerged; MergeRequestLabelRemoved --> MergeRequestLabelAdded; MergeRequestLabelRemoved --> MergeRequestLabelRemoved; ``` diff --git a/doc/development/wikis.md b/doc/development/wikis.md index 2f97931f924..c9ca181575b 100644 --- a/doc/development/wikis.md +++ b/doc/development/wikis.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: "GitLab's development guidelines for Wikis" --- @@ -30,7 +30,7 @@ Some notable gems that are used for wikis are: | Component | Description | Gem name | GitLab project | Upstream project | |:--------------|:-----------------------------------------------|:-------------------------------|:--------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------| | `gitlab` | Markup renderer, depends on various other gems | `gitlab-markup` | [`gitlab-org/gitlab-markup`](https://gitlab.com/gitlab-org/gitlab-markup) | [`github/markup`](https://github.com/github/markup) | -| `gitaly-ruby` | Main Gollum library | `gitlab-gollum-lib` | [`gitlab-org/gollum-lib`](https://gitlab.com/gitlab-org/gollum-lib) | [`gollum/gollum-lib`](https://github.com/gollum/gollum-lib) | +| `gollum-lib` | Main Gollum library | `gitlab-gollum-lib` | [`gitlab-org/gollum-lib`](https://gitlab.com/gitlab-org/gollum-lib) | [`gollum/gollum-lib`](https://github.com/gollum/gollum-lib) | | | Gollum Git adapter for Rugged | `gitlab-gollum-rugged_adapter` | [`gitlab-org/gitlab-gollum-rugged_adapter`](https://gitlab.com/gitlab-org/gitlab-gollum-rugged_adapter) | [`gollum/rugged_adapter`](https://github.com/gollum/rugged_adapter) | | | Rugged (also used in Gitaly itself) | `rugged` | - | [`libgit2/rugged`](https://github.com/libgit2/rugged) | diff --git a/doc/development/work_items_widgets.md b/doc/development/work_items_widgets.md index 5b9602595bb..bafceccdafe 100644 --- a/doc/development/work_items_widgets.md +++ b/doc/development/work_items_widgets.md @@ -137,3 +137,27 @@ Each record in the table defines mapping of a widget to a work item type. Curren | 4 | 1 | 1 | 1 | 0 | MyWeight | false | | 5 | 2 | 0 | 1 | 1 | Other Weight | false | | 6 | 3 | 0 | 1 | 1 | Weight | true | + +## Backend architecture + +You can update widgets using custom fine-grained mutations (for example, `WorkItemCreateFromTask`) or as part of the +`workItemCreate` or `workItemUpdate` mutations. + +### Widget callbacks + +When updating the widget together with the work item's mutation, backend code should be implemented using +callback classes that inherit from `Issuable::Callbacks::Base`. These classes have callback methods +that are named similar to ActiveRecord callbacks and behave similarly. + +Callback classes with the same name as the widget are automatically used. For example, `Issuable::Callbacks::Milestone` +is called when the work item has the milestone widget. To use a different class, you can override the `callback_class` +class method. + +#### Available callbacks + +- `after_initialize` is called after the work item is initialized by the `BuildService` and before + the work item is saved by the `CreateService` and `UpdateService`. This callback runs outside the + creation or update database transaction. +- `after_update_commit` is called after the DB update transaction is committed by the `UpdateService`. +- `after_save_commit` is called after the creation or DB update transaction is committed by the + `CreateService` or `UpdateService`. diff --git a/doc/development/workhorse/configuration.md b/doc/development/workhorse/configuration.md index 9fc106b8f04..e69f16c41f8 100644 --- a/doc/development/workhorse/configuration.md +++ b/doc/development/workhorse/configuration.md @@ -296,4 +296,4 @@ GITLAB_CONTINUOUS_PROFILING="stackdriver?service=workhorse&service_version=1.0.1 ## Related topics -- [LabKit monitoring documentation](https://gitlab.com/gitlab-org/labkit/-/blob/master/monitoring/doc.go). +- [LabKit monitoring documentation](https://gitlab.com/gitlab-org/labkit/-/blob/master/monitoring/doc.go) diff --git a/doc/development/workhorse/index.md b/doc/development/workhorse/index.md index 91795336f78..680dd0c205b 100644 --- a/doc/development/workhorse/index.md +++ b/doc/development/workhorse/index.md @@ -15,9 +15,14 @@ Workhorse itself is not a feature, but there are The canonical source for Workhorse is [`gitlab-org/gitlab/workhorse`](https://gitlab.com/gitlab-org/gitlab/tree/master/workhorse). -Prior to [epic #4826](https://gitlab.com/groups/gitlab-org/-/epics/4826), it was -[`gitlab-org/gitlab-workhorse`](https://gitlab.com/gitlab-org/gitlab-workhorse/tree/master), -but that repository is no longer used for development. + +## Learning Resources + +- Workhorse documentation (this page) +- [GitLab Workhorse Deep Dive: Dependency Proxy](https://www.youtube.com/watch?v=9cRd-k0TRqI) _video_ +- [How Dependency Proxy via Workhorse works](https://gitlab.com/gitlab-org/gitlab/-/issues/370235) +- [Workhorse overview for the Dependency Proxy](https://www.youtube.com/watch?v=WmBibT9oQms) +- [Workhorse architecture discussion](https://www.youtube.com/watch?v=QlHdh-yudtw) ## Install Workhorse diff --git a/doc/gitlab-basics/feature_branch_workflow.md b/doc/gitlab-basics/feature_branch_workflow.md index ce4aa0007c6..65445c226ca 100644 --- a/doc/gitlab-basics/feature_branch_workflow.md +++ b/doc/gitlab-basics/feature_branch_workflow.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/workflow.html' --- # Feature branch workflow **(FREE)** diff --git a/doc/gitlab-basics/start-using-git.md b/doc/gitlab-basics/start-using-git.md index 194647259b8..5fcaa68656f 100644 --- a/doc/gitlab-basics/start-using-git.md +++ b/doc/gitlab-basics/start-using-git.md @@ -103,7 +103,7 @@ To use the repository in the examples on this page: The project becomes available at `https://gitlab.com/<your-namespace>/sample-project/`. -You can [fork](../user/project/repository/forking_workflow.md#creating-a-fork) any project you have access to. +You can [fork](../user/project/repository/forking_workflow.md#create-a-fork) any project you have access to. ## Clone a repository diff --git a/doc/index.md b/doc/index.md index 49684705686..852ecccdd37 100644 --- a/doc/index.md +++ b/doc/index.md @@ -2,7 +2,6 @@ stage: none group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false description: 'Learn how to use and administer GitLab, the most scalable Git-based fully integrated platform for software development.' --- @@ -75,7 +74,7 @@ If you are coming to GitLab from another platform, the following information is | Topic | Description | |:----------------------------------------------------|:------------| | [Importing to GitLab](user/project/import/index.md) | Import your projects from GitHub, Bitbucket, GitLab.com, FogBugz, and SVN into GitLab. | -| [Migrating from SVN](user/project/import/svn.md) | Convert a SVN repository to Git and GitLab. | +| [Migrating from SVN](user/project/import/index.md#import-from-subversion) | Convert a SVN repository to Git and GitLab. | ## Build an integration with GitLab diff --git a/doc/install/aws/gitlab_hybrid_on_aws.md b/doc/install/aws/gitlab_hybrid_on_aws.md index b8c840782b1..e4eb0117410 100644 --- a/doc/install/aws/gitlab_hybrid_on_aws.md +++ b/doc/install/aws/gitlab_hybrid_on_aws.md @@ -48,7 +48,7 @@ The Beta version deploys Aurora PostgreSQL, but the release version will deploy | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | | Overview and Vision | [AWS Quick Start](https://aws.amazon.com/solutions/implementations/amazon-eks/) | [GitLab Environment Toolkit](https://gitlab.com/gitlab-org/gitlab-environment-toolkit/-/blob/main/README.md) | | Licensing | [Open Source (Apache 2.0)](https://github.com/aws-quickstart/quickstart-eks-gitlab/blob/main/LICENSE.txt) | [GitLab Enterprise Edition license](https://gitlab.com/gitlab-org/gitlab-environment-toolkit/-/blob/main/LICENSE) ([GitLab Premium tier](https://gitlab.com/gitlab-org/gitlab-environment-toolkit/-/blob/main/README.md)) | -| GitLab Support | [GitLab Beta Support](../../policy/alpha-beta-support.md#beta-features) | [GitLab GA Support](../../policy/alpha-beta-support.md#generally-available-ga) | +| GitLab Support | [GitLab Beta Support](../../policy/alpha-beta-support.md#beta) | [GitLab GA Support](../../policy/alpha-beta-support.md#generally-available-ga) | | GitLab Reference Architecture Compliant | Yes | Yes | | GitLab Performance Tool (GPT) Tested | Yes | Yes | | Amazon Well Architected Compliant | Yes<br />(via Quick Start program) | Critical portions <br />reviewed by AWS | diff --git a/doc/install/aws/gitlab_sre_for_aws.md b/doc/install/aws/gitlab_sre_for_aws.md index e68aea00b36..f957dfa8a65 100644 --- a/doc/install/aws/gitlab_sre_for_aws.md +++ b/doc/install/aws/gitlab_sre_for_aws.md @@ -2,7 +2,6 @@ stage: Systems group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false description: Doing SRE for GitLab instances and runners on AWS. --- diff --git a/doc/install/aws/index.md b/doc/install/aws/index.md index 8f540691a45..4dcf2ce0927 100644 --- a/doc/install/aws/index.md +++ b/doc/install/aws/index.md @@ -2,7 +2,6 @@ stage: Systems group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false description: Read through the GitLab installation methods. type: index --- diff --git a/doc/install/aws/manual_install_aws.md b/doc/install/aws/manual_install_aws.md index 51ae16ccd17..a59854ba6ed 100644 --- a/doc/install/aws/manual_install_aws.md +++ b/doc/install/aws/manual_install_aws.md @@ -845,6 +845,6 @@ If you see this page when trying to set a password via the web interface, make s ### Some job logs are not uploaded to object storage -When the GitLab deployment is scaled up to more than one node, some job logs may not be uploaded to [object storage](../../administration/object_storage.md) properly. [Incremental logging is required](../../administration/object_storage.md#other-alternatives-to-file-system-storage) for CI to use object storage. +When the GitLab deployment is scaled up to more than one node, some job logs may not be uploaded to [object storage](../../administration/object_storage.md) properly. [Incremental logging is required](../../administration/object_storage.md#alternatives-to-file-system-storage) for CI to use object storage. Enable [incremental logging](../../administration/job_logs.md#enable-or-disable-incremental-logging) if it has not already been enabled. diff --git a/doc/install/cloud_providers.md b/doc/install/cloud_providers.md new file mode 100644 index 00000000000..36cda77143f --- /dev/null +++ b/doc/install/cloud_providers.md @@ -0,0 +1,17 @@ +--- +stage: Systems +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +description: Install GitLab on a cloud provider. +type: index +--- + +# Installing GitLab on a cloud provider **(FREE SELF)** + +You can install GitLab on several cloud providers. + +| Cloud provider | Description | +|---------------------------------------------------------------|-------------| +| [AWS](aws/index.md) | Install GitLab on AWS using the community AMIs provided by GitLab. | +| [Google Cloud Platform (GCP)](google_cloud_platform/index.md) | Install GitLab on a VM in GCP. | +| [Azure](azure/index.md) | Install GitLab from Azure Marketplace. | diff --git a/doc/install/docker.md b/doc/install/docker.md index 40eb3a9796e..8d3c81d72c8 100644 --- a/doc/install/docker.md +++ b/doc/install/docker.md @@ -456,6 +456,30 @@ web browser under `<hostIP>:8929` and push using SSH under the port `2289`. A `docker-compose.yml` example that uses different ports can be found in the [Docker compose](#install-gitlab-using-docker-compose) section. +### Configure multiple database connections + +In GitLab 16.0, GitLab will default to using two database connections that point to the same PostgreSQL database. + +If you want to opt-in for this feature: + +1. Edit `/etc/gitlab/gitlab.rb` inside the container: + + ```shell + sudo docker exec -it gitlab editor /etc/gitlab/gitlab.rb + ``` + +1. Add the following line: + + ```ruby + gitlab_rails['databases']['ci']['enable'] = true + ``` + +1. Restart the container: + +```shell +sudo docker restart gitlab +``` + ## Upgrade In most cases, upgrading GitLab is as easy as downloading the newest Docker diff --git a/doc/install/index.md b/doc/install/index.md index 4aef471ad5c..15556117b51 100644 --- a/doc/install/index.md +++ b/doc/install/index.md @@ -2,7 +2,6 @@ stage: Systems group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false description: Read through the GitLab installation methods. type: index --- @@ -11,10 +10,15 @@ type: index You can install GitLab on most GNU/Linux distributions, on several cloud providers, and in Kubernetes clusters. - To get the best experience, you should balance performance, reliability, ease of administration (backups, upgrades, and troubleshooting) with the cost of hosting. -To get started, [choose your installation method](install_methods.md). - -If you already have a running instance, learn how to [configure it](next_steps.md). +- [Requirements](requirements.md) +- [Installation methods](install_methods.md) +- [Cloud provider guides](cloud_providers.md) +- [Offline GitLab](../topics/offline/index.md) +- [Reference architectures](../administration/reference_architectures/index.md) +- [Steps after installing](next_steps.md) +- [Upgrade GitLab](../update/index.md) +- [Install GitLab Runner](https://docs.gitlab.com/runner/install/) +- [Configure GitLab Runner](https://docs.gitlab.com/runner/configuration/) diff --git a/doc/install/install_methods.md b/doc/install/install_methods.md index af15dc3f085..0463c926286 100644 --- a/doc/install/install_methods.md +++ b/doc/install/install_methods.md @@ -2,14 +2,14 @@ stage: Systems group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false description: Read through the GitLab installation methods. type: index --- -# Installation methods +# Installation methods **(FREE SELF)** -You can install GitLab by using any of the following methods. +You can install GitLab on several [cloud providers](cloud_providers.md), +or use one of the following methods. | Installation method | Description | When to choose | |----------------------------------------------------------------|-------------|----------------| @@ -20,16 +20,6 @@ You can install GitLab by using any of the following methods. | [GitLab Environment Toolkit (GET)](https://gitlab.com/gitlab-org/gitlab-environment-toolkit#documentation) | A set of automation tools. | Use to deploy a [reference architecture](../administration/reference_architectures/index.md) on most major cloud providers. Has some [limitations](https://gitlab.com/gitlab-org/gitlab-environment-toolkit#missing-features-to-be-aware-of) and manual setup for production environments. | | [GitLab Operator](https://docs.gitlab.com/operator/) | An installation and management method that follows the [Kubernetes Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/). | Use to run GitLab in an [OpenShift](openshift_and_gitlab/index.md) environment. | -## Cloud providers - -You can install GitLab on several cloud providers. - -| Cloud provider | Description | -|---------------------------------------------------------------|-------------| -| [AWS](aws/index.md) | Install GitLab on AWS using the community AMIs provided by GitLab. | -| [Google Cloud Platform (GCP)](google_cloud_platform/index.md) | Install GitLab on a VM in GCP. | -| [Azure](azure/index.md) | Install GitLab from Azure Marketplace. | - ## Unsupported Linux distributions and Unix-like operating systems - Arch Linux diff --git a/doc/install/installation.md b/doc/install/installation.md index 1cd6a4fc5d2..888f28b8318 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -218,18 +218,7 @@ below to use a system Ruby. Linux distributions generally have older versions of Ruby available, so these instructions are designed to install Ruby from the official source code. -Download Ruby and compile it: - -```shell -mkdir /tmp/ruby && cd /tmp/ruby -curl --remote-name --location --progress-bar "https://cache.ruby-lang.org/pub/ruby/3.0/ruby-3.0.5.tar.gz" -echo '9afc6380a027a4fe1ae1a3e2eccb6b497b9c5ac0631c12ca56f9b7beb4848776 ruby-3.0.5.tar.gz' | sha256sum -c - && tar xzf ruby-3.0.5.tar.gz -cd ruby-2.7.6 - -./configure --disable-install-rdoc --enable-shared -make -sudo make install -``` +[Install Ruby](https://www.ruby-lang.org/en/documentation/installation/). ## 3. Go diff --git a/doc/install/openshift_and_gitlab/index.md b/doc/install/openshift_and_gitlab/index.md index 6086716be1c..330832ec2ce 100644 --- a/doc/install/openshift_and_gitlab/index.md +++ b/doc/install/openshift_and_gitlab/index.md @@ -31,7 +31,7 @@ The GitLab Operator does not include the GitLab Runner. To install and manage a ### Secure -- [License Compliance](../../user/compliance/license_compliance/index.md) +- [License Compliance via the `License-Scanning.gitlab-ci.yml` CI/CD template](../../user/compliance/license_compliance/index.md). Note that [license scanning of CycloneDX files](../../user/compliance/license_scanning_of_cyclonedx_files/index.md) is supported on OpenShift. - [Code Quality scanning](../../ci/testing/code_quality.md) - [Operational Container Scanning](../../user/clusters/agent/vulnerabilities.md) (Note: Pipeline [Container Scanning](../../user/application_security/container_scanning/index.md) is supported) diff --git a/doc/integration/advanced_search/elasticsearch.md b/doc/integration/advanced_search/elasticsearch.md index 389c79a281c..2cffad0b0e0 100644 --- a/doc/integration/advanced_search/elasticsearch.md +++ b/doc/integration/advanced_search/elasticsearch.md @@ -418,7 +418,7 @@ To disable the Elasticsearch integration: 1. Expand **Advanced Search**. 1. Clear the **Pause Elasticsearch indexing** checkbox. -## Zero downtime reindexing +## Zero-downtime reindexing The idea behind this reindexing method is to leverage the [Elasticsearch reindex API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html) and Elasticsearch index alias feature to perform the operation. We set up an index alias which connects to a @@ -427,6 +427,10 @@ the writes to the `primary` index. Then, we create another index and invoke the index data onto the new index. After the reindexing job is complete, we switch to the new index by connecting the index alias to it which becomes the new `primary` index. At the end, we resume the writes and typical operation resumes. +### Using zero-downtime reindexing + +You can use zero-downtime reindexing to configure index settings or mappings that cannot be changed without creating a new index and copying existing data. You should not use zero-downtime reindexing to fix missing data. Zero-downtime reindexing does not add data to the search cluster if the data is not already indexed. You must complete all [advanced search migrations](#advanced-search-migrations) before you start reindexing. + ### Trigger the reindex via the advanced search administration > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34069) in GitLab 13.2. @@ -662,7 +666,7 @@ For basic guidance on choosing a cluster configuration you may refer to [Elastic - Generally, you want to use at least a 2-node cluster configuration with one replica, which allows you to have resilience. If your storage usage is growing quickly, you may want to plan horizontal scaling (adding more nodes) beforehand. - It's not recommended to use HDD storage with the search cluster, because it takes a hit on performance. It's better to use SSD storage (NVMe or SATA SSD drives for example). -- You should not use [coordinating-only nodes](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#coordinating-only-node) with large instances. Coordinating-only nodes are smaller than [data nodes](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#data-node), which can impact performance and advanced search migrations. +- You should not use [coordinating-only nodes](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#coordinating-only-node) with large instances. Coordinating-only nodes are smaller than [data nodes](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#data-node), which can impact performance and [advanced search migrations](#advanced-search-migrations). - You can use the [GitLab Performance Tool](https://gitlab.com/gitlab-org/quality/performance) to benchmark search performance with different search cluster sizes and configurations. - `Heap size` should be set to no more than 50% of your physical RAM. Additionally, it shouldn't be set to more than the threshold for zero-based compressed oops. The exact threshold varies, but 26 GB is safe on most systems, but can also be as large as 30 GB on some systems. See [Heap size settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#heap-size-settings) and [Setting JVM options](https://www.elastic.co/guide/en/elasticsearch/reference/current/jvm-options.html) for more details. - Number of CPUs (CPU cores) per node usually corresponds to the `Number of Elasticsearch shards` setting described below. diff --git a/doc/integration/alicloud.md b/doc/integration/alicloud.md index d861d32e96a..4270444f0bb 100644 --- a/doc/integration/alicloud.md +++ b/doc/integration/alicloud.md @@ -59,7 +59,7 @@ Sign in to the AliCloud platform and create an application on it. AliCloud gener sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `alicloud` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/integration/arkose.md b/doc/integration/arkose.md index ae90f1f9574..8f6cec0ac0a 100644 --- a/doc/integration/arkose.md +++ b/doc/integration/arkose.md @@ -156,5 +156,5 @@ The [Anti-abuse team](https://about.gitlab.com/handbook/engineering/development/ ArkoseLabs also maintains the following resources: - [ArkoseLabs portal](https://portal.arkoselabs.com/) -- [ArkoseLabs Zendesk](https://support.arkoselabs.com/) +- [ArkoseLabs Zendesk](https://support.arkoselabs.com/hc/en-us) - [ArkoseLabs documentation](https://developer.arkoselabs.com/docs/documentation-guide) diff --git a/doc/integration/auth0.md b/doc/integration/auth0.md index ad20057f452..07a750143d6 100644 --- a/doc/integration/auth0.md +++ b/doc/integration/auth0.md @@ -42,7 +42,7 @@ application. sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `auth0` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/integration/azure.md b/doc/integration/azure.md index cc479dbf65d..9e9ce536591 100644 --- a/doc/integration/azure.md +++ b/doc/integration/azure.md @@ -68,7 +68,7 @@ Alternatively, add the `User.Read.All` application permission. sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `azure_oauth2` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/integration/cas.md b/doc/integration/cas.md index 750c9aeb8a4..9cbd8cf03d5 100644 --- a/doc/integration/cas.md +++ b/doc/integration/cas.md @@ -32,7 +32,7 @@ configure CAS for back-channel logout. sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `cas3` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/integration/datadog.md b/doc/integration/datadog.md index 2c86b7a8ed3..041ea80ccb8 100644 --- a/doc/integration/datadog.md +++ b/doc/integration/datadog.md @@ -53,4 +53,4 @@ section of your Datadog account. ## Related topics -- [Datadog CI Visibility](https://docs.datadoghq.com/continuous_integration/) documentation. +- [Datadog CI Visibility documentation](https://docs.datadoghq.com/continuous_integration/) diff --git a/doc/integration/ding_talk.md b/doc/integration/ding_talk.md index cb7ba6849b9..97ffab146a0 100644 --- a/doc/integration/ding_talk.md +++ b/doc/integration/ding_talk.md @@ -52,7 +52,7 @@ Sign in to DingTalk Open Platform and create an application on it. DingTalk gene sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `dingtalk` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/integration/facebook.md b/doc/integration/facebook.md index b35e3c585e0..aeea798715f 100644 --- a/doc/integration/facebook.md +++ b/doc/integration/facebook.md @@ -72,7 +72,7 @@ Facebook. Facebook generates an app ID and secret key for you to use. sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `facebook` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/integration/github.md b/doc/integration/github.md index 7b8927054ee..00303754d85 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -35,7 +35,7 @@ your website could enable the covert redirect attack. ## Enable GitHub OAuth in GitLab -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `github` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md index 168d55b0fa5..59f122a5110 100644 --- a/doc/integration/gitlab.md +++ b/doc/integration/gitlab.md @@ -51,7 +51,7 @@ GitLab.com generates an application ID and secret key for you to use. sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `gitlab` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. Add the provider configuration: diff --git a/doc/integration/gitpod.md b/doc/integration/gitpod.md index 6df36cff638..0ba227c2a85 100644 --- a/doc/integration/gitpod.md +++ b/doc/integration/gitpod.md @@ -1,7 +1,7 @@ --- type: reference, how-to stage: Create -group: Editor +group: IDE info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- diff --git a/doc/integration/glab/index.md b/doc/integration/glab/index.md index 427751a0297..aa7032412e7 100644 --- a/doc/integration/glab/index.md +++ b/doc/integration/glab/index.md @@ -81,8 +81,7 @@ to send us feedback. - [Install the CLI](https://gitlab.com/gitlab-org/cli/-/blob/main/README.md#installation) - [Documentation](https://gitlab.com/gitlab-org/cli/-/tree/main/docs/source) -- The extension source code is available in the - [`cli`](https://gitlab.com/gitlab-org/cli/) project. +- Extension source code in the [`cli`](https://gitlab.com/gitlab-org/cli/) project ## Troubleshooting diff --git a/doc/integration/google.md b/doc/integration/google.md index bb76d7c8ff1..d60c1b43ed6 100644 --- a/doc/integration/google.md +++ b/doc/integration/google.md @@ -71,7 +71,7 @@ On your GitLab server: sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `google_oauth2` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. Add the provider configuration: diff --git a/doc/integration/index.md b/doc/integration/index.md index 02860339b6f..6be17640c6c 100644 --- a/doc/integration/index.md +++ b/doc/integration/index.md @@ -2,7 +2,6 @@ stage: Manage group: Integrations info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Integrate with GitLab **(FREE)** diff --git a/doc/integration/jenkins.md b/doc/integration/jenkins.md index 668038ef386..d24cdc983bc 100644 --- a/doc/integration/jenkins.md +++ b/doc/integration/jenkins.md @@ -190,26 +190,21 @@ This issue can occur when the request exceeds the [webhook timeout](../user/project/integrations/webhooks.md#webhook-fails-or-multiple-webhook-requests-are-triggered), which is set to 10 seconds by default. -Check the [service hook logs](../user/project/integrations/index.md#troubleshooting-integrations) -for request failures or check the `/var/log/gitlab/gitlab-rails/production.log` -file for messages like: +For this issue, check: -```plaintext -WebHook Error => Net::ReadTimeout -``` +- [Integration webhook logs](../user/project/integrations/index.md#troubleshooting) +for request failures. +- `/var/log/gitlab/gitlab-rails/production.log` for messages like: -or + ```plaintext + WebHook Error => Net::ReadTimeout + ``` -```plaintext -WebHook Error => execution expired -``` + or -Or check for duplicate messages in `/var/log/gitlab/gitlab-rail`, like: - -```plaintext -2019-10-25_04:22:41.25630 2019-10-25T04:22:41.256Z 1584 TID-ovowh4tek WebHookWorker JID-941fb7f40b69dff3d833c99b INFO: start -2019-10-25_04:22:41.25630 2019-10-25T04:22:41.256Z 1584 TID-ovowh4tek WebHookWorker JID-941fb7f40b69dff3d833c99b INFO: start -``` + ```plaintext + WebHook Error => execution expired + ``` On self-managed GitLab instances, you can fix this issue by [increasing the webhook timeout value](../administration/instance_limits.md#webhook-timeout). diff --git a/doc/integration/jira/configure.md b/doc/integration/jira/configure.md index 3ef4dfac3f4..f1f7a34ac94 100644 --- a/doc/integration/jira/configure.md +++ b/doc/integration/jira/configure.md @@ -4,12 +4,11 @@ group: Integrations info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Configure Jira **(FREE)** +# Jira integration **(FREE)** -You can set up the [Jira integration](index.md#jira-integration) -by configuring your project settings in GitLab. -You can also configure these settings at a [group level](../../user/admin_area/settings/project_integration_management.md#manage-group-level-default-settings-for-a-project-integration), -and for self-managed GitLab, at an [instance level](../../user/admin_area/settings/project_integration_management.md#manage-instance-level-default-settings-for-a-project-integration). +The Jira integration connects one or more GitLab projects to a Jira instance. You can host the Jira instance yourself or in [Atlassian Cloud](https://www.atlassian.com/migration/assess/why-cloud). The supported Jira versions are `6.x`, `7.x`, `8.x`, and `9.x`. + +## Configure the integration Prerequisites: @@ -20,7 +19,13 @@ Prerequisites: and the email address you used to create the token. See [authentication in Jira](index.md#authentication-in-jira). -To configure your project: +You can enable the Jira integration by configuring your project settings in GitLab. +You can also configure these settings at the: + +- [Group level](../../user/admin_area/settings/project_integration_management.md#manage-group-level-default-settings-for-a-project-integration) +- [Instance level](../../user/admin_area/settings/project_integration_management.md#manage-instance-level-default-settings-for-a-project-integration) for self-managed GitLab + +To configure your project settings in GitLab: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. @@ -59,6 +64,28 @@ To configure your project: Your GitLab project can now interact with all Jira projects in your instance and the project now displays a Jira link that opens the Jira project. +## Create a Jira Cloud API token + +To [integrate with Jira](index.md) in Atlassian Cloud, you must create an API token. + +To create a Jira Cloud API token: + +1. Sign in to [Atlassian](https://id.atlassian.com/manage-profile/security/api-tokens) + from an account with **write** access to Jira projects. + + The link opens the **API tokens** page. Alternatively, from your Atlassian + profile, select **Account Settings > Security > Create and manage API tokens**. + +1. Select **Create API token**. +1. In the dialog, enter a label for your token and select **Create**. + +To copy the API token, select **Copy** and paste the token somewhere safe. + +To configure GitLab, you need: + +- The newly created token +- The email address you used when you created the token + ## Migrate from Jira Server to Jira Cloud in GitLab To migrate from Jira Server to Jira Cloud in GitLab and maintain your Jira integration: @@ -68,7 +95,9 @@ To migrate from Jira Server to Jira Cloud in GitLab and maintain your Jira integ 1. Select **Jira**. 1. In **Web URL**, enter the new Jira site URL (for example, `https://myjirasite.atlassian.net`). 1. In **Username or Email**, enter the username or email registered on your Jira profile. -1. [Create an API token](jira_cloud_configuration.md), and copy that value. +1. [Create an API token](#create-a-jira-cloud-api-token), and copy that value. 1. In **Password or API token**, paste the API token value. 1. Optional. Select **Test settings** to check if the connection is working. 1. Select **Save changes**. + +To update existing Jira issue references in GitLab to use the new Jira site URL, you must [invalidate the Markdown cache](../../administration/invalidate_markdown_cache.md#invalidate-the-cache). diff --git a/doc/integration/jira/connect-app.md b/doc/integration/jira/connect-app.md index 8bbac021849..fa7d48ba44d 100644 --- a/doc/integration/jira/connect-app.md +++ b/doc/integration/jira/connect-app.md @@ -6,64 +6,47 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab for Jira Cloud app **(FREE)** -You can integrate GitLab and Jira Cloud using the -[GitLab for Jira Cloud](https://marketplace.atlassian.com/apps/1221011/gitlab-com-for-jira-cloud) -app in the Atlassian Marketplace. - -Only Jira users with administrator access can install or configure -the GitLab for Jira Cloud app. - -## Install the GitLab for Jira Cloud app **(FREE SAAS)** +With the [GitLab for Jira Cloud](https://marketplace.atlassian.com/apps/1221011/gitlab-com-for-jira-cloud) app, +you can integrate GitLab and Jira Cloud. If you use GitLab.com and Jira Cloud, you can install the GitLab for Jira Cloud app. -If you do not use both of these environments, use the [Jira DVCS Connector](dvcs/index.md) or -[install the GitLab for Jira Cloud app manually](#install-the-gitlab-for-jira-cloud-app-manually). -We recommend the GitLab for Jira Cloud app, because data is -synchronized in real time. The DVCS connector updates data only once per hour. - -To configure the GitLab for Jira Cloud app, you must have -at least the Maintainer role in the GitLab.com namespace. +If you do not use both of these environments, use the [Jira DVCS Connector](dvcs/index.md) instead +or [install the GitLab for Jira Cloud app manually](#install-the-gitlab-for-jira-cloud-app-manually). +You should use the GitLab for Jira Cloud app because data is synchronized +in real time. The Jira DVCS connector updates data only once per hour. This integration method supports [Smart Commits](dvcs/index.md#smart-commits). -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see -[Configure the GitLab for Jira Cloud app from the Atlassian Marketplace](https://youtu.be/SwR-g1s1zTo). - -To install the GitLab for Jira Cloud app: +## Install the GitLab for Jira Cloud app **(FREE SAAS)** -1. In Jira, go to **Jira Settings > Apps > Find new apps**, then search for GitLab. -1. Select **GitLab for Jira Cloud**, then select **Get it now**, or go to the - [App in the marketplace directly](https://marketplace.atlassian.com/apps/1221011/gitlab-com-for-jira-cloud). +Prerequisites: -  -1. After installing, to go to the configurations page, select **Get started**. - This page is always available under **Jira Settings > Apps > Manage apps**. +- You must have at least the Maintainer role for the GitLab namespace. +- You must have administrator access to the Jira instance. -  -1. To add namespaces, ensure you're signed in to GitLab.com - as a user with at least the Maintainer role. +To install the GitLab for Jira Cloud app: -  -1. To open the list of available namespaces, select **Add namespace**. +1. In Jira, select **Jira Settings > Apps > Find new apps**, and search for GitLab. +1. Select **GitLab for Jira Cloud**, and select **Get it now**. -1. Identify the namespace you want to link, and select **Link**. - - You must have at least the Maintainer role for the namespace. - - Only Jira site administrators can add or remove namespaces for an installation. + Alternatively, [get the app directly from the Atlassian Marketplace](https://marketplace.atlassian.com/apps/1221011/gitlab-com-for-jira-cloud). -  +1. To go to the configurations page, select **Get started**. + You can always access this page in **Jira Settings > Apps > Manage apps**. +1. To open the list of available namespaces, select **Add namespace**. +1. To link to a namespace, select **Link**. -NOTE: -The GitLab.com user only needs access when adding a new namespace. For syncing with -Jira, we do not depend on the user's token. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see +[Configure the GitLab for Jira Cloud app from the Atlassian Marketplace](https://youtu.be/SwR-g1s1zTo). After a namespace is added: -- All future commits, branches, and merge requests of all projects under that namespace +- All future commits, branches, and merge requests of all projects in that namespace are synced to Jira. -- From GitLab 13.8, past merge request data is synced to Jira. +- In GitLab 13.8 and later, existing merge request data is synced to Jira. -Support for syncing past branch and commit data is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/263240). +For more information about syncing existing branch and commit data, see [issue 263240](https://gitlab.com/gitlab-org/gitlab/-/issues/263240). ## Update the GitLab for Jira Cloud app @@ -73,15 +56,15 @@ for details. If the app requires additional permissions, [the update must first be manually approved in Jira](https://developer.atlassian.com/platform/marketplace/upgrading-and-versioning-cloud-apps/#changes-that-require-manual-customer-approval). -## Set up OAuth authentication +## Set up OAuth authentication for self-managed instances **(FREE SELF)** The GitLab for Jira Cloud app is [switching to OAuth authentication](https://gitlab.com/gitlab-org/gitlab/-/issues/387299). To enable OAuth authentication, you must create an OAuth application on the GitLab instance. -Enabling OAuth authentication is: +You must enable OAuth authentication to: -- Required to [connect the GitLab for Jira Cloud app for self-managed instances](#connect-the-gitlab-for-jira-cloud-app-for-self-managed-instances). -- Recommended to [install the GitLab for Jira Cloud app manually](#install-the-gitlab-for-jira-cloud-app-manually). +- [Connect the GitLab for Jira Cloud app for self-managed instances](#connect-the-gitlab-for-jira-cloud-app-for-self-managed-instances). +- [Install the GitLab for Jira Cloud app manually](#install-the-gitlab-for-jira-cloud-app-manually). To create an OAuth application: @@ -105,25 +88,29 @@ To create an OAuth application: > Introduced in GitLab 15.7. -Prerequisites: - -- GitLab.com must serve as a proxy for the instance. -- The instance must be publicly available. -- The instance must be on version 15.7 or later. - You can link self-managed instances after installing the GitLab for Jira Cloud app from the marketplace. Jira apps can only link to one URL per marketplace listing. The official listing links to GitLab.com. -If your instance doesn't meet the prerequisites or you don't want to use the official marketplace listing, you can +NOTE: +With this method, GitLab.com serves as a proxy for Jira traffic from your instance. + +If your instance doesn't meet the [prerequisites](#prerequisites) or you don't want to use the official marketplace listing, you can [install the app manually](#install-the-gitlab-for-jira-cloud-app-manually). It's not possible to create branches from Jira for self-managed instances. For more information, see [issue 391432](https://gitlab.com/gitlab-org/gitlab/-/issues/391432). +### Prerequisites + +- The instance must be publicly available. +- The instance must be on GitLab version 15.7 or later. +- You must set up [OAuth authentication](#set-up-oauth-authentication-for-self-managed-instances). + ### Set up your instance +See [prerequisites](#prerequisites). + To set up your self-managed instance for the GitLab for Jira Cloud app in GitLab 15.7 and later: -1. [Set up OAuth authentication](#set-up-oauth-authentication). 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General** (`/admin/application_settings/general`). 1. Expand the **GitLab for Jira App** section. @@ -132,6 +119,8 @@ To set up your self-managed instance for the GitLab for Jira Cloud app in GitLab ### Link your instance +See [prerequisites](#prerequisites). + To link your self-managed instance to the GitLab for Jira Cloud app: 1. Install the [GitLab for Jira Cloud app](https://marketplace.atlassian.com/apps/1221011/gitlab-com-for-jira-cloud?tab=overview&hosting=cloud). @@ -144,13 +133,6 @@ To link your self-managed instance to the GitLab for Jira Cloud app: If your GitLab instance is self-managed and you don't want to use the official marketplace listing, you can install the app manually. -Prerequisites: - -- The instance must be publicly available. -- You must set up [OAuth authentication](#set-up-oauth-authentication). - -### Set up your Jira app - Each Jira Cloud application must be installed from a single location. Jira fetches a [manifest file](https://developer.atlassian.com/cloud/jira/platform/connect-app-descriptor/) from the location you provide. The manifest file describes the application to the system. To support @@ -159,30 +141,30 @@ self-managed GitLab instances with Jira Cloud, you can do one of the following: - [Install the application in development mode](#install-the-application-in-development-mode). - [Create a Marketplace listing](#create-a-marketplace-listing). -#### Install the application in development mode +### Prerequisites + +- The instance must be publicly available. +- You must set up [OAuth authentication](#set-up-oauth-authentication-for-self-managed-instances). + +### Install the application in development mode -You can configure your Atlassian Cloud instance to allow you to install applications -from outside the Marketplace, which allows you to install the application: +See [prerequisites](#prerequisites-1). + +To configure your Atlassian Cloud instance so you can install applications +from outside the Marketplace: 1. Sign in to your Jira instance as an administrator. 1. Place your Jira instance into [development mode](https://developer.atlassian.com/cloud/jira/platform/getting-started-with-connect/#step-2--enable-development-mode). 1. Sign in to your GitLab application as a user with administrator access. -1. Install the GitLab application from your Jira instance, as - described in the [Atlassian developer guides](https://developer.atlassian.com/cloud/jira/platform/getting-started-with-connect/#step-3--install-and-test-your-app): +1. Install the GitLab application from your Jira instance as + described in the [Atlassian developer guide](https://developer.atlassian.com/cloud/jira/platform/getting-started-with-connect/#step-3--install-and-test-your-app): 1. In your Jira instance, go to **Apps > Manage Apps** and select **Upload app**: - -  - - 1. For **App descriptor URL**, provide the full URL to your manifest file, based + 1. For **App descriptor URL**, provide the full URL to your manifest file based on your instance configuration. By default, your manifest file is located at `/-/jira_connect/app_descriptor.json`. For example, if your GitLab self-managed instance domain is `app.pet-store.cloud`, your manifest file is located at `https://app.pet-store.cloud/-/jira_connect/app_descriptor.json`. 1. Select **Upload**. Jira fetches the content of your `app_descriptor` file and installs it. - 1. If the upload is successful, Jira displays a modal panel: **Installed and ready to go!** - To configure the integration, select **Get started**. - -  - + 1. To configure the integration, select **Get started**. 1. Disable [development mode](https://developer.atlassian.com/cloud/jira/platform/getting-started-with-connect/#step-2--enable-development-mode) on your Jira instance. The **GitLab for Jira Cloud** app now displays under **Manage apps**. You can also @@ -192,19 +174,20 @@ NOTE: If a GitLab update makes changes to the application descriptor, you must uninstall, then reinstall the application. -#### Create a Marketplace listing +### Create a Marketplace listing -If you prefer to not use development mode on your Jira instance, you can create -your own Marketplace listing for your instance. This enables your application -to be installed from the Atlassian Marketplace. +If you don't want to use development mode on your Jira instance, you can create +your own Marketplace listing. This way, your application +can be installed from the Atlassian Marketplace. + +See [prerequisites](#prerequisites-1). -For full instructions, review the Atlassian [guide to creating a marketplace listing](https://developer.atlassian.com/platform/marketplace/installing-cloud-apps/#creating-the-marketplace-listing). To create a Marketplace listing: 1. Register as a Marketplace vendor. -1. List your application using the application descriptor URL. +1. List your application with the application descriptor URL. - Your manifest file is located at: `https://your.domain/your-path/-/jira_connect/app_descriptor.json` - - We recommend you list your application as `private`, because public + - You should list your application as `private` because public applications can be viewed and installed by any user. 1. Generate test license tokens for your application. @@ -212,11 +195,13 @@ NOTE: This method uses [automated updates](#update-the-gitlab-for-jira-cloud-app) the same way as our GitLab.com Marketplace listing. -## Configure your GitLab instance to serve as a proxy for the GitLab for Jira Cloud app +For more information about creating a Marketplace listing, see the [Atlassian documentation](https://developer.atlassian.com/platform/marketplace/installing-cloud-apps/#creating-the-marketplace-listing). + +## Configure your GitLab instance to serve as a proxy for the GitLab for Jira Cloud app **(FREE SELF)** -A GitLab instance can serve as a proxy for other GitLab instances using the GitLab for Jira Cloud app. -This can be useful if you are managing multiple GitLab instance but only want to [manually install](#install-the-gitlab-for-jira-cloud-app-manually) -the GitLab for Jira app once. +A GitLab instance can serve as a proxy for other GitLab instances through the GitLab for Jira Cloud app. +You might want to use a proxy if you're managing multiple GitLab instances but only want to +[manually install](#install-the-gitlab-for-jira-cloud-app-manually) the GitLab for Jira Cloud app once. To configure your GitLab instance to serve as a proxy: @@ -225,9 +210,36 @@ To configure your GitLab instance to serve as a proxy: 1. Expand the **GitLab for Jira App** section. 1. Select **Enable public key storage**. 1. Select **Save changes**. -1. [Install the GitLab for Jira Cloud app manually](#install-the-gitlab-for-jira-cloud-app-manually) +1. [Install the GitLab for Jira Cloud app manually](#install-the-gitlab-for-jira-cloud-app-manually). + +Other GitLab instances that use the proxy must configure the **Jira Connect Proxy URL** and the [OAuth application](#set-up-oauth-authentication-for-self-managed-instances) **Redirect URI** settings to point to the proxy instance. + +## Security considerations + +The GitLab for Jira Cloud app connects GitLab and Jira. Data must be shared between the two applications, and access must be granted in both directions. + +### Access to GitLab through OAuth **(FREE SELF)** + +GitLab does not share an access token with Jira. However, users must authenticate through OAuth to configure the app. + +An access token is retrieved through a [PKCE](https://www.rfc-editor.org/rfc/rfc7636) OAuth flow and stored only on the client side. +The app frontend that initializes the OAuth flow is a JavaScript application that's loaded from GitLab through an iframe on Jira. + +The OAuth application must have the `api` scope, which grants complete read and write access to the API. +This access includes all groups and projects, the container registry, and the package registry. +However, the GitLab for Jira Cloud app only uses this access to: + +- Display namespaces to be linked. +- Link namespaces. + +Access through OAuth is only needed for the time a user configures the GitLab for Jira Cloud app. For more information, see [Access token expiration](../oauth_provider.md#access-token-expiration). -Other GitLab instances using the proxy must configure the **Jira Connect Proxy URL** setting and the [OAuth application](#set-up-oauth-authentication) **Redirect URI** to point to the proxy instance. +### Access to Jira through access token + +Jira shares an access token with GitLab to authenticate and authorize data pushes to Jira. +As part of the app installation process, Jira sends a handshake request to GitLab containing the access token. +The handshake is signed with an [asymmetric JWT](https://developer.atlassian.com/cloud/jira/platform/understanding-jwt-for-connect-apps/), +and the access token is stored encrypted with `AES256-GCM` on GitLab. ## Troubleshooting @@ -243,7 +255,7 @@ You need to sign in or sign up before continuing. The GitLab for Jira Cloud app uses an iframe to add namespaces on the settings page. Some browsers block cross-site cookies, which can lead to this issue. -To resolve this issue, set up [OAuth authentication](#set-up-oauth-authentication) and enable the `jira_connect_oauth` [feature flag](../../administration/feature_flags.md). +To resolve this issue, set up [OAuth authentication](#set-up-oauth-authentication-for-self-managed-instances) and enable the `jira_connect_oauth` [feature flag](../../administration/feature_flags.md). ### Manual installation fails @@ -286,28 +298,33 @@ To resolve this issue on GitLab self-managed, follow one of the solutions below, - In all GitLab versions: - Re-install the GitLab for Jira Cloud app. This might remove all already synced development panel data. -## Security considerations - -The GitLab for Jira Cloud app connects GitLab and Jira, as data must be shared between the two applications and access must be granted in both directions. +### `Failed to update GitLab version` error when setting up the GitLab for Jira Cloud app for self-managed instances -## Access to GitLab through OAuth +When you set up the GitLab for Jira Cloud app, you might get the following message after you enter your +self-managed instance URL: -GitLab does not share an access token with Jira. However, users must authenticate via OAuth to configure the app. +```plaintext +Failed to update GitLab version. Please try again. +``` -An access token is retrieved via [PKCE](https://www.rfc-editor.org/rfc/rfc7636) OAuth flow, and stored only on the client side. -The app-frontend that initializes the OAuth flow is a JavaScript application, which is loaded from GitLab through an iframe on Jira. +To resolve this issue, ensure all prerequisites for your installation method have been met: -The OAuth application requires the `api` scope that grants complete read/write access to the API, including to all groups and projects, the container registry, and the package registry. -However, the GitLab for Jira Cloud app only uses this access to: +- [Prerequisites for connecting the GitLab for Jira Cloud app](#prerequisites) +- [Prerequisites for installing the GitLab for Jira Cloud app manually](#prerequisites-1) -- Display namespaces to be linked. -- Link namespaces. +If you're using GitLab 15.8 and earlier and have previously enabled both the `jira_connect_oauth_self_managed` +and the `jira_connect_oauth` feature flags, you must disable the `jira_connect_oauth_self_managed` flag +due to a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388943). To check for these flags: -Access through OAuth is only needed for the time a user configures the GitLab for Jira Cloud app. For more information, see [Access token expiration](../oauth_provider.md#access-token-expiration). +1. Open a [Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session). +1. Execute the following code: -## Access to Jira through access token + ```ruby + # Check if both feature flags are enabled. + # If the flags are enabled, these commands return `true`. + Feature.enabled?(:jira_connect_oauth) + Feature.enabled?(:jira_connect_oauth_self_managed) -Jira shares an access token with GitLab to authenticate and authorize data pushes to Jira. -As part of the app installation process, Jira sends a handshake request to GitLab containing the access token. -The handshake is signed with an [asymmetric JWT](https://developer.atlassian.com/cloud/jira/platform/understanding-jwt-for-connect-apps/) -and the access token is stored encrypted with `AES256-GCM` on GitLab. + # If both flags are enabled, disable the `jira_connect_oauth_self_managed` flag. + Feature.disable(:jira_connect_oauth_self_managed) + ``` diff --git a/doc/integration/jira/development_panel.md b/doc/integration/jira/development_panel.md index 57308c9a19e..9ae97c22919 100644 --- a/doc/integration/jira/development_panel.md +++ b/doc/integration/jira/development_panel.md @@ -8,7 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/233149) from GitLab Premium to GitLab Free in 13.4. -You can view GitLab activity from the Jira development panel. +The Jira development panel connects all GitLab projects in a group or personal namespace +where you can view GitLab activity. When you're in GitLab, you can refer to a Jira issue by ID. The [activity for that issue](https://support.atlassian.com/jira-software-cloud/docs/view-development-information-for-an-issue/) @@ -33,7 +34,7 @@ The information displayed in the Jira development panel depends on where you men | GitLab: where you mention the Jira issue ID | Jira development panel: what information is displayed | |------------------------------------------------|-------------------------------------------------------| -| Merge request title or description | Link to the merge request | +| Merge request title or description | Link to the merge request<br>[GitLab 15.11 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/354373): Link to the branch | | Branch name | Link to the branch | | Commit message | Link to the commit | | [Jira Smart Commit](#jira-smart-commits) | Custom comment, logged time, or workflow transition | diff --git a/doc/integration/jira/dvcs/index.md b/doc/integration/jira/dvcs/index.md index e07f0e579b2..759fea41ba5 100644 --- a/doc/integration/jira/dvcs/index.md +++ b/doc/integration/jira/dvcs/index.md @@ -6,44 +6,39 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Jira DVCS connector **(FREE)** +WARNING: +The Jira DVCS connector for Jira Cloud was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/362168) in GitLab 15.1 +and is planned for removal in 16.0. Use the [GitLab for Jira Cloud app](../connect-app.md) instead. + Use the Jira DVCS (distributed version control system) connector if you self-host -your Jira instance, and you want to sync information -between GitLab and Jira. If you use Jira Cloud, you should use the -[GitLab for Jira Cloud app](../connect-app.md) unless you specifically need the -DVCS connector. +your Jira instance and want to sync information between GitLab and Jira. +If you're using the Jira DVCS connector with Jira Cloud, [migrate to the GitLab for Jira Cloud app](#migrate-to-the-gitlab-for-jira-cloud-app). When you configure the Jira DVCS connector, make sure your GitLab and Jira instances are accessible. - **Self-managed GitLab**: Your GitLab instance must be accessible by Jira. - **Jira Server**: Your network must allow access to your instance. -- **Jira Cloud**: Your instance must be accessible through the internet. - -NOTE: -When using GitLab 15.0 and later (including GitLab.com) with Jira Server, you might experience a [session token bug in Jira](https://jira.atlassian.com/browse/JSWSERVER-21389). As a workaround, ensure Jira Server is version 9.1.0 and later or 8.20.11 and later. ## Smart Commits -When connecting GitLab with Jira with DVCS, you can process your Jira issues using -special commands, called -[Smart Commits](https://support.atlassian.com/jira-software-cloud/docs/process-issues-with-smart-commits/), -in your commit messages. With Smart Commits, you can: +When connecting GitLab and Jira with the Jira DVCS connector, you can process your Jira issues with +special commands called [Smart Commits](https://support.atlassian.com/jira-software-cloud/docs/process-issues-with-smart-commits/). +With Smart Commits, you can: - Comment on issues. - Record time-tracking information against issues. - Transition issues to any status defined in the Jira project's workflow. -Commands must be in the first line of the commit message. The -[Jira Software documentation](https://support.atlassian.com/jira-software-cloud/docs/process-issues-with-smart-commits/) -contains more information about how Smart Commits work, and what commands are available -for your use. +Commands must be in the first line of the commit message. For more information about how Smart Commits work and what commands are available +for use, see the [Atlassian documentation](https://support.atlassian.com/jira-software-cloud/docs/process-issues-with-smart-commits/). -For Smart Commits to work, the committing user on GitLab must have a corresponding -user on Jira with the same email address or username. +For Smart Commits to work, the GitLab user must have a corresponding +Jira user with the same email address or username. ### Smart Commit syntax -Smart Commits should follow the pattern of: +Smart Commits must follow this pattern: ```plaintext <ISSUE_KEY> <ignored text> #<command> <optional command parameters> @@ -55,7 +50,7 @@ Some examples: - Record time tracking: `KEY-123 #time 2w 4d 10h 52m Tracking work time.` - Close an issue: `KEY-123 #close Closing issue` -A Smart Commit message must not span more than one line (no carriage returns) but +A Smart Commit message must not span more than one line (no carriage returns), but you can still perform multiple actions in a single commit. For example: - Add time tracking, add a comment, and transition to **Closed**: @@ -72,64 +67,43 @@ you can still perform multiple actions in a single commit. For example: ## Configure a GitLab application for DVCS -For projects in a single group we recommend you create a [group application](../../oauth_provider.md#create-a-group-owned-application). -For projects across multiple groups we recommend you create and use a `jira` user in GitLab, and use the account -only for integration work. A separate account ensures regular account -maintenance does not affect your integration. If a `jira` user or group application is not feasible, -you can set up this integration as an [instance-wide application](../../oauth_provider.md#create-an-instance-wide-application) -or with a [user owned application](../../oauth_provider.md#create-a-user-owned-application) instead. - -1. Navigate to the [appropriate **Applications** section](../../oauth_provider.md). -1. In the **Name** field, enter a descriptive name for the integration, such as `Jira`. -1. In the **Redirect URI** field, enter the URI appropriate for your version of GitLab, - replacing `<gitlab.example.com>` with your GitLab instance domain: - - *For GitLab versions 13.0 and later* **and** *Jira versions 8.14 and later,* use the - generated `Redirect URL` from - [Linking GitLab accounts with Jira](https://confluence.atlassian.com/adminjiraserver/linking-gitlab-accounts-1027142272.html). - - *For GitLab versions 13.0 and later* **and** *Jira Cloud,* use `https://<gitlab.example.com>/login/oauth/callback`. - - *For GitLab versions 11.3 and later* **and** *Jira versions 8.13 and earlier,* use `https://<gitlab.example.com>/login/oauth/callback`. - If you use GitLab.com, the URL is `https://gitlab.com/login/oauth/callback`. - - *For GitLab versions 11.2 and earlier,* use - `https://<gitlab.example.com>/-/jira/login/oauth/callback`. - -1. For **Scopes**, select `api` and clear any other checkboxes. - - The DVCS connector requires a _write-enabled_ `api` scope to automatically create and manage required webhooks. +For projects in a single group, you should create a [group application](../../oauth_provider.md#create-a-group-owned-application). + +For projects across multiple groups, you should create a new user account in GitLab for Jira integration work only. +A separate account ensures regular account maintenance does not affect your integration. + +If a separate user account or group application is not possible, you can set up this integration +as an [instance-wide application](../../oauth_provider.md#create-an-instance-wide-application) +or with a [user-owned application](../../oauth_provider.md#create-a-user-owned-application). + +1. Go to the [appropriate **Applications** section](../../oauth_provider.md). +1. In the **Name** text box, enter a descriptive name for the integration (for example, `Jira`). +1. In the **Redirect URI** text box, enter the generated **Redirect URL** from + [linking GitLab accounts](https://confluence.atlassian.com/adminjiraserver/linking-gitlab-accounts-1027142272.html). +1. In **Scopes**, select `api` and clear any other checkboxes. + The DVCS connector requires a **write-enabled** `api` scope to automatically create and manage required webhooks. 1. Select **Submit**. 1. Copy the **Application ID** and **Secret** values. - You need them to configure Jira. + You need these values to configure Jira. ## Configure Jira for DVCS -Configure this connection when you want to import all GitLab commits and branches, -for the groups you specify, into Jira. This import takes a few minutes and, after +To import all GitLab commits and branches into Jira for the groups you specify, +configure Jira for DVCS. This import takes a few minutes and, after it completes, refreshes every 60 minutes: 1. Complete the [GitLab configuration](#configure-a-gitlab-application-for-dvcs). -1. Go to your DVCS accounts: - - *For Jira Server,* select **Settings (gear) > Applications > DVCS accounts**. - - *For Jira Cloud,* select **Settings (gear) > Products > DVCS accounts**. -1. To create a new integration, select the appropriate value for **Host**: - - *For Jira versions 8.14 and later:* Select **GitLab** or - **GitLab Self-Managed**. - - *For Jira Cloud or Jira versions 8.13 and earlier:* Select **GitHub Enterprise**. -1. For **Team or User Account**, enter either: - - *For Jira versions 8.14 and later:* - - The relative path of a top-level GitLab group that - [the GitLab user](#configure-a-gitlab-application-for-dvcs) has access to. - - *For Jira Cloud or Jira versions 8.13 and earlier:* - - The relative path of a top-level GitLab group that - [the GitLab user](#configure-a-gitlab-application-for-dvcs) has access to. - - The relative path of your personal namespace. - -1. In the **Host URL** field, enter the URI appropriate for your version of GitLab, - replacing `<gitlab.example.com>` with your GitLab instance domain: - - *For GitLab versions 11.3 and later,* use `https://<gitlab.example.com>`. - - *For GitLab versions 11.2 and earlier,* use - `https://<gitlab.example.com>/-/jira`. - -1. For **Client ID**, use the **Application ID** value from the previous section. -1. For **Client Secret**, use the **Secret** value from the previous section. -1. Ensure that the rest of the checkboxes are selected. +1. Go to your DVCS account: + - **For Jira Server**, select **Settings (gear) > Applications > DVCS accounts**. +1. To create a new integration, for **Host**, select **GitLab** or **GitLab Self-Managed**. +1. For **Team or User Account**, enter the relative path of a top-level GitLab group that [the GitLab user](#configure-a-gitlab-application-for-dvcs) can access. +1. In the **Host URL** text box, enter the appropriate URL. + Replace `<gitlab.example.com>` with your GitLab instance domain. + Use `https://<gitlab.example.com>`. + +1. For **Client ID**, use the [**Application ID** value](#configure-a-gitlab-application-for-dvcs). +1. For **Client Secret**, use the [**Secret** value](#configure-a-gitlab-application-for-dvcs). +1. Ensure that all other checkboxes are selected. 1. To create the DVCS account, select **Add** and then **Continue**. 1. Jira redirects to GitLab where you have to confirm the authorization. GitLab then redirects back to Jira where the synced @@ -138,7 +112,7 @@ it completes, refreshes every 60 minutes: To connect additional GitLab projects from other GitLab top-level groups or personal namespaces, repeat the previous steps with additional Jira DVCS accounts. -After you configure the integration, read more about [how to test and use it](../development_panel.md). +For more information about how to use the integration, see [Jira development panel](../development_panel.md). ## Refresh data imported to Jira @@ -153,23 +127,21 @@ can refresh the data manually from the Jira interface: ## Migrate to the GitLab for Jira Cloud app -If you are using DVCS with Jira Cloud, you should consider migrating to the GitLab for Jira app. -[DVCS for Jira Cloud will be deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/362168) in GitLab 16.0. - -To get started using the GitLab for Jira Cloud app, follow the guide [Install the GitLab for Jira Cloud app](../connect-app.md#install-the-gitlab-for-jira-cloud-app) . +If you're using the Jira DVCS connector with Jira Cloud, migrate to the GitLab for Jira Cloud app. +For more information, see [Install the GitLab for Jira Cloud app](../connect-app.md#install-the-gitlab-for-jira-cloud-app). ### Feature comparison of DVCS and GitLab for Jira Cloud app -| Feature | DVCS (Jira Cloud) | GitLab for Jira Cloud app | -|--------------------|--------------------|---------------------------------------| +| Feature | DVCS | GitLab for Jira Cloud app | +|--------------------|------------------------|---------------------------| | Smart Commits | **{check-circle}** Yes | **{check-circle}** Yes | | Sync MRs | **{check-circle}** Yes | **{check-circle}** Yes | | Sync branches | **{check-circle}** Yes | **{check-circle}** Yes | -| Sync commits | **{check-circle}** Yes | **{check-circle}** Yes| +| Sync commits | **{check-circle}** Yes | **{check-circle}** Yes | | Sync builds | **{dotted-circle}** No | **{check-circle}** Yes | | Sync deployments | **{dotted-circle}** No | **{check-circle}** Yes | | Sync feature flags | **{dotted-circle}** No | **{check-circle}** Yes | -| Sync interval | 60 Minutes | Real time | +| Sync interval | 60 Minutes | Real time | | Create branches | **{dotted-circle}** No | **{check-circle}** Yes (Only GitLab SaaS) | | Historic data sync | **{check-circle}** Yes | **{dotted-circle}** No | diff --git a/doc/integration/jira/dvcs/troubleshooting.md b/doc/integration/jira/dvcs/troubleshooting.md index e40c188c1c3..6a2e1e052ba 100644 --- a/doc/integration/jira/dvcs/troubleshooting.md +++ b/doc/integration/jira/dvcs/troubleshooting.md @@ -18,6 +18,12 @@ appear in any logs: Error obtaining access token. Cannot access https://gitlab.example.com from Jira. ``` +## Session token bug in Jira + +When using GitLab 15.0 and later (including GitLab.com) with Jira Server, you might experience +a [session token bug in Jira](https://jira.atlassian.com/browse/JSWSERVER-21389). As a workaround, +ensure Jira Server is version 9.1.0 and later or 8.20.11 and later. + ## SSL and TLS problems Problems with SSL and TLS can cause this error message: diff --git a/doc/integration/jira/img/jira-upload-app-success_v13_11.png b/doc/integration/jira/img/jira-upload-app-success_v13_11.png Binary files differdeleted file mode 100644 index c0d4c9744b6..00000000000 --- a/doc/integration/jira/img/jira-upload-app-success_v13_11.png +++ /dev/null diff --git a/doc/integration/jira/img/jira-upload-app_v13_11.png b/doc/integration/jira/img/jira-upload-app_v13_11.png Binary files differdeleted file mode 100644 index 88d1573f778..00000000000 --- a/doc/integration/jira/img/jira-upload-app_v13_11.png +++ /dev/null diff --git a/doc/integration/jira/img/jira_dev_panel_setup_com_1.png b/doc/integration/jira/img/jira_dev_panel_setup_com_1.png Binary files differdeleted file mode 100644 index 18f0d5da043..00000000000 --- a/doc/integration/jira/img/jira_dev_panel_setup_com_1.png +++ /dev/null diff --git a/doc/integration/jira/img/jira_dev_panel_setup_com_2.png b/doc/integration/jira/img/jira_dev_panel_setup_com_2.png Binary files differdeleted file mode 100644 index 31dc13e1271..00000000000 --- a/doc/integration/jira/img/jira_dev_panel_setup_com_2.png +++ /dev/null diff --git a/doc/integration/jira/img/jira_dev_panel_setup_com_3_v13_9.png b/doc/integration/jira/img/jira_dev_panel_setup_com_3_v13_9.png Binary files differdeleted file mode 100644 index fe791637d31..00000000000 --- a/doc/integration/jira/img/jira_dev_panel_setup_com_3_v13_9.png +++ /dev/null diff --git a/doc/integration/jira/img/jira_dev_panel_setup_com_4_v13_9.png b/doc/integration/jira/img/jira_dev_panel_setup_com_4_v13_9.png Binary files differdeleted file mode 100644 index 08787f12b67..00000000000 --- a/doc/integration/jira/img/jira_dev_panel_setup_com_4_v13_9.png +++ /dev/null diff --git a/doc/integration/jira/index.md b/doc/integration/jira/index.md index 3ec4c7aee87..41d6cda4c84 100644 --- a/doc/integration/jira/index.md +++ b/doc/integration/jira/index.md @@ -64,7 +64,7 @@ The authentication method in Jira depends on whether you host Jira on your own s how to [set up a user in Jira Server](jira_server_configuration.md). - **Jira on Atlassian cloud** supports authentication through an API token. When connecting to Jira on Atlassian cloud, an email and API token are required. For more information, see - [Create a Jira Cloud API token](jira_cloud_configuration.md). + [Create a Jira Cloud API token](configure.md#create-a-jira-cloud-api-token). ## Privacy considerations diff --git a/doc/integration/jira/issues.md b/doc/integration/jira/issues.md index 58da871926b..9f8f98bfa5e 100644 --- a/doc/integration/jira/issues.md +++ b/doc/integration/jira/issues.md @@ -62,6 +62,41 @@ After you enable this feature, a merge request that doesn't reference an associa Jira issue can't be merged. The merge request displays the message **To merge, a Jira issue key must be mentioned in the title or description.** +## Customize Jira issue matching in GitLab + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112826) in GitLab 15.10. + +You can configure custom rules for how GitLab matches Jira issue keys by defining: + +- [A regex pattern](#use-regular-expression) +- [A prefix](#use-a-prefix) + +When you don't configure custom rules, the [default behavior](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/regex.rb#L509) is used. For more information, see the [RE2 wiki](https://github.com/google/re2/wiki/Syntax). + +### Use regular expression + +To define a regex pattern for Jira issue keys: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Integrations**. +1. Select **Jira**. +1. Go to the **Jira issue matching** section. +1. In the **Jira issue regex** text box, enter a regex pattern. +1. Select **Save changes**. + +For more information, see the [Atlassian documentation](https://confluence.atlassian.com/adminjiraserver073/changing-the-project-key-format-861253229.html). + +### Use a prefix + +To define a prefix for Jira issue keys: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Integrations**. +1. Select **Jira**. +1. Go to the **Jira issue matching** section. +1. In the **Jira issue prefix** text box, enter a prefix. +1. Select **Save changes**. + ## Close Jira issues in GitLab If you have configured GitLab transition IDs, you can close a Jira issue directly diff --git a/doc/integration/jira/jira_cloud_configuration.md b/doc/integration/jira/jira_cloud_configuration.md index 9d52368f528..83321df0420 100644 --- a/doc/integration/jira/jira_cloud_configuration.md +++ b/doc/integration/jira/jira_cloud_configuration.md @@ -1,23 +1,11 @@ --- -stage: Manage -group: Integrations -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'configure.md#create-a-jira-cloud-api-token' +remove_date: '2023-07-07' --- -# Create a Jira Cloud API token **(FREE)** +This document was moved to [another location](configure.md#create-a-jira-cloud-api-token). -You need an API token to [integrate with Jira](index.md) -in Atlassian Cloud. To create the API token: - -1. Sign in to [Atlassian](https://id.atlassian.com/manage-profile/security/api-tokens) - using an account with *write* access to Jira projects. - - The link opens the API tokens page. Alternatively, to go to this page from your Atlassian - profile, select **Account Settings > Security > Create and manage API tokens**. -1. Select **Create API token**. -1. In the dialog, enter a label for your token and select **Create**. -1. To copy the API token, select **Copy**, then paste the token somewhere safe. - -You need the newly created token, and the email -address you used when you created it, when you -[configure GitLab](configure.md). +<!-- This redirect file can be deleted after <2023-07-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/integration/jira/troubleshooting.md b/doc/integration/jira/troubleshooting.md index bf347c4698d..33ba989699d 100644 --- a/doc/integration/jira/troubleshooting.md +++ b/doc/integration/jira/troubleshooting.md @@ -4,14 +4,14 @@ group: Integrations info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Troubleshooting Jira integrations **(FREE)** +# Troubleshooting Jira **(FREE)** This page contains a list of common issues you might encounter when working with Jira integrations. ## GitLab cannot comment on a Jira issue If GitLab cannot comment on Jira issues, make sure the Jira user you -set up for the integration has permission to: +set up for the [Jira integration](configure.md) has permission to: - Post comments on a Jira issue. - Transition the Jira issue. diff --git a/doc/integration/kerberos.md b/doc/integration/kerberos.md index f0c1a75041e..dc4dd501363 100644 --- a/doc/integration/kerberos.md +++ b/doc/integration/kerberos.md @@ -99,7 +99,7 @@ to authenticate with Kerberos tokens. #### Enable single sign-on -Edit the [common configuration file settings](omniauth.md#configure-common-settings) +Configure the [common settings](omniauth.md#configure-common-settings) to add `kerberos` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. @@ -358,6 +358,16 @@ to a larger value in [the NGINX configuration](https://nginx.org/en/docs/http/ng ## Troubleshooting +### Using Google Chrome with Kerberos authentication against Windows AD + +When you use Google Chrome to sign in to GitLab with Kerberos, you must enter your full username. For example, `username@domain.com`. + +If you do not enter your full username, the sign-in fails. Check the logs to see the following event message as evidence of this sign-in failure: + +```plain +"message":"OmniauthKerberosController: failed to process Negotiate/Kerberos authentication: gss_accept_sec_context did not return GSS_S_COMPLETE: An unsupported mechanism was requested\nUnknown error"`. +``` + ### Test connectivity between the GitLab and Kerberos servers You can use utilities like [`kinit`](https://web.mit.edu/kerberos/krb5-1.12/doc/user/user_commands/kinit.html) and [`klist`](https://web.mit.edu/kerberos/krb5-1.12/doc/user/user_commands/klist.html) to test connectivity between the GitLab server diff --git a/doc/integration/oauth2_generic.md b/doc/integration/oauth2_generic.md index 2c0439a328c..79238c78421 100644 --- a/doc/integration/oauth2_generic.md +++ b/doc/integration/oauth2_generic.md @@ -54,7 +54,7 @@ To configure the provider: :::TabTitle Linux package (Omnibus) - 1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) + 1. Configure the [common settings](omniauth.md#configure-common-settings) to add `oauth2_generic` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. Edit `/etc/gitlab/gitlab.rb` to add the configuration for your provider. For example: @@ -98,7 +98,7 @@ To configure the provider: :::TabTitle Helm chart (Kubernetes) - 1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) + 1. Configure the [common settings](omniauth.md#configure-common-settings) to add `oauth2_generic` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. Export the Helm values: @@ -107,39 +107,45 @@ To configure the provider: helm get values gitlab > gitlab_values.yaml ``` - 1. Edit `gitlab_values.yaml`. + 1. Put the following content in a file named `oauth2_generic.yaml` for use as a + [Kubernetes Secret](https://docs.gitlab.com/charts/charts/globals.html#providers): - NOTE: - The following example exposes the `app_secret` value in the main YAML file. - You're strongly advised to use - [Helm secrets](https://docs.gitlab.com/charts/installation/secrets.html) - instead. + ```yaml + name: "oauth2_generic" + label: "Provider name" # optional label for login button defaults to "Oauth2 Generic" + app_id: "<your_app_client_id>" + app_secret: "<your_app_client_secret>" + args: + client_options: + site: "<your_auth_server_url>" + user_info_url: "/oauth2/v1/userinfo" + authorize_url: "/oauth2/v1/authorize" + token_url: "/oauth2/v1/token" + user_response_structure: + root_path: [] + id_path: ["sub"] + attributes: + email: "email" + name: "name" + authorize_params: + scope: "openid profile email" + strategy_class: "OmniAuth::Strategies::OAuth2Generic" + ``` + + 1. Create the Kubernetes Secret: + + ```shell + kubectl create secret generic -n <namespace> gitlab-oauth2-generic --from-file=provider=oauth2_generic.yaml + ``` + + 1. Edit `gitlab_values.yaml` and add the provider configuration: ```yaml global: appConfig: omniauth: - enabled: true providers: - - name: "oauth2_generic" - label: "Provider name" # optional label for login button defaults to "Oauth2 Generic" - app_id: "<your_app_client_id>" - app_secret: "<your_app_client_secret>" - args: - client_options: - site: "<your_auth_server_url>" - user_info_url: "/oauth2/v1/userinfo" - authorize_url: "/oauth2/v1/authorize" - token_url: "/oauth2/v1/token" - user_response_structure: - root_path: [] - id_path: ["sub"] - attributes: - email: "email" - name: "name" - authorize_params: - scope: "openid profile email" - strategy_class: "OmniAuth::Strategies::OAuth2Generic" + - secret: gitlab-oauth2-generic ``` 1. Save the file and apply the new values: @@ -150,7 +156,7 @@ To configure the provider: :::TabTitle Self-compiled (source) - 1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) + 1. Configure the [common settings](omniauth.md#configure-common-settings) to add `oauth2_generic` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. Edit `/home/git/gitlab/config/gitlab.yml`: diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index 66d4ccb871f..907d96d8185 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -307,7 +307,7 @@ To enable automatic linking for SAML, see the [SAML setup instructions](saml.md# ## Create an external providers list You can define a list of external OmniAuth providers. -Users who create accounts or sign in to GitLab through the listed providers do not get access to [internal projects](../user/public_access.md#internal-projects-and-groups). +Users who create accounts or sign in to GitLab through the listed providers do not get access to [internal projects](../user/public_access.md#internal-projects-and-groups) and are marked as [external users](../user/admin_area/external_users.md). To define the external providers list, use the full name of the provider, for example, `google_oauth2` for Google. For provider names, see the diff --git a/doc/integration/partner_marketplace.md b/doc/integration/partner_marketplace.md index 5ed131145f4..76988af99b2 100644 --- a/doc/integration/partner_marketplace.md +++ b/doc/integration/partner_marketplace.md @@ -60,7 +60,7 @@ To access the Marketplace API you need to: - Retrieve an OAuth access token. Marketplace API endpoints are secured with [OAuth 2.0](https://oauth.net/2/). OAuth is an authorization framework -that grants 3rd party or client applications, like a GitLab Partner application, limited access to resources on an +that grants 3rd party or client applications, like a Marketplace partner application, limited access to resources on an HTTP service, like the Customers Portal. OAuth 2.0 uses _grant types_ (or _flows_) that describe how a client application gets authorization in @@ -72,14 +72,14 @@ own resources, instead of accessing resources on behalf of a user. ### Step 1: Request access -Before you can use the Marketplace API, you must contact your GitLab Partner Manager or email [`partnerorderops`](mailto:partnerorderops@gitlab.com) +Before you can use the Marketplace API, you must contact your Marketplace partner Manager or email [`partnerorderops`](mailto:partnerorderops@gitlab.com) to request access. After you request access, GitLab configures the following accounts and credentials for you: 1. Client credentials. Marketplace APIs are secured with OAuth 2.0. The client credentials include the client ID and client secret that you need to retrieve the OAuth access token. 1. Invoice owner account in Zuora system. Required for invoice processing. 1. Distributor account in Salesforce system. -1. Trading partner account in Salesforce system. +1. Trading partner account in Salesforce system. GitLab adds the Trading Partner ID to a permitted list to pass the validations. ### Step 2: Retrieve an access token @@ -121,7 +121,7 @@ curl \ ## Create a new customer subscription -To create a new customer subscription from a GitLab Partner client application, +To create a new customer subscription from a Marketplace partner client application, - Make an authorized POST request to the [`/api/v1/marketplace/subscriptions`](https://customers.staging.gitlab.com/openapi_docs/marketplace#/marketplace/post_api_v1_marketplace_subscriptions) @@ -129,8 +129,8 @@ endpoint in the Customers Portal with the following parameters in JSON format: | Parameter | Type | Required | Description | |--------------------------|--------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------| -| `externalSubscriptionId` | string | yes | ID of the subscription on the GitLab Partner system. | -| `tradingPartnerId` | string | yes | ID of the GitLab Partner account on the Customers Portal. | +| `externalSubscriptionId` | string | yes | ID of the subscription on the Marketplace partner system. | +| `tradingPartnerId` | string | yes | ID of the Trading Partner account in Salesforce. Received from GitLab. | | `customer` | object | yes | Information about the customer. Must include company name. Contact must include `firstName`, `lastName` and `email`. Address must include `country`. | | `orderLines` | array | yes | Specifies the product purchased. Must include `quantity` and `productId`. | @@ -147,7 +147,7 @@ To get the status of a given subscription, [`/api/v1/marketplace/subscriptions/{external_subscription_id}`](https://customers.staging.gitlab.com/openapi_docs/marketplace#/marketplace/get_api_v1_marketplace_subscriptions__external_subscription_id_) endpoint in the Customers Portal. -The request must include the GitLab partner system ID of the subscription to fetch the status for. +The request must include the Marketplace partner system ID of the subscription to fetch the status for. If the request is successful, the response body contains the status of the subscription provision. The status can be: diff --git a/doc/integration/salesforce.md b/doc/integration/salesforce.md index 16432d3ca5d..8c43f038f86 100644 --- a/doc/integration/salesforce.md +++ b/doc/integration/salesforce.md @@ -48,7 +48,7 @@ To get the credentials (a pair of Client ID and Client Secret), you must [create sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `salesforce` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 888dd47a968..ef32311f85d 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -19,7 +19,7 @@ To set up SAML on GitLab.com, see [SAML SSO for GitLab.com groups](../user/group For more information on: - OmniAuth provider settings, see the [OmniAuth documentation](omniauth.md). -- Commonly-used terms, see the [glossary of common terms](#glossary-of-common-terms). +- Commonly-used terms, see the [glossary](#glossary). ## Configure SAML support in GitLab @@ -28,7 +28,7 @@ For more information on: :::TabTitle Linux package (Omnibus) 1. Make sure GitLab is [configured with HTTPS](https://docs.gitlab.com/omnibus/settings/ssl/). -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `saml` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. To allow your users to use SAML to sign up without having to manually create @@ -51,7 +51,7 @@ For more information on: 1. Configure the following attributes so your SAML users cannot change them: - - [`NameID`](../user/group/saml_sso/index.md#nameid). + - [`NameID`](../user/group/saml_sso/index.md#manage-user-saml-identity). - `Email` when used with `omniauth_auto_link_saml_user`. If users can change these attributes, they can sign in as other authorized users. @@ -98,7 +98,7 @@ For more information on: :::TabTitle Helm chart (Kubernetes) 1. Make sure GitLab is [configured with HTTPS](https://docs.gitlab.com/charts/installation/tls.html). -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `saml` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. Export the Helm values: @@ -134,7 +134,7 @@ For more information on: 1. Configure the following attributes so your SAML users cannot change them: - - [`NameID`](../user/group/saml_sso/index.md#nameid). + - [`NameID`](../user/group/saml_sso/index.md#manage-user-saml-identity). - `Email` when used with `omniauth_auto_link_saml_user`. If users can change these attributes, they can sign in as other authorized users. @@ -193,7 +193,7 @@ For more information on: :::TabTitle Docker 1. Make sure GitLab is [configured with HTTPS](https://docs.gitlab.com/omnibus/settings/ssl/). -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `saml` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. To allow your users to use SAML to sign up without having to manually create @@ -226,7 +226,7 @@ For more information on: 1. Configure the following attributes so your SAML users cannot change them: - - [`NameID`](../user/group/saml_sso/index.md#nameid). + - [`NameID`](../user/group/saml_sso/index.md#manage-user-saml-identity). - `Email` when used with `omniauth_auto_link_saml_user`. If users can change these attributes, they can sign in as other authorized users. @@ -278,7 +278,7 @@ For more information on: :::TabTitle Self-compiled (source) 1. Make sure GitLab is [configured with HTTPS](../install/installation.md#using-https). -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `saml` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. 1. To allow your users to use SAML to sign up without having to manually create @@ -306,7 +306,7 @@ For more information on: 1. Configure the following attributes so your SAML users cannot change them: - - [`NameID`](../user/group/saml_sso/index.md#nameid). + - [`NameID`](../user/group/saml_sso/index.md#manage-user-saml-identity). - `Email` when used with `omniauth_auto_link_saml_user`. If users can change these attributes, they can sign in as other authorized users. @@ -382,7 +382,7 @@ To configure a SAML application on your IdP, you need at least the following inf - Assertion consumer service URL. - Issuer. -- [`NameID`](../user/group/saml_sso/index.md#nameid). +- [`NameID`](../user/group/saml_sso/index.md#manage-user-saml-identity). - [Email address claim](#configure-assertions). For an example configuration, see [set up identity providers](#set-up-identity-providers). @@ -653,7 +653,7 @@ IdPs, contact your provider's support. 1. Complete the SAML general configuration. Enter: - `"Single sign-on URL"`: Use the assertion consumer service URL. - `"Audience URI"`: Use the issuer. - - [`NameID`](../user/group/saml_sso/index.md#nameid). + - [`NameID`](../user/group/saml_sso/index.md#manage-user-saml-identity). - [Assertions](#configure-assertions). 1. In the feedback section, enter that you're a customer and creating an app for internal use. @@ -699,7 +699,7 @@ When configuring the Google Workspace SAML application, record the following inf | | Value | Description | |-------------|--------------|-----------------------------------------------------------------------------------| | SSO URL | Depends | Google Identity Provider details. Set to the GitLab `idp_sso_target_url` setting. | -| Certificate | Downloadable | Run `openssl x509 -in <your_certificate.crt> -noout -fingerprint` to generate the SHA1 fingerprint that can be used in the `idp_cert_fingerprint` setting. | +| Certificate | Downloadable | Run `openssl x509 -in <your_certificate.crt> -noout -fingerprint -sha1` to generate the SHA1 fingerprint that can be used in the `idp_cert_fingerprint` setting. | Google Workspace Administrator also provides the IdP metadata, Entity ID, and SHA-256 fingerprint. However, GitLab does not need this information to connect to the @@ -2139,7 +2139,7 @@ instead of `email`, let GitLab know by setting it on your configuration: By default, the local part of the email address in the SAML response is used to generate the user's GitLab username. -Configure `nickname` in `attribute_statements` to specify one or more attributes that contain a user's desired username: +Configure [`username` or `nickname`](omniauth.md#per-provider-configuration) in `attribute_statements` to specify one or more attributes that contain a user's desired username: ::Tabs @@ -2442,7 +2442,7 @@ The value given is added to the current time at which the response is validated. Before setting the `uid` to a unique attribute, make sure that you have configured the following attributes so your SAML users cannot change them: -- [`NameID`](../user/group/saml_sso/index.md#nameid). +- [`NameID`](../user/group/saml_sso/index.md#manage-user-saml-identity). - `Email` when used with `omniauth_auto_link_saml_user`. If users can change these attributes, they can sign in as other authorized users. @@ -3120,12 +3120,12 @@ such as the following: | Sign SAML assertion | Optional | Validates the integrity of a SAML assertion. When active, signs the whole response. | | Check SAML request signature | Optional | Checks the signature on the SAML response. | | Default RelayState | Optional | Specifies the URL users should end up on after successfully signing in through SAML at your IdP. | -| NameID format | Persistent | See [NameID format details](../user/group/saml_sso/index.md#nameid-format). | +| NameID format | Persistent | See [NameID format details](../user/group/saml_sso/index.md#manage-user-saml-identity). | | Additional URLs | Optional | May include the issuer, identifier, or assertion consumer service URL in other fields on some providers. | For example configurations, see the [notes on specific providers](#set-up-identity-providers). -## Glossary of common terms +## Glossary | Term | Description | |--------------------------------|-------------| diff --git a/doc/integration/security_partners/index.md b/doc/integration/security_partners/index.md index 1ad8ab03914..61519057c09 100644 --- a/doc/integration/security_partners/index.md +++ b/doc/integration/security_partners/index.md @@ -15,7 +15,7 @@ each security partner: - [Anchore](https://docs.anchore.com/current/docs/configuration/integration/ci_cd/gitlab/) - [Bridgecrew](https://docs.bridgecrew.io/docs/integrate-with-gitlab-self-managed) - [Checkmarx](https://checkmarx.atlassian.net/wiki/spaces/SD/pages/1929937052/GitLab+Integration) -- [Deepfactor](https://docs.deepfactor.io/hc/en-us/articles/1500008981941) +- [Deepfactor](https://www.deepfactor.io/docs/gitlab/) - [GrammaTech](https://www.grammatech.com/codesonar-gitlab-integration) - [Indeni](https://docs.cloudrail.app/#/integrations/gitlab) - [JScrambler](https://docs.jscrambler.com/code-integrity/documentation/gitlab-ci-integration) @@ -23,7 +23,7 @@ each security partner: - [Semgrep](https://semgrep.dev/for/gitlab) - [StackHawk](https://docs.stackhawk.com/continuous-integration/gitlab.html) - [Tenable](https://docs.tenable.com/tenableio/Content/ContainerSecurity/GetStarted.htm) -- [Venafi](https://marketplace.venafi.com/details/gitlab-ci-cd/) +- [Venafi](https://marketplace.venafi.com/xchange/620d2d6ed419fb06a5c5bd36/solution/6292c2ef7550f2ee553cf223) - [Veracode](https://community.veracode.com/s/knowledgeitem/gitlab-ci-MCEKSYPRWL35BRTGOVI55SK5RI4A) - [Fortify](https://www.microfocus.com/en-us/fortify-integrations/gitlab) diff --git a/doc/integration/sourcegraph.md b/doc/integration/sourcegraph.md index a40290683a5..d90a1fa2cca 100644 --- a/doc/integration/sourcegraph.md +++ b/doc/integration/sourcegraph.md @@ -89,23 +89,16 @@ When visiting one of these views, you can now hover over a code reference to see ## Sourcegraph for GitLab.com -Sourcegraph powered code intelligence is available for all public projects on GitLab.com. +Sourcegraph is available for all public projects on GitLab.com. +Private projects are not supported. +For more information, see [epic 2201](https://gitlab.com/groups/gitlab-org/-/epics/2201). -Support for private projects is not yet available for GitLab.com; -follow the epic [&2201](https://gitlab.com/groups/gitlab-org/-/epics/2201) -for updates. +## Sourcegraph and privacy + +See the [Sourcegraph browser extension documentation](https://docs.sourcegraph.com/integration/browser_extension/references/privacy). ## Troubleshooting ### Sourcegraph isn't working If you enabled Sourcegraph for your project but it isn't working, Sourcegraph may not have indexed the project yet. You can check if Sourcegraph is available for your project by visiting `https://sourcegraph.com/gitlab.com/<project-path>`replacing `<project-path>` with the path to your GitLab project. - -## Sourcegraph and Privacy - -From the Sourcegraph [extension documentation](https://docs.sourcegraph.com/integration/browser_extension#privacy) which is the -engine behind the native GitLab integration: - -> Sourcegraph integrations never send any logs, pings, usage statistics, or telemetry to Sourcegraph.com. -> They connect only to Sourcegraph.com as required to provide code intelligence or other functionality on public code. -> As a result, no private code, private repository names, usernames, or any other specific data is sent to Sourcegraph.com. diff --git a/doc/integration/twitter.md b/doc/integration/twitter.md index f1bfc5a3662..0ccda59f9b3 100644 --- a/doc/integration/twitter.md +++ b/doc/integration/twitter.md @@ -62,7 +62,7 @@ Twitter. Twitter generates a client ID and secret key for you to use. sudo -u git -H editor config/gitlab.yml ``` -1. Edit the [common configuration file settings](omniauth.md#configure-common-settings) +1. Configure the [common settings](omniauth.md#configure-common-settings) to add `twitter` as a single sign-on provider. This enables Just-In-Time account provisioning for users who do not have an existing GitLab account. diff --git a/doc/integration/vault.md b/doc/integration/vault.md index 78456596723..c505097d65c 100644 --- a/doc/integration/vault.md +++ b/doc/integration/vault.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/legal/index.md b/doc/legal/index.md index ea0301284ea..17a17ba5509 100644 --- a/doc/legal/index.md +++ b/doc/legal/index.md @@ -2,7 +2,6 @@ stage: none group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Legal diff --git a/doc/operations/feature_flags.md b/doc/operations/feature_flags.md index 968649e94b2..a3874b54ccd 100644 --- a/doc/operations/feature_flags.md +++ b/doc/operations/feature_flags.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -38,7 +38,7 @@ with GitLab, so it's up to developers to use a compatible client library and To create and enable a feature flag: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Feature Flags**. +1. On the left sidebar, select **Deployments > Feature flags**. 1. Select **New feature flag**. 1. Enter a name that starts with a letter and contains only lowercase letters, digits, underscores (`_`), or dashes (`-`), and does not end with a dash (`-`) or underscore (`_`). @@ -85,7 +85,7 @@ and the supported strategies are: - [User List](#user-list) Strategies can be added to feature flags when [creating a feature flag](#create-a-feature-flag), -or by editing an existing feature flag after creation by navigating to **Deployments > Feature Flags** +or by editing an existing feature flag after creation by navigating to **Deployments > Feature flags** and selecting **Edit** (**{pencil}**). ### All users @@ -181,7 +181,7 @@ For example: To create a user list: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Feature Flags**. +1. On the left sidebar, select **Deployments > Feature flags**. 1. Select **View user lists** 1. Select **New user list**. 1. Enter a name for the list. @@ -197,7 +197,7 @@ When viewing a list, you can rename it by selecting **Edit** (**{pencil}**). To add users to a user list: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Feature Flags**. +1. On the left sidebar, select **Deployments > Feature flags**. 1. Select **Edit** (**{pencil}**) next to the list you want to add users to. 1. Select **Add Users**. 1. Enter the user IDs as a comma-separated list of values. For example, @@ -211,7 +211,7 @@ To add users to a user list: To remove users from a user list: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Feature Flags**. +1. On the left sidebar, select **Deployments > Feature flags**. 1. Select **Edit** (**{pencil}**) next to the list you want to change. 1. Select **Remove** (**{remove}**) next to the ID you want to remove. @@ -225,7 +225,7 @@ code so that you can clean it up when it's time to remove the feature flag. To search for code references of a feature flag: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Feature Flags**. +1. On the left sidebar, select **Deployments > Feature flags**. 1. Edit the feature flag you want to remove. 1. Select **More actions** (**{ellipsis_v}**). 1. Select **Search code references**. @@ -236,7 +236,7 @@ In [GitLab 13.0 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/8621) to disable a feature flag for a specific environment: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Feature Flags**. +1. On the left sidebar, select **Deployments > Feature flags**. 1. For the feature flag you want to disable, select **Edit** (**{pencil}**). 1. To disable the flag: @@ -251,7 +251,7 @@ to disable a feature flag for a specific environment: To disable a feature flag for all environments: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Feature Flags**. +1. On the left sidebar, select **Deployments > Feature flags**. 1. For the feature flag you want to disable, slide the Status toggle to **Disabled**. The feature flag is displayed on the **Disabled** tab. @@ -266,7 +266,7 @@ Then prepare your application with a client library. To get the access credentials that your application needs to communicate with GitLab: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Feature Flags**. +1. On the left sidebar, select **Deployments > Feature flags**. 1. Select **Configure** to view the following: - **API URL**: URL where the client (application) connects to get a list of feature flags. - **Instance ID**: Unique token that authorizes the retrieval of the feature flags. diff --git a/doc/operations/incident_management/incident_timeline_events.md b/doc/operations/incident_management/incident_timeline_events.md index 7913e4668eb..d509061eca0 100644 --- a/doc/operations/incident_management/incident_timeline_events.md +++ b/doc/operations/incident_management/incident_timeline_events.md @@ -83,6 +83,17 @@ of an incident.  +### When labels change (Experiment) + +> [Introduced]([issue-link](https://gitlab.com/gitlab-org/gitlab/-/issues/365489)) in GitLab 15.3 [with a flag](../../administration/feature_flags.md) named `incident_timeline_events_from_labels`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `incident_timeline_events_from_labels`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +A new timeline event is created when someone adds or removes [labels](../../user/project/labels.md) on an incident. + ## Delete an event > Ability to delete an event when editing it [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/372265) in GitLab 15.7. @@ -113,10 +124,7 @@ Alternatively: > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8741) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `incident_event_tags`. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/387647) in GitLab 15.9. > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/387647) in GitLab 15.10. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `incident_event_tags`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/387647) in GitLab 15.11. Feature flag `incident_event_tags` removed. [When creating an event using the form](#using-the-form) or editing it, you can specify incident tags to capture relevant incident timestamps. diff --git a/doc/operations/incident_management/incidents.md b/doc/operations/incident_management/incidents.md index 45f1e10d2f1..9bfaf0efa7e 100644 --- a/doc/operations/incident_management/incidents.md +++ b/doc/operations/incident_management/incidents.md @@ -94,7 +94,7 @@ If an incident template is configured for the project, then the template content Comments are displayed in threads, but can be displayed chronologically [by toggling on the recent updates view](#recent-updates-view). -When you make changes to an incident, GitLab creates system notes and +When you make changes to an incident, GitLab creates [system notes](../../user/project/system_notes.md) and displays them below the summary. ### Metrics **(PREMIUM)** @@ -187,7 +187,7 @@ label to the incident. - [Toggle notifications](../../user/profile/notifications.md#edit-notification-settings-for-issues-merge-requests-and-epics) - [Track spent time](../../user/project/time_tracking.md) - [Add a Zoom meeting to an incident](../../user/project/issues/associate_zoom_meeting.md) the same - way you add it to an issue. + way you add it to an issue - [Linked resources in incidents](linked_resources.md) -- Create incidents and receive incident notifications [directly from Slack](slack.md). -- Use the [Issues API](../../api/issues.md) to interact with incidents. +- Create incidents and receive incident notifications [directly from Slack](slack.md) +- Use the [Issues API](../../api/issues.md) to interact with incidents diff --git a/doc/operations/incident_management/slack.md b/doc/operations/incident_management/slack.md index 434c481900c..62f77ec9f06 100644 --- a/doc/operations/incident_management/slack.md +++ b/doc/operations/incident_management/slack.md @@ -4,13 +4,14 @@ group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Incident management for Slack **(FREE SAAS)** +# Incident management for Slack (Beta) **(FREE SAAS)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344856) in GitLab 15.7 [with a flag](../../administration/feature_flags.md) named `incident_declare_slash_command`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344856) in GitLab 15.7 [with a flag](../../administration/feature_flags.md) named `incident_declare_slash_command`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/378072) in GitLab 15.10 in [Beta](../../policy/alpha-beta-support.md#beta). FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `incident_declare_slash_command`. -On GitLab.com, this feature is not available. +On self-managed GitLab, this feature is not available. +On GitLab.com, this feature is available. The feature is not ready for production use. Many teams receive alerts and collaborate in real time during incidents in Slack. diff --git a/doc/operations/index.md b/doc/operations/index.md index d4a5f895947..56b27af316b 100644 --- a/doc/operations/index.md +++ b/doc/operations/index.md @@ -76,4 +76,4 @@ an environment. - Deploy to different [environments](../ci/environments/index.md). - Connect your project to a [Kubernetes cluster](../user/infrastructure/clusters/index.md). -- Create, toggle, and remove [Feature Flags](feature_flags.md). +- Create, toggle, and remove [feature flags](feature_flags.md). diff --git a/doc/operations/metrics/dashboards/panel_types.md b/doc/operations/metrics/dashboards/panel_types.md index 39893f279ff..5d52691eed8 100644 --- a/doc/operations/metrics/dashboards/panel_types.md +++ b/doc/operations/metrics/dashboards/panel_types.md @@ -234,7 +234,7 @@ For example, if you have a query value of `53.6`, adding `%` as the unit results ## Gauge WARNING: -This panel type is an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) feature, and is subject to change at any time +This panel type is an [Experiment](../../../policy/alpha-beta-support.md#experiment) and is subject to change at any time without prior notice! > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207044) in GitLab 13.3. diff --git a/doc/operations/metrics/dashboards/templating_variables.md b/doc/operations/metrics/dashboards/templating_variables.md index a27e75aacef..7af4b9a9e43 100644 --- a/doc/operations/metrics/dashboards/templating_variables.md +++ b/doc/operations/metrics/dashboards/templating_variables.md @@ -27,7 +27,7 @@ described [in Using Variables](variables.md). ## `text` variable type WARNING: -This variable type is an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) feature, and is subject to change at any time +This variable type is an [Experiment](../../../policy/alpha-beta-support.md#experiment), and is subject to change at any time without prior notice! For each `text` variable defined in the dashboard YAML, a free text field displays @@ -64,7 +64,7 @@ templating: ## `custom` variable type WARNING: -This variable type is an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) feature, and is subject to change at any time +This variable type is an [Experiment](../../../policy/alpha-beta-support.md#experiment), and is subject to change at any time without prior notice! Each `custom` variable defined in the dashboard YAML creates a dropdown list @@ -112,7 +112,7 @@ templating: ## `metric_label_values` variable type WARNING: -This variable type is an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) feature, and is subject to change at any time +This variable type is an [Experiment](../../../policy/alpha-beta-support.md#experiment), and is subject to change at any time without prior notice! ### Full syntax diff --git a/doc/policy/alpha-beta-support.md b/doc/policy/alpha-beta-support.md index 98910f9a3bb..1d5bacf5a2b 100644 --- a/doc/policy/alpha-beta-support.md +++ b/doc/policy/alpha-beta-support.md @@ -4,68 +4,48 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Support for Alpha, Beta, Limited Availability, and Generally Available features **(PREMIUM)** +# Support for Experiment, Beta, and Generally Available features **(PREMIUM)** -Some GitLab features are released as Alpha or Beta versions and are +Some GitLab features are released as Experiment or Beta versions and are [not fully supported](https://about.gitlab.com/support/statement-of-support/#alpha-beta-features). All other features are considered to be Generally Available (GA). -## Alpha features +## Experiment -Support is **not** provided for Alpha features and issues with them should be opened in the [GitLab issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues). +Support is not provided for features listed as "Experimental" or "Alpha" or any similar designation. Issues regarding such features should be opened in the GitLab issue tracker. -Characteristics of Alpha features: +- Not ready for production use +- No support available +- May be unstable or have performance issues +- Can be removed at any time +- Data loss may occur +- Documentation may not exist or just be in a blog format +- Behind a feature flag that is on by default and the UI reflects Experiment status. +- Behind a toggle that is off by default and the UI reflects Experiment status. +- Feedback issue to engage with team +- UX not finalized, might be just quick action access +- Not announced in a release post -- Not ready for production use. -- Unstable and can cause performance and stability issues. -- Configuration and dependencies are likely to change. -- Features and functions may be removed. Breaking changes may occur outside of major releases or with less notice than for Beta or Generally Available features. -- Data loss can occur (be that through bugs or updates). -- Documentation reflects the Alpha status. -- Behind flags that are off by default. -- Not announced in release posts. +## Beta -## Beta features +Commercially-reasonable efforts are made to provide limited support for features designated as "Beta," with the expectation that issues require extra time and assistance from development to troubleshoot. -Your Support Contract provides **commercially-reasonable effort** support for Beta features, with the expectation that issues require extra time and assistance from development to troubleshoot. - -### Closed Beta features - -Closed Beta features are available to selected users only. - -- Not ready for production use. -- Unstable and can cause performance and stability issues. -- Configuration and dependencies unlikely to change. -- Features and functions unlikely to change. However, breaking changes may occur outside of major releases or with less notice than for Generally Available features. -- Data loss less likely. -- Behind a feature flag that is off by default and the UI reflects Beta status. -- Documentation reflects Beta status. -- Can be announced in a release post that reflects Beta status. - -### Open Beta features - -- Not ready for production use. -- Unstable and can cause performance and stability issues. +- May not be ready for production use and the UI and documentation will reflect this status +- May be unstable and can cause performance and stability issues. - Configuration and dependencies unlikely to change. - Features and functions unlikely to change. However, breaking changes may occur outside of major releases or with less notice than for Generally Available features. - Data loss not likely. - Support on a commercially-reasonable effort basis. - Documentation reflects Beta status. +- UX complete or near completion. - Behind a feature flag that is on by default and the UI reflects Beta status. - Behind a toggle that is off by default and the UI reflects Beta status. - Can be announced in a release post that reflects Beta status. -## Limited Availability (LA) - -Characteristics of Limited Availability features: - -- Ready for production use by a small set of customers. -- Can be booked by Deal Desk as part of an order. -- Fully documented and [supported](https://about.gitlab.com/support/statement-of-support/#starter-premium-and-ultimate-users). - ## Generally Available (GA) Generally Available features means that they passed the [Production Readiness Review](https://gitlab.com/gitlab-com/gl-infra/readiness/-/blob/master/.gitlab/issue_templates/production_readiness.md) for GitLab.com, and are: - Ready for production use at any scale. - Fully documented and supported. +- UX complete and in line with GitLab design standards. diff --git a/doc/raketasks/backup_gitlab.md b/doc/raketasks/backup_gitlab.md index 5b4df8752f4..e04d63da2ad 100644 --- a/doc/raketasks/backup_gitlab.md +++ b/doc/raketasks/backup_gitlab.md @@ -353,9 +353,17 @@ to create an incremental backup from: To create an incremental backup, run: -```shell -sudo gitlab-backup create INCREMENTAL=yes PREVIOUS_BACKUP=<timestamp_of_backup> -``` +- In GitLab 15.0 or later: + + ```shell + sudo gitlab-backup create INCREMENTAL=yes PREVIOUS_BACKUP=<timestamp_of_backup> + ``` + +- In GitLab 14.9 and 14.10: + + ```shell + sudo gitlab-backup create INCREMENTAL=yes BACKUP=<timestamp_of_backup> + ``` To create an [untarred](#skipping-tar-creation) incremental backup from a tarred backup, use `SKIP=tar`: diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md index 3f716900d17..11a6e06fcf5 100644 --- a/doc/raketasks/cleanup.md +++ b/doc/raketasks/cleanup.md @@ -111,6 +111,8 @@ I, [2018-07-27T12:08:33.755624 #89817] INFO -- : Did fix /opt/gitlab/embedded/s I, [2018-07-27T12:08:33.760257 #89817] INFO -- : Did move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png ``` +If using object storage, run the [All-in-one Rake task](../administration/raketasks/uploads/migrate.md#all-in-one-rake-task) to ensure all uploads are migrated to object storage and there are no files on disk in the uploads folder. + ### Clean up project upload files from object storage > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20918) in GitLab 11.2. diff --git a/doc/raketasks/index.md b/doc/raketasks/index.md index e559605d147..a44f053bc7b 100644 --- a/doc/raketasks/index.md +++ b/doc/raketasks/index.md @@ -2,7 +2,6 @@ stage: Systems group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Rake tasks **(FREE SELF)** diff --git a/doc/raketasks/migrate_snippets.md b/doc/raketasks/migrate_snippets.md index e51edc5c133..16f78d7fc71 100644 --- a/doc/raketasks/migrate_snippets.md +++ b/doc/raketasks/migrate_snippets.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/raketasks/restore_gitlab.md b/doc/raketasks/restore_gitlab.md index 13422817503..30395c36c91 100644 --- a/doc/raketasks/restore_gitlab.md +++ b/doc/raketasks/restore_gitlab.md @@ -30,14 +30,21 @@ is usually not allowed to create or delete the SQL database needed to import data into (`gitlabhq_production`). All existing data is either erased (SQL) or moved to a separate directory (such as repositories and uploads). -To restore a backup, you must restore `/etc/gitlab/gitlab-secrets.json` -(for Omnibus packages) or `/home/git/gitlab/.secret` (for installations from -source). This file contains the database encryption key, -[CI/CD variables](../ci/variables/index.md), and +To restore a backup, **you must also restore the GitLab secrets**. + +These include the database encryption key, [CI/CD variables](../ci/variables/index.md), and variables used for [two-factor authentication](../user/profile/account/two_factor_authentication.md). -If you fail to restore this encryption key file along with the application data -backup, users with two-factor authentication enabled and GitLab Runner -loses access to your GitLab server. + +Without the keys, [multiple issues occur](backup_restore.md#when-the-secrets-file-is-lost), +including loss of access by users with [two-factor authentication enabled](../user/profile/account/two_factor_authentication.md), +and GitLab Runners cannot log in. + +Restore: + +- `/etc/gitlab/gitlab-secrets.json` (Linux package) +- `/home/git/gitlab/.secret` (self-compiled installations) +- Rails secret (cloud-native GitLab) + - [This can be converted to the Linux package format](https://docs.gitlab.com/charts/installation/migration/helm_to_package.html), if required. You may also want to restore your previous `/etc/gitlab/gitlab.rb` (for Omnibus packages) or `/home/git/gitlab/config/gitlab.yml` (for installations from source) and @@ -155,9 +162,34 @@ the restore target directories are empty. For both these installation types, the backup tarball has to be available in the backup location (default location is `/var/opt/gitlab/backups`). -If you use Docker Swarm, [first disable the health check](#restore-gitlab-from-backup-using-docker-swarm). +### Restore for Helm chart installations + +The GitLab Helm chart uses the process documented in +[restoring a GitLab Helm chart installation](https://docs.gitlab.com/charts/backup-restore/restore.html#restoring-a-gitlab-installation) + +### Restore for Docker image installations + +If you're using [Docker Swarm](../install/docker.md#install-gitlab-using-docker-swarm-mode), +the container might restart during the restore process because Puma is shut down, +and so the container health check fails. To work around this problem, +temporarily disable the health check mechanism. -For Docker installations, the restore task can be run from host: +1. Edit `docker-compose.yml`: + + ```yaml + healthcheck: + disable: true + ``` + +1. Deploy the stack: + + ```shell + docker stack deploy --compose-file docker-compose.yml mystack + ``` + +For more information, see [issue 6846](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6846 "GitLab restore can fail owing to `gitlab-healthcheck`"). + +The restore task can be run from the host: ```shell # Stop the processes that are connected to the database @@ -177,31 +209,6 @@ docker restart <name of container> docker exec -it <name of container> gitlab-rake gitlab:check SANITIZE=true ``` -Users of GitLab 12.1 and earlier should use the command `gitlab-rake gitlab:backup:create` instead. - -WARNING: -`gitlab-rake gitlab:backup:restore` doesn't set the correct file system -permissions on your Registry directory. This is a [known issue](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/62759). -In GitLab 12.2 or later, you can use `gitlab-backup restore` to avoid this -issue. - -The GitLab Helm chart uses a different process, documented in -[restoring a GitLab Helm chart installation](https://gitlab.com/gitlab-org/charts/gitlab/blob/master/doc/backup-restore/restore.md). - -### Restore GitLab from backup using Docker Swarm - -Docker Swarm might restart the container during the restore process because Puma is shut down, -and so the container health check fails. To work around this problem, disable the health check -mechanism in Docker compose file: - -```yaml -healthcheck: - disable: true -``` - -Read more in issue #6846, -[GitLab restore can fail owing to `gitlab-healthcheck`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6846). - ## Restore for installation from source First, ensure your backup tar file is in the backup directory described in the diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md index 7756774e432..bb3f1f404b4 100644 --- a/doc/raketasks/user_management.md +++ b/doc/raketasks/user_management.md @@ -180,4 +180,4 @@ sudo /etc/init.d/gitlab start ## Related topics -- [Reset a user's password](../security/reset_user_password.md#use-a-rake-task). +- [Reset a user's password](../security/reset_user_password.md#use-a-rake-task) diff --git a/doc/security/email_verification.md b/doc/security/email_verification.md new file mode 100644 index 00000000000..1072475680c --- /dev/null +++ b/doc/security/email_verification.md @@ -0,0 +1,46 @@ +--- +stage: Anti-Abuse +group: Anti-Abuse +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Account email verification **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86352) in GitLab 15.2 [with a flag](../administration/feature_flags.md) named `require_email_verification`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `require_email_verification`. On GitLab.com, this feature is not available. + +Account email verification provides an additional layer of GitLab account security. +When certain conditions are met, an account is locked. If your account is locked, +you must verify your identity or reset your password to sign in to GitLab. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo, see [Require email verification - demo](https://www.youtube.com/watch?v=wU6BVEGB3Y0). + +## Accounts without two-factor authentication (2FA) + +An account is locked when either: + +- There are three or more failed sign-in attempts in 24 hours. +- A user attempts to sign in from a new IP address and the + `check_ip_address_for_email_verification` feature flag is enabled. + +A locked account without 2FA is not unlocked automatically. + +After a successful sign in, an email with a six-digit verification code is sent. +The verification code expires after 60 minutes. + +To unlock your account, sign in and enter the verification code. You can also +[reset your password](https://gitlab.com/users/password/new). + +## Accounts with 2FA + +An account is locked when there are five or more failed sign-in attempts in 10 minutes. + +Accounts with 2FA are automatically unlocked after 10 minutes. To unlock an account manually, +reset your password. + +## Related topics + +- [Locked and blocked account support](https://about.gitlab.com/handbook/support/workflows/reinstating-blocked-accounts.html) diff --git a/doc/security/index.md b/doc/security/index.md index e447b8ffe64..7a78461d717 100644 --- a/doc/security/index.md +++ b/doc/security/index.md @@ -2,7 +2,6 @@ stage: Manage group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false type: index --- diff --git a/doc/security/rate_limits.md b/doc/security/rate_limits.md index a9ccbccaa90..108510ae7c2 100644 --- a/doc/security/rate_limits.md +++ b/doc/security/rate_limits.md @@ -148,6 +148,14 @@ There is a rate limit for the endpoint `project/:id/jobs`, which is enforced to The **rate limit** is 600 calls per minute per authenticated user. +### AI action + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118010) in GitLab 16.0. + +There is a rate limit for the GraphQL `aiAction` mutation, which is enforced to prevent from abusing this endpoint. + +The **rate limit** is 20 calls per hour per authenticated user. + ## Troubleshooting ### Rack Attack is denylisting the load balancer diff --git a/doc/security/two_factor_authentication.md b/doc/security/two_factor_authentication.md index a419ca4f3eb..9ac799610bb 100644 --- a/doc/security/two_factor_authentication.md +++ b/doc/security/two_factor_authentication.md @@ -11,7 +11,10 @@ Two-factor authentication (2FA) provides an additional level of security to your users' GitLab account. When enabled, users are prompted for a code generated by an application in addition to supplying their username and password to sign in. -Read more about [two-factor authentication (2FA)](../user/profile/account/two_factor_authentication.md) +NOTE: +If you are [using and enforcing SSO](../user/group/saml_sso/index.md#sso-enforcement), you might already be enforcing 2FA on the identity provider (IDP) side. Enforcing 2FA on GitLab as well might be unnecessary. + +Read more about [two-factor authentication (2FA)](../user/profile/account/two_factor_authentication.md). ## Enforce 2FA for all users **(FREE SELF)** diff --git a/doc/security/user_email_confirmation.md b/doc/security/user_email_confirmation.md index c3f19c92f91..962f6f0fd61 100644 --- a/doc/security/user_email_confirmation.md +++ b/doc/security/user_email_confirmation.md @@ -5,7 +5,7 @@ group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# User email confirmation at sign-up **(FREE SELF)** +# Make new users confirm email **(FREE SELF)** GitLab can be configured to require confirmation of a user's email address when the user signs up. When this setting is enabled, the user is unable to sign in until diff --git a/doc/security/user_file_uploads.md b/doc/security/user_file_uploads.md index 80f4b1a8a2a..498245c4efb 100644 --- a/doc/security/user_file_uploads.md +++ b/doc/security/user_file_uploads.md @@ -7,20 +7,27 @@ info: To determine the technical writer assigned to the Stage/Group associated w # User file uploads **(FREE)** +Users can upload files to: + +- Issues or merge requests in a project. +- Epics in a group. + +GitLab generates direct URLs for these images with a random 32-character ID to prevent unauthorized users from guessing the URLs. This randomization offers some security for images containing sensitive information. + +## Access control for uploaded files + > - Enforced authorization checks [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80117) in GitLab 14.8 [with a flag](../administration/feature_flags.md) named `enforce_auth_checks_on_uploads`. Disabled by default. > - Enforced authorization checks became [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352291) in GitLab 15.3. Feature flag `enforce_auth_checks_on_uploads` removed. > - Project settings in the user interface [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88567) in GitLab 15.3. -In private or internal projects, GitLab restricts access to uploaded files (such as PDFs) -to authenticated users only. By default, image files are not subject to the same -restriction, and unauthenticated users can use the URL to view the -file. If you enable authorization checks for all media files, images -receive the same protection and are viewable only by authenticated users. +Access to non-image files uploaded to: -Users can upload files to issues, merge requests, or comments in a project. Direct URLs -to these images in GitLab contain a random 32-character ID to help prevent -unauthorized users from guessing image URLs. This randomization provides some protection -if an image contains sensitive information. +- Issues or merge requests is determined by the project visibility. +- Group epics is determined by the group visibility. + +For public projects or groups, anyone can access these files through the direct attachment URL, even if the issue, merge request, or epic is confidential. +For private and internal projects, GitLab ensures only authenticated project members can access non-image file uploads, such as PDFs. +By default, image files do not have the same restriction, and anyone can view them using the URL. To protect image files, [enable authorization checks for all media files](#enable-authorization-checks-for-all-media-files), making them viewable only by authenticated users. Authentication checks for images can cause display issues in the body of notification emails. Emails are frequently read from clients (such as Outlook, Apple Mail, or your mobile device) @@ -29,8 +36,9 @@ the client is not authorized to GitLab. ## Enable authorization checks for all media files -Non-image attachments (including PDFs) always require authentication to be viewed. -You can use this setting to extend this protection to image files. +Only authenticated project members can view non-image attachments (including PDFs) in private and internal projects. + +To apply authentication requirements to image files in private or internal projects: Prerequisite: @@ -43,7 +51,9 @@ To configure authentication settings for all media files: 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. Scroll to **Project visibility** and select **Require authentication to view media files**. - You cannot select this option for projects with **Public** visibility. + +NOTE: +You cannot select this option for public projects. ## Delete uploaded files @@ -69,6 +79,7 @@ mutation{ ``` Project members that do not have the Owner or Maintainer role cannot access this GraphQL endpoint. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md index 6164fb794c2..81e7878c9df 100644 --- a/doc/security/webhooks.md +++ b/doc/security/webhooks.md @@ -152,15 +152,10 @@ If you can't enable this setting, do one of the following: ### Public runner releases URL is blocked Most GitLab instances have their `public_runner_releases_url` set to -`https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-runner/releases`. -You can't change or disable this setting in the Admin Area, which can prevent you from [filtering requests](#filter-requests). +`https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-runner/releases`, +which can prevent you from [filtering requests](#filter-requests). -To enable the setting, use the [Rails console](../administration/operations/rails_console.md) to set `public_runner_releases_url` to the instance host: - -```ruby -current_settings = ApplicationSetting.find_or_create_without_cache; -ApplicationSettings::UpdateService.new(current_settings, nil, public_runner_releases_url: Gitlab.config.gitlab.base_url).execute -``` +To resolve this issue, [configure GitLab to no longer fetch runner release version data from GitLab.com](../user/admin_area/settings/continuous_integration.md#disable-runner-version-management). ### GitLab subscription management is blocked diff --git a/doc/subscriptions/bronze_starter.md b/doc/subscriptions/bronze_starter.md index accc0627a41..b076987c0ae 100644 --- a/doc/subscriptions/bronze_starter.md +++ b/doc/subscriptions/bronze_starter.md @@ -18,7 +18,7 @@ the tiers are no longer mentioned in GitLab documentation: - [Activate GitLab EE with a license](../user/admin_area/license.md) - [Add a help message to the sign-in page](../user/admin_area/settings/help_page.md#add-a-help-message-to-the-sign-in-page) - [Burndown and burnup charts](../user/project/milestones/burndown_and_burnup_charts.md) in the [Milestone View](../user/project/milestones/index.md#burndown-charts), -- [Code owners](../user/project/code_owners.md) +- [Code owners](../user/project/codeowners/index.md) - Description templates: - [Setting a default template for merge requests and issues](../user/project/description_templates.md#set-a-default-template-for-merge-requests-and-issues) - [Email from GitLab](../user/admin_area/email_from_gitlab.md) @@ -71,7 +71,7 @@ the tiers are no longer mentioned in GitLab documentation: - [Code Owners as eligible approvers](../user/project/merge_requests/approvals/rules.md#code-owners-as-eligible-approvers) - [Approval rules](../user/project/merge_requests/approvals/rules.md) features - [Restricting push and merge access to certain users](../user/project/protected_branches.md) - - [Visual Reviews](../ci/review_apps/index.md#visual-reviews) + - [Visual Reviews (deprecated)](../ci/review_apps/index.md#visual-reviews-deprecated) - Metrics and analytics: - [Contribution Analytics](../user/group/contribution_analytics/index.md) - [Merge Request Analytics](../user/analytics/merge_request_analytics.md) diff --git a/doc/subscriptions/choosing_subscription.md b/doc/subscriptions/choosing_subscription.md new file mode 100644 index 00000000000..07b04c83bd7 --- /dev/null +++ b/doc/subscriptions/choosing_subscription.md @@ -0,0 +1,61 @@ +--- +stage: Fulfillment +group: Purchase +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Choosing a GitLab subscription + +To choose the right GitLab subscription, choose an offering and a tier. + +## Choose a subscription + +Choose which GitLab subscription suits your needs: + +- [GitLab SaaS](gitlab_com/index.md): The GitLab software-as-a-service offering. + You don't need to install anything to use GitLab SaaS, you only need to + [sign up](https://gitlab.com/users/sign_up) and start using GitLab straight away. +- [GitLab Dedicated](gitlab_dedicated/index.md): A single-tenant SaaS service for highly regulated and large enterprises. +- [GitLab self-managed](self_managed/index.md): Install, administer, and maintain + your own GitLab instance. + +On a GitLab self-managed instance, a GitLab subscription provides the same set of +features for _all_ users. On GitLab SaaS, you can apply a subscription to a group +namespace. You cannot apply a subscription to a personal namespace. + +NOTE: +Subscriptions cannot be transferred between GitLab SaaS and GitLab self-managed. +A new subscription must be purchased and applied as needed. + +## Choose a GitLab tier + +Pricing is [tier-based](https://about.gitlab.com/pricing/), allowing you to choose +the features which fit your budget. For information on what features are available +at each tier for each product, see the [GitLab self-managed feature comparison](https://about.gitlab.com/pricing/feature-comparison/). + +## Find your subscription + +The following chart should help you determine your subscription model. Select +the list item to go to the respective help page. + +```mermaid +graph TD + +A(Is your user account on GitLab.com?) +A --> B(Yes) +A --> C(No) +B --> D(fa:fa-link View your subscription on GitLab.com) +C --> E(fa:fa-link View your self-hosted subscription) + +click D "./gitlab_com/index.html#view-your-gitlabcom-subscription" +click E "./self_managed/index.html#view-your-subscription" +``` + +## Contact Support + +- See the tiers of [GitLab Support](https://about.gitlab.com/support/). +- [Submit a request](https://support.gitlab.com/hc/en-us/requests/new) through the Support Portal. + +We also encourage all users to search our project trackers for known issues and existing feature requests in the [GitLab project](https://gitlab.com/gitlab-org/gitlab/-/issues/). + +These issues are the best avenue for getting updates on specific product plans and for communicating directly with the relevant GitLab team members. diff --git a/doc/subscriptions/community_programs.md b/doc/subscriptions/community_programs.md new file mode 100644 index 00000000000..05c5d66b0e5 --- /dev/null +++ b/doc/subscriptions/community_programs.md @@ -0,0 +1,80 @@ +--- +stage: Fulfillment +group: Purchase +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Community programs + +GitLab provides the following community program subscriptions. + +## GitLab for Education + +For qualifying non-profit educational institutions, the [GitLab for Education Program](https://about.gitlab.com/solutions/education/) provides GitLab Ultimate, plus 50,000 CI/CD minutes per month. The subscription granted under GitLab for Education can only be used for instructional use or non-commercial academic research. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Education Program page](https://about.gitlab.com/solutions/education/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/education-program/). + +## GitLab for Open Source + +For qualifying open source projects, the [GitLab for Open Source Program](https://about.gitlab.com/solutions/open-source/) provides GitLab Ultimate, plus 50,000 CI/CD minutes per month. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Open Source Program page](https://about.gitlab.com/solutions/open-source/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/opensource-program/). + +### Meeting GitLab for Open Source Program requirements + +To meet GitLab for Open Source Program requirements, first add an OSI-approved open source license to all projects in your namespace. + +To add a license to a project: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the overview page, select **Add LICENSE**. If the license you want is not available as a license template, manually copy the entire, unaltered [text of your chosen license](https://opensource.org/licenses/) into the `LICENSE` file. Note that GitLab defaults to **All rights reserved** if users do not perform this action. + + + +Applicants must add the correct license to each project in their respective groups or namespaces. When you're sure you're using OSI-approved licenses for your projects, you can take your screenshots. + +NOTE: +GitLab for Open Source Program benefits apply to an entire GitLab namespace. To qualify for the GitLab for Open Source Program, all projects in an applicant's namespace must meet program requirements. Applicants submit materials related to one project in the applying namespace, and the open source program team uses that project to verify eligibility of the entire namespace. + +### Verification for Open Source Program + +Next, take screenshots of your project to confirm that project's eligibility. You must upload three screenshots: + +- [OSI-approved license overview](#screenshot-1-license-overview) +- [OSI-approved license contents](#screenshot-2-license-contents) +- [Publicly visible settings](#screenshot-3-publicly-visible-settings) + +NOTE: +Benefits of the GitLab Open Source Program apply to all projects in a GitLab namespace. All projects in an eligible namespace must meet program requirements. + +#### Screenshot 1: License overview + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select your project avatar. If you haven't specified an avatar for your project, the avatar displays as a single letter. +1. Take a screenshot of the project overview that clearly displays the license you've chosen for your project. + + + +#### Screenshot 2: License contents + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository** and locate the project's `LICENSE` file. +1. Take a screenshot of the contents of the file. Make sure the screenshot includes the title of the license. + + + +#### Screenshot 3: Publicly visible settings + +To be eligible for the GitLab Open Source Program, projects must be publicly visible. To check your project's public visibility settings: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. From the left sidebar, select **Settings > General**. +1. Expand **Visibility, project features, permissions**. +1. From the **Project visibility** dropdown list, select **Public**. +1. Select the **Users can request access** checkbox. +1. Take a screenshot of this view. Include as much of the publicly visible settings as possible. Make sure to include your project's name in the upper-left of the screenshot. + + + +NOTE: +Exceptions to this public visibility requirement apply in select circumstances (for example, in cases where a project in an applicant's namespace may hold sensitive data). Email `opensource@gitlab.com` with details of your use case to request written permission for exceptions. + +## GitLab for Startups + +For qualifying startups, the [GitLab for Startups](https://about.gitlab.com/solutions/startups/) program provides GitLab Ultimate, plus 50,000 CI/CD minutes per month for 12 months. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Startups Program page](https://about.gitlab.com/solutions/startups/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/startups-program/). diff --git a/doc/subscriptions/customers_portal.md b/doc/subscriptions/customers_portal.md new file mode 100644 index 00000000000..3f493f255dc --- /dev/null +++ b/doc/subscriptions/customers_portal.md @@ -0,0 +1,109 @@ +--- +stage: Fulfillment +group: Purchase +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# The Customers Portal + +For some management tasks for your subscription and account, you use the Customers Portal. + +The Customers Portal is available to customers who purchased their +subscription from GitLab. If you made your purchase through a partner or +reseller, contact them directly for assistance with your subscription. + +You can also specifically manage your [GitLab SaaS subscription](gitlab_com/index.md) +or [self-managed subscription](self_managed/index.md). + +## Change account owner information + +Account owner personal details are used on invoices. The account owner email address is used for the Customers Portal legacy login and license-related email. + +If you have registered a Customers Portal account through a GitLab.com account, the GitLab.com account is used for login. + +To change account owner information, including name, billing address, and email address: + +1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Select **My account > Account details**. +1. Expand the **Personal details** section. +1. Edit the personal details. +1. Select **Save changes**. + +If you want to transfer ownership of the Customers Portal account +to another person, after you enter that person's personal details, you must also: + +- [Change the Customers Portal account password](#change-customers-portal-account-password). +- [Change the linked GitLab.com account](#change-the-linked-account), if you have one linked. + +## Change your company details + +To change your company details, including company name and VAT number: + +1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Select **My account > Account details**. +1. Expand the **Company details** section. +1. Edit the company details. +1. Select **Save changes**. + +## Change your payment method + +Purchases in the Customers Portal require a credit card on record as a payment method. You can add +multiple credit cards to your account, so that purchases for different products are charged to the +correct card. + +If you would like to use an alternative method to pay, please +[contact our Sales team](https://about.gitlab.com/sales/). + +To change your payment method: + +1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Select **My account > Payment methods**. +1. **Edit** an existing payment method's information or **Add new payment method**. +1. Select **Save Changes**. + +### Set a default payment method + +Automatic renewal of a subscription is charged to your default payment method. To mark a payment +method as the default: + +1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Select **My account > Payment methods**. +1. **Edit** the selected payment method and check the **Make default payment method** checkbox. +1. Select **Save Changes**. + +## Link a GitLab.com account + +Follow this guideline if you have a legacy Customers Portal account and use an email and password to log in. + +To link a GitLab.com account to your Customers Portal account: + +1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in?legacy=true) using email and password. +1. On the Customers Portal page, select **My account > Account details**. +1. Under **Your GitLab.com account**, select **Link account**. +1. Log in to the [GitLab.com](https://gitlab.com/users/sign_in) account you want to link to the Customers Portal account. + +## Change the linked account + +Customers are required to use their GitLab.com account to register for a new Customers Portal account. + +If you have a legacy Customers Portal account that is not linked to a GitLab.com account, you may still [sign in](https://customers.gitlab.com/customers/sign_in?legacy=true) using an email and password. However, you should [create](https://gitlab.com/users/sign_up) and [link a GitLab.com account](#change-the-linked-account) to ensure continued access to the Customers Portal. + +Customers of resellers do not have access to this portal and should contact their reseller for any +changes to their subscription. + +To change the GitLab.com account linked to your Customers Portal account: + +1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. In a separate browser tab, go to [GitLab.com](https://gitlab.com/users/sign_in) and ensure you are not logged in. +1. On the Customers Portal page, select **My account > Account details**. +1. Under **Your GitLab.com account**, select **Change linked account**. +1. Log in to the [GitLab.com](https://gitlab.com/users/sign_in) account you want to link to the Customers Portal account. + +## Change Customers Portal account password + +To change the password for this customers portal account: + +1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Select the **My account** dropdown list and select **Account details**. +1. Make the required changes to the **Your password** section. +1. Select **Save changes**. diff --git a/doc/subscriptions/gitlab_com/index.md b/doc/subscriptions/gitlab_com/index.md index cd36e426852..4d11a6fffc8 100644 --- a/doc/subscriptions/gitlab_com/index.md +++ b/doc/subscriptions/gitlab_com/index.md @@ -237,7 +237,7 @@ amounts at which the alert displays. To change the namespace linked to a subscription: 1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in) with a - [linked](../index.md#link-a-gitlabcom-account) GitLab.com account. + [linked](../customers_portal.md#link-a-gitlabcom-account) GitLab.com account. 1. Go to the **Manage Purchases** page. 1. Select **Change linked namespace**. 1. Select the desired group from the **This subscription is for** dropdown list. For a group to appear here, you must have the Owner role for that group. @@ -312,7 +312,7 @@ For details on upgrading your subscription tier, see ### Automatic subscription renewal When a subscription is set to auto-renew, it renews automatically on the -expiration date without a gap in available service. Subscriptions purchased through Customers Portal or GitLab.com are set to auto-renew by default. The number of seats is adjusted to fit the [number of billable users in your group](#view-seat-usage) at the time of renewal. You can view and download your renewal invoice on the Customers Portal [View invoices](https://customers.gitlab.com/receipts) page. If your account has a [saved credit card](../index.md#change-your-payment-method), the card is charged for the invoice amount. If we are unable to process a payment or the auto-renewal fails for any other reason, you have 14 days to renew your subscription, after which your access is downgraded. +expiration date without a gap in available service. Subscriptions purchased through Customers Portal or GitLab.com are set to auto-renew by default. The number of seats is adjusted to fit the [number of billable users in your group](#view-seat-usage) at the time of renewal. You can view and download your renewal invoice on the Customers Portal [View invoices](https://customers.gitlab.com/receipts) page. If your account has a [saved credit card](../customers_portal.md#change-your-payment-method), the card is charged for the invoice amount. If we are unable to process a payment or the auto-renewal fails for any other reason, you have 14 days to renew your subscription, after which your access is downgraded. #### Email notifications diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md index f1a960a6aa9..9f329700c74 100644 --- a/doc/subscriptions/index.md +++ b/doc/subscriptions/index.md @@ -5,259 +5,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index, reference --- -# GitLab subscription **(PREMIUM)** - -GitLab offers tiers of features. Your subscription determines which tier you -have access to. Subscriptions are valid for 12 months. - -GitLab provides special subscriptions to participants in: - -- [Education](#gitlab-for-education) -- [Open Source](#gitlab-for-open-source) - -## Choose a GitLab subscription - -When choosing a subscription, there are two factors to consider: - -- [GitLab SaaS or GitLab self-managed](#choose-between-gitlab-saas-or-gitlab-self-managed) -- [GitLab tier](#choose-a-gitlab-tier) - -### Choose between GitLab SaaS or GitLab self-managed - -There are some differences in how a subscription applies, depending if you use -GitLab SaaS or GitLab self-managed: - -- [GitLab SaaS](gitlab_com/index.md): The GitLab software-as-a-service offering. - You don't need to install anything to use GitLab SaaS, you only need to - [sign up](https://gitlab.com/users/sign_up) and start using GitLab straight away. -- [GitLab Dedicated](gitlab_dedicated/index.md): a single-tenant SaaS service for highly regulated and large enterprises. -- [GitLab self-managed](self_managed/index.md): Install, administer, and maintain - your own GitLab instance. - -On a GitLab self-managed instance, a GitLab subscription provides the same set of -features for _all_ users. On GitLab SaaS, you can apply a subscription to a group -namespace. You cannot apply a subscription to a personal namespace. - -NOTE: -Subscriptions cannot be transferred between GitLab SaaS and GitLab self-managed. -A new subscription must be purchased and applied as needed. - -### Choose a GitLab tier - -Pricing is [tier-based](https://about.gitlab.com/pricing/), allowing you to choose -the features which fit your budget. For information on what features are available -at each tier for each product, see: [GitLab self-managed feature comparison](https://about.gitlab.com/pricing/feature-comparison/) - -## Find your subscription - -The following chart should help you determine your subscription model. Select -the list item to go to the respective help page. - -```mermaid -graph TD - -A(Is your user account on GitLab.com?) -A --> B(Yes) -A --> C(No) -B --> D(fa:fa-link View your subscription on GitLab.com) -C --> E(fa:fa-link View your self-hosted subscription) - -click D "./gitlab_com/index.html#view-your-gitlabcom-subscription" -click E "./self_managed/index.html#view-your-subscription" -``` - -## Customers Portal - -With the [Customers Portal](https://customers.gitlab.com/) you can: - -- [Change account owner information](#change-account-owner-information) -- [Change your company details](#change-your-company-details) -- [Change your payment method](#change-your-payment-method) -- [Link a GitLab.com account](#link-a-gitlabcom-account) -- [Change the linked account](#change-the-linked-account) -- [Change the namespace the subscription is linked to](gitlab_com/index.md#change-the-linked-namespace) -- [Change customers portal account password](#change-customers-portal-account-password) - -The Customers Portal is available only to customers who purchased their -subscription from GitLab. If you made your purchase through a partner or -reseller, you must contact them directly for assistance with your subscription. - -### Change account owner information - -Account owner personal details are used on invoices. The account owner email address is used for the Customers Portal legacy login and license-related email. - -If you have registered a Customers Portal account through a GitLab.com account, the GitLab.com account is used for login. - -To change account owner information, including name, billing address, and email address: - -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). -1. Select **My account > Account details**. -1. Expand the **Personal details** section. -1. Edit the personal details. -1. Select **Save changes**. - -If you want to transfer ownership of the Customers Portal account -to another person, after you enter that person's personal details, you must also: - -- [Change the Customers Portal account password](#change-customers-portal-account-password). -- [Change the linked GitLab.com account](#change-the-linked-account), if you have one linked. - -### Change your company details - -To change your company details, including company name and VAT number: - -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). -1. Select **My account > Account details**. -1. Expand the **Company details** section. -1. Edit the company details. -1. Select **Save changes**. - -### Change your payment method - -Purchases in the Customers Portal require a credit card on record as a payment method. You can add -multiple credit cards to your account, so that purchases for different products are charged to the -correct card. - -If you would like to use an alternative method to pay, please -[contact our Sales team](https://about.gitlab.com/sales/). - -To change your payment method: - -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). -1. Select **My account > Payment methods**. -1. **Edit** an existing payment method's information or **Add new payment method**. -1. Select **Save Changes**. - -#### Set a default payment method - -Automatic renewal of a subscription is charged to your default payment method. To mark a payment -method as the default: - -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). -1. Select **My account > Payment methods**. -1. **Edit** the selected payment method and check the **Make default payment method** checkbox. -1. Select **Save Changes**. - -### Link a GitLab.com account - -Follow this guideline if you have a legacy Customers Portal account and use an email and password to log in. - -To link a GitLab.com account to your Customers Portal account: - -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in?legacy=true) using email and password. -1. On the Customers Portal page, select **My account > Account details**. -1. Under **Your GitLab.com account**, select **Link account**. -1. Log in to the [GitLab.com](https://gitlab.com/users/sign_in) account you want to link to the Customers Portal account. - -### Change the linked account - -To change the GitLab.com account linked to your Customers Portal account: - -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). -1. In a separate browser tab, go to [GitLab.com](https://gitlab.com/users/sign_in) and ensure you are not logged in. -1. On the Customers Portal page, select **My account > Account details**. -1. Under **Your GitLab.com account**, select **Change linked account**. -1. Log in to the [GitLab.com](https://gitlab.com/users/sign_in) account you want to link to the Customers Portal account. - -### Change Customers Portal account password - -To change the password for this customers portal account: - -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). -1. Select the **My account** dropdown list and select **Account details**. -1. Make the required changes to the **Your password** section. -1. Select **Save changes**. - -## Community program subscriptions - -### GitLab for Education - -For qualifying non-profit educational institutions, the [GitLab for Education Program](https://about.gitlab.com/solutions/education/) provides GitLab Ultimate, plus 50,000 CI/CD minutes per month. The subscription granted under GitLab for Education can only be used for instructional use or non-commercial academic research. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Education Program page](https://about.gitlab.com/solutions/education/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/education-program/). - -### GitLab for Open Source - -For qualifying open source projects, the [GitLab for Open Source Program](https://about.gitlab.com/solutions/open-source/) provides GitLab Ultimate, plus 50,000 CI/CD minutes per month. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Open Source Program page](https://about.gitlab.com/solutions/open-source/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/opensource-program/). - -#### Meeting GitLab for Open Source Program requirements - -To meet GitLab for Open Source Program requirements, first add an OSI-approved open source license to all projects in your namespace. - -To add a license to a project: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the overview page, select **Add LICENSE**. If the license you want is not available as a license template, manually copy the entire, unaltered [text of your chosen license](https://opensource.org/licenses/) into the `LICENSE` file. Note that GitLab defaults to **All rights reserved** if users do not perform this action. - - - -Applicants must add the correct license to each project in their respective groups or namespaces. When you're sure you're using OSI-approved licenses for your projects, you can take your screenshots. - -NOTE: -GitLab for Open Source Program benefits apply to an entire GitLab namespace. To qualify for the GitLab for Open Source Program, all projects in an applicant's namespace must meet program requirements. Applicants submit materials related to one project in the applying namespace, and the open source program team uses that project to verify eligibility of the entire namespace. - -#### Verification for Open Source Program - -Next, take screenshots of your project to confirm that project's eligibility. You must upload three screenshots: - -- [OSI-approved license overview](#screenshot-1-license-overview) -- [OSI-approved license contents](#screenshot-2-license-contents) -- [Publicly visible settings](#screenshot-3-publicly-visible-settings) - -NOTE: -Benefits of the GitLab Open Source Program apply to all projects in a GitLab namespace. All projects in an eligible namespace must meet program requirements. - -##### Screenshot 1: License overview - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select your project avatar. If you haven't specified an avatar for your project, the avatar displays as a single letter. -1. Take a screenshot of the project overview that clearly displays the license you've chosen for your project. - - - -##### Screenshot 2: License contents - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Repository** and locate the project's `LICENSE` file. -1. Take a screenshot of the contents of the file. Make sure the screenshot includes the title of the license. - - - -##### Screenshot 3: Publicly visible settings - -To be eligible for the GitLab Open Source Program, projects must be publicly visible. To check your project's public visibility settings: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. From the left sidebar, select **Settings > General**. -1. Expand **Visibility, project features, permissions**. -1. From the **Project visibility** dropdown list, select **Public**. -1. Select the **Users can request access** checkbox. -1. Take a screenshot of this view. Include as much of the publicly visible settings as possible. Make sure to include your project's name in the upper-left of the screenshot. - - - -NOTE: -Exceptions to this public visibility requirement apply in select circumstances (for example, in cases where a project in an applicant's namespace may hold sensitive data). Email `opensource@gitlab.com` with details of your use case to request written permission for exceptions. - -### GitLab for Startups - -For qualifying startups, the [GitLab for Startups](https://about.gitlab.com/solutions/startups/) program provides GitLab Ultimate, plus 50,000 CI/CD minutes per month for 12 months. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Startups Program page](https://about.gitlab.com/solutions/startups/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/startups-program/). - -## Contact Support - -- See the tiers of [GitLab Support](https://about.gitlab.com/support/). -- [Submit a request](https://support.gitlab.com/hc/en-us/requests/new) through the Support Portal. - -We also encourage all users to search our project trackers for known issues and existing feature requests in the [GitLab project](https://gitlab.com/gitlab-org/gitlab/-/issues/). - -These issues are the best avenue for getting updates on specific product plans and for communicating directly with the relevant GitLab team members. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +# Subscribe to GitLab **(PREMIUM)** + +Choose and manage the subscription that's right for you and your organization. + +- [Choose a subscription](choosing_subscription.md) +- [Compare self-managed to SaaS](../install/migrate/compare_sm_to_saas.md) +- [GitLab SaaS](gitlab_com/index.md) +- [GitLab self-managed](self_managed/index.md) +- [GitLab Dedicated](gitlab_dedicated/index.md) +- [Community programs](community_programs.md) +- [Customers Portal](customers_portal.md) +- [Quarterly reconciliation](quarterly_reconciliation.md) +- [Storage usage quota](../user/usage_quotas.md) +- [CI/CD minutes quota](../ci/pipelines/cicd_minutes.md) +- [Features available to Starter and Bronze subscribers](../subscriptions/bronze_starter.md) diff --git a/doc/subscriptions/quarterly_reconciliation.md b/doc/subscriptions/quarterly_reconciliation.md index 126a34334c4..16f1828f2c3 100644 --- a/doc/subscriptions/quarterly_reconciliation.md +++ b/doc/subscriptions/quarterly_reconciliation.md @@ -91,7 +91,7 @@ sent and subject to your payment terms. - You purchased your subscription from a reseller or another channel partner. - You purchased a multi-year subscription. - You purchased your subscription with a purchasing order. -- You are a pubic sector customer. +- You are a public sector customer. - You have an offline environment and used a license file to activate your subscription. - You are enrolled in a program that provides a free tier such as the GitLab for Education, GitLab for Open Source Program, or GitLab for Startups. diff --git a/doc/subscriptions/self_managed/index.md b/doc/subscriptions/self_managed/index.md index 52a39fb3f49..b5faf3bb0f3 100644 --- a/doc/subscriptions/self_managed/index.md +++ b/doc/subscriptions/self_managed/index.md @@ -7,39 +7,22 @@ type: index, reference # GitLab self-managed subscription **(PREMIUM SELF)** -You can install, administer, and maintain your own GitLab instance. +After you subscribe to GitLab, you can manage the details of your self-managed subscription. -This page covers the details of your GitLab self-managed subscription. +## Obtain a self-managed subscription -GitLab subscription management requires access to the Customers Portal. +To subscribe to GitLab for a GitLab self-managed installation: -## Customers Portal - -GitLab provides the [Customers Portal](../index.md#customers-portal) where you can -manage your subscriptions and your account details. - -Customers are required to use their GitLab.com account to register for a new Customers Portal account. - -If you have a legacy Customers Portal account that is not linked to a GitLab.com account, you may still [sign in](https://customers.gitlab.com/customers/sign_in?legacy=true) using an email and password. However, you should [create](https://gitlab.com/users/sign_up) and [link a GitLab.com account](../index.md#change-the-linked-account) to ensure continued access to the Customers Portal. - -Customers of resellers do not have access to this portal and should contact their reseller for any -changes to their subscription. - -## Subscription - -The cost of a GitLab self-managed subscription is determined by the following: - -- [GitLab tier](https://about.gitlab.com/pricing/) -- [Subscription seats](#subscription-seats) - -## Choose a GitLab tier +1. Go to the [Customers Portal](https://customers.gitlab.com/) and purchase a GitLab self-managed plan. +1. After purchase, an activation code is sent to the email address associated with the Customers Portal account. + You must [add this code to your GitLab instance](../../user/admin_area/license.md). -Pricing is [tier-based](https://about.gitlab.com/pricing/), so you can choose -the features that fit your budget. For information on the features available -for each tier, see the -[GitLab self-managed feature comparison](https://about.gitlab.com/pricing/feature-comparison/). +NOTE: +If you're purchasing a subscription for an existing **Free** GitLab self-managed +instance, ensure you're purchasing enough seats to +[cover your users](../../user/admin_area/index.md#administering-users). -## Subscription seats +### Subscription seats A GitLab self-managed subscription uses a hybrid model. You pay for a subscription according to the [maximum number](#maximum-users) of users enabled during the subscription period. @@ -49,7 +32,7 @@ simultaneous users in the GitLab self-managed installation is checked each quart If an instance is unable to generate a quarterly usage report, the existing [true up model](#users-over-subscription) is used. Prorated charges are not possible without a quarterly usage report. -### View user totals +## View user totals You can view users for your license and determine if you've gone over your subscription. @@ -58,7 +41,7 @@ You can view users for your license and determine if you've gone over your subsc The lists of users are displayed. -#### Billable users +### Billable users A _billable user_ counts against the number of subscription seats. Every user is considered a billable user, with the following exceptions: @@ -67,7 +50,8 @@ billable user, with the following exceptions: [blocked users](../../user/admin_area/moderate_users.md#block-a-user) don't count as billable users in the current subscription. When they are either deactivated or blocked they release a _billable user_ seat. However, they may count toward overages in the subscribed seat count. - Users who are [pending approval](../../user/admin_area/moderate_users.md#users-pending-approval). -- Members with the [Guest role on an Ultimate subscription](#free-guest-users). +- Users with only the [Minimal Access role](../../user/permissions.md#users-with-minimal-access) on self-managed Ultimate subscriptions or any GitLab.com subscriptions. +- Users with only the [Guest or Minimal Access roles on an Ultimate subscription](#free-guest-users). - Users without project or group memberships on an Ultimate subscription. - GitLab-created service accounts: - [Ghost User](../../user/profile/account/delete_account.md#associated-records). @@ -75,14 +59,15 @@ billable user, with the following exceptions: - [Support Bot](../../user/project/service_desk.md#support-bot-user). - [Bot users for projects](../../user/project/settings/project_access_tokens.md#bot-users-for-projects). - [Bot users for groups](../../user/group/settings/group_access_tokens.md#bot-users-for-groups). + - Other [internal users](../../development/internal_users.md#internal-users). **Billable users** as reported in the `/admin` section is updated once per day. -#### Maximum users +### Maximum users The number of _maximum users_ reflects the highest number of billable users for the current license period. -#### Users over subscription +### Users over subscription The number of _users over subscription_ shows how many users are in excess of the number allowed by the subscription. This number reflects the current subscription period. @@ -101,7 +86,7 @@ If you add more users to your GitLab instance than you are licensed for, payment If you do not add these users during the renewal process, your license key will not work. -#### Free Guest users **(ULTIMATE)** +### Free Guest users **(ULTIMATE)** In the **Ultimate** tier, users who are assigned the Guest role do not consume a seat. The user must not be assigned any other role, anywhere in the instance. @@ -117,7 +102,7 @@ If a user creates a project, they are assigned the Maintainer or Owner role. To prevent a user from creating projects, as an administrator, you can mark the user as [external](../../user/admin_area/external_users.md). -### Tips for managing users and subscription seats +## Tips for managing users and subscription seats Managing the number of users against the number of subscription seats can be a challenge: @@ -237,19 +222,6 @@ You can manually synchronize your subscription details at any time. A job is queued. When the job finishes, the subscription details are updated. -## Obtain a subscription - -To subscribe to GitLab through a GitLab self-managed installation: - -1. Go to the [Customers Portal](https://customers.gitlab.com/) and purchase a GitLab self-managed plan. -1. After purchase, an activation code is sent to the email address associated with the Customers Portal account. - You must [add this code to your GitLab instance](../../user/admin_area/license.md). - -NOTE: -If you're purchasing a subscription for an existing **Free** GitLab self-managed -instance, ensure you're purchasing enough seats to -[cover your users](../../user/admin_area/index.md#administering-users). - ## View your subscription If you are an administrator, you can view the status of your subscription: @@ -404,7 +376,7 @@ Before auto-renewal you should [prepare for the renewal](#prepare-for-renewal-by you must have enabled the [synchronization of subscription data](#subscription-data-synchronization). You can view and download your renewal invoice on the Customers Portal -[View invoices](https://customers.gitlab.com/receipts) page. If your account has a [saved credit card](../index.md#change-your-payment-method), the card is charged for the invoice amount. If we are unable to process a payment or the auto-renewal fails for any other reason, you have 14 days to renew your subscription, after which your GitLab tier is downgraded. +[View invoices](https://customers.gitlab.com/receipts) page. If your account has a [saved credit card](../customers_portal.md#change-your-payment-method), the card is charged for the invoice amount. If we are unable to process a payment or the auto-renewal fails for any other reason, you have 14 days to renew your subscription, after which your GitLab tier is downgraded. #### Email notifications diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index 55a58377b76..a7611784284 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Authentication **(FREE)** -This page gathers all the resources for the topic **Authentication** within GitLab. +This page gathers all the resources for the topic **Authentication** in GitLab. ## GitLab users @@ -17,7 +17,7 @@ This page gathers all the resources for the topic **Authentication** within GitL - [Support for Universal 2nd Factor Authentication - YubiKeys](https://about.gitlab.com/blog/2016/06/22/gitlab-adds-support-for-u2f/) - [Security Webcast with Yubico](https://about.gitlab.com/blog/2016/08/31/gitlab-and-yubico-security-webcast/) - **Integrations:** - - [GitLab as OAuth2 authentication service provider](../../integration/oauth_provider.md) + - [GitLab as OAuth 2.0 authentication service provider](../../integration/oauth_provider.md) - [GitLab as OpenID Connect identity provider](../../integration/openid_connect_provider.md) ## GitLab administrators @@ -38,7 +38,7 @@ This page gathers all the resources for the topic **Authentication** within GitL ## API -- [OAuth 2 Tokens](../../api/rest/index.md#oauth2-tokens) +- [OAuth 2.0 tokens](../../api/rest/index.md#oauth-20-tokens) - [Personal access tokens](../../api/rest/index.md#personalprojectgroup-access-tokens) - [Project access tokens](../../api/rest/index.md#personalprojectgroup-access-tokens) - [Group access tokens](../../api/rest/index.md#personalprojectgroup-access-tokens) diff --git a/doc/topics/autodevops/cicd_variables.md b/doc/topics/autodevops/cicd_variables.md index b22b4677f24..4242958d962 100644 --- a/doc/topics/autodevops/cicd_variables.md +++ b/doc/topics/autodevops/cicd_variables.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_ec2.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_ec2.md index 6bebe9eb9e3..381326d68ea 100644 --- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_ec2.md +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_ec2.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_ecs.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_ecs.md index 76fd6ad82d8..0d865b5cf0d 100644 --- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_ecs.md +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_ecs.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md index 30cbe06ab95..32a47bb790b 100644 --- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -29,7 +29,7 @@ To deploy a project to EKS: ## Configure your Amazon account Before you create and connect your Kubernetes cluster to your GitLab project, -you need an [Amazon Web Services account](https://https://aws.amazon.com/). +you need an [Amazon Web Services account](https://aws.amazon.com/). Sign in with an existing Amazon account or create a new one. ## Create a Kubernetes cluster @@ -217,7 +217,7 @@ you to common environment tasks: - **Re-deploy to environment** (**{repeat}**) - For more information, see [Retrying and rolling back](../../../ci/environments/index.md#retry-or-roll-back-a-deployment) - **Stop environment** (**{stop}**) - For more information, see - [Stopping an environment](../../../ci/environments/index.md#stop-an-environment) + [Stopping an environment](../../../ci/environments/index.md#stopping-an-environment) GitLab displays the [deploy board](../../../user/project/deploy_boards.md) below the environment's information, with squares representing pods in your diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md index 06b772cc455..1a03a66d17a 100644 --- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -221,7 +221,7 @@ you to common environment tasks: - **Re-deploy to environment** (**{repeat}**) - For more information, see [Retrying and rolling back](../../../ci/environments/index.md#retry-or-roll-back-a-deployment) - **Stop environment** (**{stop}**) - For more information, see - [Stopping an environment](../../../ci/environments/index.md#stop-an-environment) + [Stopping an environment](../../../ci/environments/index.md#stopping-an-environment) GitLab displays the [deploy board](../../../user/project/deploy_boards.md) below the environment's information, with squares representing pods in your diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index b8ea242df1a..e7fe4ce6e0b 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 2a0cc7f39b1..87810193a8d 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/multiple_clusters_auto_devops.md b/doc/topics/autodevops/multiple_clusters_auto_devops.md index 3411beedefb..0b1699d8230 100644 --- a/doc/topics/autodevops/multiple_clusters_auto_devops.md +++ b/doc/topics/autodevops/multiple_clusters_auto_devops.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/prepare_deployment.md b/doc/topics/autodevops/prepare_deployment.md index ebba20ca821..6f434e8fb84 100644 --- a/doc/topics/autodevops/prepare_deployment.md +++ b/doc/topics/autodevops/prepare_deployment.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/requirements.md b/doc/topics/autodevops/requirements.md index a409c6f1520..be242a89c2e 100644 --- a/doc/topics/autodevops/requirements.md +++ b/doc/topics/autodevops/requirements.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index 0c5c87cdf0f..aca8c99b180 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/troubleshooting.md b/doc/topics/autodevops/troubleshooting.md index dd715e95512..21a74cca07a 100644 --- a/doc/topics/autodevops/troubleshooting.md +++ b/doc/topics/autodevops/troubleshooting.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md index 858562eef48..d981313b413 100644 --- a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md +++ b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- @@ -179,7 +179,7 @@ used the `v0.17.0` chart, add `AUTO_DEVOPS_FORCE_DEPLOY_V2`. ## Early adopters -If you want to use the latest [Beta](../../policy/alpha-beta-support.md#beta-features) or unstable version of `auto-deploy-image`, include +If you want to use the latest [Beta](../../policy/alpha-beta-support.md#beta) or unstable version of `auto-deploy-image`, include the latest Auto Deploy template into your `.gitlab-ci.yml`: ```yaml @@ -189,7 +189,7 @@ include: ``` WARNING: -Using a [Beta](../../policy/alpha-beta-support.md#beta-features) or unstable `auto-deploy-image` could cause unrecoverable damage to +Using a [Beta](../../policy/alpha-beta-support.md#beta) or unstable `auto-deploy-image` could cause unrecoverable damage to your environments. Do not test it with important projects or environments. The next stable template update is [planned for GitLab v14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/232788). diff --git a/doc/topics/autodevops/upgrading_postgresql.md b/doc/topics/autodevops/upgrading_postgresql.md index daf1d5341d3..2cf85dc07d7 100644 --- a/doc/topics/autodevops/upgrading_postgresql.md +++ b/doc/topics/autodevops/upgrading_postgresql.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/topics/awesome_co.md b/doc/topics/awesome_co.md index ffda564cd91..f6cad7eb804 100644 --- a/doc/topics/awesome_co.md +++ b/doc/topics/awesome_co.md @@ -3,7 +3,6 @@ stage: Manage group: Foundations info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments description: AwesomeCo test data harness created by the Test Data Working Group https://about.gitlab.com/company/team/structure/working-groups/demo-test-data/ -comments: false --- # AwesomeCo @@ -142,6 +141,25 @@ create(:project, name: 'No longer throws error', owner: @owner, namespace: creat create(:epic, group: create(:group), author: @owner) ``` +#### `parsing id "my id" as "my_id"` + +See [specifying variables](#specify-a-variable) + +#### `id is invalid` + +Given that non-Ruby parsers parse IDs as Ruby Objects, the [naming conventions](https://docs.ruby-lang.org/en/2.0.0/syntax/methods_rdoc.html#label-Method+Names) of Ruby must be followed when specifying an ID. + +Examples of invalid IDs: + +- IDs that start with a number +- IDs that have special characters (-, !, $, @, `, =, <, >, ;, :) + +#### ActiveRecord::AssociationTypeMismatch: Model expected, got ... which is an instance of String + +This is currently a limitation for the seeder. + +See the issue for [allowing parsing of raw Ruby objects](https://gitlab.com/gitlab-org/gitlab/-/issues/403079). + ## YAML Factories ### Generator to generate _n_ amount of records @@ -211,3 +229,50 @@ epics: start_date: <%= 1.day.ago %> due_date: <%= 1.month.from_now %> ``` + +## Variables + +Each created factory can be assigned an identifier to be used in future seeding. + +You can specify an ID for any created factory that you may use later in the seed file. + +### Specify a variable + +You may pass an `_id` attribute on any factory to refer back to it later in non-Ruby parsers. + +Variables are under the factory definitions that they reside in. + +```yaml +--- +group_labels: + - _id: my_label #=> group_labels.my_label + +projects: + - _id: my_project #=> projects.my_project +``` + +Variables: + +NOTE: +It is not advised, but you may specify variables with spaces. These variables may be referred back to with underscores. + +### Referencing a variable + +Given a YAML seed file: + +```yaml +--- +group_labels: + - _id: my_group_label #=> group_labels.my_group_label + name: My Group Label + color: "#FF0000" + - _id: my_other_group_label #=> group_labels.my_other_group_label + color: <%= group_labels.my_group_label.color %> + +projects: + - _id: my_project #=> projects.my_project + name: My Project +``` + +When referring to a variable, the variable refers to the _already seeded_ models. In other words, the model's `id` attribute will +be populated. diff --git a/doc/topics/git/cherry_picking.md b/doc/topics/git/cherry_picking.md index ce2a1d4b954..bd589562ed1 100644 --- a/doc/topics/git/cherry_picking.md +++ b/doc/topics/git/cherry_picking.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Cherry-pick a Git commit **(FREE)** diff --git a/doc/topics/git/feature_branch_development.md b/doc/topics/git/feature_branch_development.md deleted file mode 100644 index 4125d8e8fdb..00000000000 --- a/doc/topics/git/feature_branch_development.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'index.md' -remove_date: '2023-03-31' ---- - -This document was moved to [another location](index.md). - -<!-- This redirect file can be deleted after <2023-03-31>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/topics/git/git_add.md b/doc/topics/git/git_add.md index 722701dbb49..1089202bbd3 100644 --- a/doc/topics/git/git_add.md +++ b/doc/topics/git/git_add.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Git add **(FREE)** diff --git a/doc/topics/git/git_rebase.md b/doc/topics/git/git_rebase.md index ec28d0b6431..bc9337481d4 100644 --- a/doc/topics/git/git_rebase.md +++ b/doc/topics/git/git_rebase.md @@ -226,5 +226,4 @@ git push --force origin my-feature ## Related topics - [Numerous undo possibilities in Git](numerous_undo_possibilities_in_git/index.md#undo-staged-local-changes-without-modifying-history) -for a deeper look into interactive rebases. -- [Git documentation for branches and rebases](https://git-scm.com/book/en/v2/Git-Branching-Rebasing). +- [Git documentation for branches and rebases](https://git-scm.com/book/en/v2/Git-Branching-Rebasing) diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index fa802952b32..dbd046fc162 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -30,7 +30,7 @@ The following resources can help you get started with Git: - [Git Basics](https://git-scm.com/book/en/v2/Getting-Started-Git-Basics) - [Git on the Server - GitLab](https://git-scm.com/book/en/v2/Git-on-the-Server-GitLab) - [How to install Git](how_to_install_git/index.md) -- [Git terminology](terminology.md) +- [Git concepts](terminology.md) - [Start using Git on the command line](../../gitlab-basics/start-using-git.md) - [GitLab Git Cheat Sheet (download)](https://about.gitlab.com/images/press/git-cheat-sheet.pdf) - Commits: @@ -82,7 +82,6 @@ If you have problems with Git, the following may help: ## Branching strategies - [Feature branch workflow](../../gitlab-basics/feature_branch_workflow.md) -- [Develop on a feature branch](feature_branch_development.md) - [GitLab Flow](../gitlab_flow.md) - [Git Branching - Branches in a Nutshell](https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell) - [Git Branching - Branching Workflows](https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows) diff --git a/doc/topics/git/lfs/index.md b/doc/topics/git/lfs/index.md index cac203ffac0..4e4b97960f8 100644 --- a/doc/topics/git/lfs/index.md +++ b/doc/topics/git/lfs/index.md @@ -3,7 +3,6 @@ stage: Create group: Source Code info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" type: reference, howto -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/lfs/lfs/index.html' --- # Git Large File Storage (LFS) **(FREE)** diff --git a/doc/topics/git/rollback_commits.md b/doc/topics/git/rollback_commits.md index 56d740f3d1b..d1f34c7cf9c 100644 --- a/doc/topics/git/rollback_commits.md +++ b/doc/topics/git/rollback_commits.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Roll back commits **(FREE)** diff --git a/doc/topics/git/stash.md b/doc/topics/git/stash.md index ee931bbbb7c..9f621308a1d 100644 --- a/doc/topics/git/stash.md +++ b/doc/topics/git/stash.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Git stash **(FREE)** diff --git a/doc/topics/git/terminology.md b/doc/topics/git/terminology.md index ac097396bef..cef9b7cc35b 100644 --- a/doc/topics/git/terminology.md +++ b/doc/topics/git/terminology.md @@ -4,9 +4,9 @@ group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Git terminology +# Git concepts -The following are commonly-used Git terms. +The following are commonly-used Git concepts. ## Repository @@ -27,7 +27,7 @@ In GitLab, a repository is contained in a **project**. ## Fork When you want to contribute to someone else's repository, you make a copy of it. -This copy is called a [**fork**](../../user/project/repository/forking_workflow.md#creating-a-fork). +This copy is called a [**fork**](../../user/project/repository/forking_workflow.md#create-a-fork). The process is called "creating a fork." When you fork a repository, you create a copy of the project in your own diff --git a/doc/topics/git/unstage.md b/doc/topics/git/unstage.md index aad5d0f44f8..3f509cdacef 100644 --- a/doc/topics/git/unstage.md +++ b/doc/topics/git/unstage.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Unstage a file in Git **(FREE)** diff --git a/doc/topics/gitlab_flow.md b/doc/topics/gitlab_flow.md index 0bf2e1fcc20..d03caab19c4 100644 --- a/doc/topics/gitlab_flow.md +++ b/doc/topics/gitlab_flow.md @@ -2,12 +2,39 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/gitlab_flow.html' --- -# Introduction to GitLab Flow **(FREE)** +# Introduction to Git workflows **(FREE)** With Git, you can use a variety of branching strategies and workflows. +Having a structured workflow for collaboration in complex projects is +crucial for several reasons: + +- **Code organization**: Keep the codebase organized, prevent + overlapping work, and ensure focused efforts towards a common goal. + +- **Version control**: Allow simultaneous work on different features + without conflicts, maintaining code stability. + +- **Code quality**: A code review and approval process helps maintain high + code quality and adherence to coding standards. + +- **Traceability and accountability**: Enable tracking of changes and their authors, + simplifying issue identification and responsibility assignment. + +- **Easier onboarding**: Help new team members quickly grasp the + development process, and start contributing effectively. + +- **Time and resource management**: Enable better planning, resource + allocation, and meeting deadlines, ensuring an efficient development + process. + +- **CI/CD**: Incorporate automated testing and deployment + processes, streamlining the release cycle and delivering high-quality + software consistently. + +A structured workflow promotes organization, efficiency, and code +quality, leading to a more successful and streamlined development process. Because the default workflow is not specifically defined, many organizations end up with workflows that are too complicated, not clearly defined, or @@ -15,12 +42,51 @@ not integrated with their issue tracking systems. Your organization can use GitLab with any workflow you choose. -However, if you are looking for guidance on best practices, you can use -the GitLab Flow. This workflow combines [feature-driven development](https://en.wikipedia.org/wiki/Feature-driven_development) -and [feature branches](https://martinfowler.com/bliki/FeatureBranch.html) with issue tracking. +## Workflow types -While this workflow used at GitLab, you can choose whichever workflow -suits your organization best. +Here are some of the most common Git workflows. + +### Centralized workflow + +Best suited for small teams transitioning from a centralized version +control system like SVN. All team members work on a single branch, +usually `main`, and push their changes directly to the central +repository. + +### Feature branch workflow + +Developers create separate branches for each feature or bugfix, +keeping the 'main' branch stable. When a feature is complete, the +developer submits a pull request or merge request to integrate the +changes back into the `main` branch after a code review. + +### Forking workflow + +Commonly used in open-source projects, this workflow allows external +contributors to work without direct access to the main repository. +Developers create a fork (a personal copy) of the main repository, +make changes in their fork, and then submit a pull request or merge +request to have their changes integrated into the main repository. + +### Git flow workflow + +This workflow is best for projects with a structured release cycle. +It introduces two long-lived branches: `main` for production-ready +code and `develop` for integrating features. Additional branches like +`feature`, `release`, and `hotfix` are used for specific purposes, +ensuring a strict and organized development process. + +### GitLab/GitHub flow + +A simplified workflow primarily used for web development and +continuous deployment. It combines aspects of the Feature branch +workflow and the Git flow workflow. Developers create feature branches +from `main`, and after the changes are complete, they are merged back +into the `main` branch, which is then immediately deployed. + +Each of these Git workflows has its advantages and is suited to +different project types and team structures. Below the most popular +workflows are reviewed in more details. ## Git workflow @@ -100,6 +166,16 @@ This flow is clean and straightforward, and many organizations have adopted it w Atlassian recommends [a similar strategy](https://www.atlassian.com/blog/git/simple-git-workflow-is-simple), although they rebase feature branches. Merging everything into the `main` branch and frequently deploying means you minimize the amount of unreleased code. This approach is in line with lean and continuous delivery best practices. However, this flow still leaves a lot of questions unanswered regarding deployments, environments, releases, and integrations with issues. + +## Introduction to GitLab Flow **(FREE)** + +However, if you are looking for guidance on best practices, you can use +the GitLab Flow. This workflow combines [feature-driven development](https://en.wikipedia.org/wiki/Feature-driven_development) +and [feature branches](https://martinfowler.com/bliki/FeatureBranch.html) with issue tracking. + +While this workflow used at GitLab, you can choose whichever workflow +suits your organization best. + With GitLab flow, we offer additional guidance for these questions. ## Production branch with GitLab flow @@ -114,7 +190,8 @@ While this is possible in some cases, such as SaaS applications, there are some In these cases, you can create a production branch that reflects the deployed code. You can deploy a new version by merging `main` into the `production` branch. -While not shown in the graph below, the work on the `main` branch works just like in GitHub flow, i.e. with feature-branches being merged into `main`. +While not shown in the graph below, the work on the `main` branch works just like in GitHub flow: +with feature branches being merged into `main`. ```mermaid graph TD @@ -256,7 +333,8 @@ In GitLab, each change to the codebase starts with an issue in the issue trackin If there is no issue yet, create the issue if the change requires more than an hour's work. In many organizations, raising an issue is part of the development process because they are used in sprint planning. The issue title should describe the desired state of the system. -For example, the issue title "As an administrator, I want to remove users without receiving an error" is better than "Administrators can't remove users." +For example, the issue title `As an administrator, I want to remove users without receiving an error` +is better than "Administrators can't remove users." When you are ready to code, create a branch for the issue from the `main` branch. This branch is the place for any work related to this change. @@ -268,7 +346,7 @@ When you are done or want to discuss the code, open a merge request. A merge request is an online place to discuss the change and review the code. If you open the merge request but do not assign it to anyone, it is a [draft merge request](../user/project/merge_requests/drafts.md). -These are used to discuss the proposed implementation but are not ready for inclusion in the `main` branch yet. +Drafts are used to discuss the proposed implementation but are not ready for inclusion in the `main` branch yet. Start the title of the merge request with `[Draft]`, `Draft:` or `(Draft)` to prevent it from being merged before it's ready. When you think the code is ready, assign the merge request to a reviewer. @@ -357,7 +435,11 @@ Sometimes you can reuse recorded resolutions (`rerere`), but merging is better, Atlassian has [a more thorough explanation of the tradeoffs between merging and rebasing](https://www.atlassian.com/blog/git/git-team-workflows-merge-or-rebase) on their blog. A good way to prevent creating many merge commits is to not frequently merge `main` into the feature branch. -There are three reasons to merge in `main`: utilizing new code, resolving merge conflicts, and updating long-running branches. +Three reasons to merge in `main`: + +1. Utilizing new code. +1. Resolving merge conflicts. +1. Updating long-running branches. If you need to use some code that was introduced in `main` after you created the feature branch, you can often solve this by just cherry-picking a commit. diff --git a/doc/topics/your_work.md b/doc/topics/your_work.md index 268a6c4df5b..e8804867c96 100644 --- a/doc/topics/your_work.md +++ b/doc/topics/your_work.md @@ -6,9 +6,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Your work sidebar -- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/384342) in GitLab 15.9. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/384342) in GitLab 15.9. -The **Your work** left sidebar provides access to your: +The **Your work** sidebar is a more focused view into the areas you have access to. + +This sidebar is part of [a redesign effort](https://gitlab.com/groups/gitlab-org/-/epics/9044) +that removes the top bar and enables more customization of your GitLab experience. - [Projects](../user/project/working_with_projects.md#view-projects) - [Groups](../user/group/index.md) diff --git a/doc/tutorials/create_compliance_pipeline.md b/doc/tutorials/create_compliance_pipeline.md new file mode 100644 index 00000000000..382a7f3fa57 --- /dev/null +++ b/doc/tutorials/create_compliance_pipeline.md @@ -0,0 +1,177 @@ +--- +stage: Govern +group: Compliance +info: For assistance with this tutorial, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects. +--- + +# Tutorial: Create a compliance pipeline **(ULTIMATE)** + +You can use [compliance pipelines](../user/group/compliance_frameworks.md#compliance-pipelines) to ensure specific +compliance-related jobs are run on pipelines for all projects in a group. Compliance pipelines are applied +to projects through [compliance frameworks](../user/group/compliance_frameworks.md). + +In this tutorial, you: + +1. Create a [new group](#create-a-new-group). +1. Create a [new project for the compliance pipeline configuration](#create-a-new-compliance-pipeline-project). +1. Configure a [compliance framework](#configure-compliance-framework) to apply to other projects. +1. Create a [new project and apply the compliance framework](#create-a-new-project-and-apply-the-compliance-framework) to it. +1. Combine [compliance pipeline configuration and regular pipeline configuration](#combine-pipeline-configurations). + +Prerequisites: + +- Permission to create new top-level groups. + +## Create a new group + +Compliance frameworks are configured in top-level groups. In this tutorial, you create a top-level group that: + +- Contains two projects: + - The compliance pipeline project to store the compliance pipeline configuration. + - Another project that must run a job in its pipeline that is defined by the compliance pipeline configuration. +- Has the compliance framework to apply to projects. + +To create the new group: + +1. On the top bar, select **Create new... > New group**. +1. Select **Create group**. +1. In the **Group name** field, enter `Tutorial group`. +1. Select **Create group**. + +## Create a new compliance pipeline project + +Now you're ready to create a compliance pipeline project. This project contains the +[compliance pipeline configuration](../user/group/compliance_frameworks.md#example-configuration) to apply to all +projects with the compliance framework applied. + +To create the compliance pipeline project: + +1. On the top bar, select **Main menu > Groups** and find the `Tutorial group` group. +1. Select **New project**. +1. Select **Create blank project**. +1. In the **Project name** field, enter `Tutorial compliance project`. +1. Select **Create project**. + +To add compliance pipeline configuration to `Tutorial compliance project`: + +1. On the top bar, select **Main menu > Projects** and find the `Tutorial compliance project` project. +1. On the left sidebar, select **CI/CD > Editor**. +1. Select **Configure pipeline**. +1. In the pipeline editor, replace the default configuration with: + + ```yaml + --- + compliance-job: + script: + - echo "Running compliance job required for every project in this group..." + ``` + +1. Select **Commit changes**. + +## Configure compliance framework + +The compliance framework is configured in the [new group](#create-a-new-group). + +To configure the compliance framework: + +1. On the top bar, select **Main menu > Groups** and find the `Tutorial group` group. +1. On the left sidebar, select **Settings > General**. +1. Expand **Compliance frameworks**. +1. Select **Add framework**. +1. In the **Name** field, enter `Tutorial compliance framework`. +1. In the **Description** field, enter `Compliance framework for tutorial`. +1. In the **Compliance pipeline configuration (optional)** field, enter + `.gitlab-ci.yml@tutorial-group/tutorial-compliance-project`. +1. In the **Background color** field, select a color of your choice. +1. Select **Add framework**. + +For convenience, make the new compliance framework the default for all new projects in the group: + +1. On the top bar, select **Main menu > Groups** and find the `Tutorial group` group. +1. On the left sidebar, select **Settings > General**. +1. Expand **Compliance frameworks**. +1. In the row for `Tutorial compliance framework`, select **Options** (**{ellipsis_v}**). +1. Select **Set default**. + +## Create a new project and apply the compliance framework + +Your compliance framework is ready, so you can now create projects in the group and they automatically run the +compliance pipeline configuration in their pipelines. + +To create a new project for running the compliance pipeline configuration: + +1. On the top bar, select **Main menu > Groups** and find the `Tutorial group` group. +1. Select **New project**. +1. Select **Create blank project**. +1. In the **Project name** field, enter `Tutorial project`. +1. Select **Create project**. + +On the project page, notice the `Tutorial compliance framework` label appears because that was set as the default +compliance framework for the group. + +Without any other pipeline configuration, `Tutorial project` can run the jobs defined in the compliance +pipeline configuration in `Tutorial compliance project`. + +To run the compliance pipeline configuration in `Tutorial project`: + +1. On the top bar, select **Main menu > Projects** and find the `Tutorial project` project. +1. Select **CI/CD Pipelines**. +1. Select **Run pipeline**. +1. On the **Run pipeline** page, select **Run pipeline**. + +Notice the pipeline runs a job called `compliance-job` in a **test** stage. Nice work, you've run your first compliance +job! + +## Combine pipeline configurations + +If you want projects to run their own jobs as well as the compliance pipeline jobs, you must combine the compliance +pipeline configuration and the regular pipeline configuration of the project. + +To combine the pipeline configurations, you must define the regular pipeline configuration and then update the +compliance pipeline configuration to refer to it. + +To create the regular pipeline configuration: + +1. On the top bar, select **Main menu > Projects** and find the `Tutorial project` project. +1. On the left sidebar, select **CI/CD > Editor**. +1. Select **Configure pipeline**. +1. In the pipeline editor, replace the default configuration with: + + ```yaml + --- + project-job: + script: + - echo "Running project job..." + ``` + +1. Select **Commit changes**. + +To combine the new project pipeline configuration with the compliance pipeline configuration: + +1. On the top bar, select **Main menu > Projects** and find the `Tutorial compliance project` project. +1. On the left sidebar, select **CI/CD > Editor**. +1. In the existing configuration, add: + + ```yaml + include: + - project: 'tutorial-group/tutorial-project' + file: '.gitlab-ci.yml' + ``` + +1. Select **Commit changes**. + +To confirm the regular pipeline configuration is combined with the compliance pipeline configuration: + +1. On the top bar, select **Main menu > Projects** and find the `Tutorial project` project. +1. Select **CI/CD Pipelines**. +1. Select **Run pipeline**. +1. On the **Run pipeline** page, select **Run pipeline**. + +Notice the pipeline runs two jobs in a **test** stage: + +- `compliance-job`. +- `project-job`. + +Congratulations, you've created and configured a compliance pipeline! + +See more [example compliance pipeline configurations](../user/group/compliance_frameworks.md#example-configuration). diff --git a/doc/tutorials/fuzz_testing_tutorial.md b/doc/tutorials/fuzz_testing_tutorial.md new file mode 100644 index 00000000000..5a8209d9716 --- /dev/null +++ b/doc/tutorials/fuzz_testing_tutorial.md @@ -0,0 +1,243 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Tutorial: Perform fuzz testing in GitLab **(ULTIMATE)** + +[Coverage-guided fuzz testing](../user/application_security/coverage_fuzzing/index.md#coverage-guided-fuzz-testing-process) sends unexpected, malformed, or random data to your application, and then monitors +your application for unstable behaviors and crashes. + +This helps you discover bugs and potential security issues that other QA processes may miss. + +You should use fuzz testing in addition to other security scanners and your own test processes. +If you're using GitLab CI/CD, you can run fuzz tests as part your CI/CD workflow. + +To set up, configure, and perform coverage-guided fuzz testing +using JavaScript in this tutorial, you: + +1. [Fork the project template](#fork-the-project-template) to create a project + to run the fuzz tests in. +1. [Create the fuzz targets](#create-the-fuzz-targets). +1. [Enable coverage-guided fuzz testing](#enable-coverage-guided-fuzz-testing) + in your forked project. +1. [Run the fuzz test](#run-the-fuzz-test) to identify security vulnerabilities. +1. [Fix any vulnerabilities](#fix-the-vulnerabilities) identified by the fuzz test. + +## Fork the project template + +First, to create a project to try out fuzz testing in, you must fork the `fuzz-testing` +project template: + +1. Open the [`fuzz-testing` project template](https://gitlab.com/gitlab-org/tutorial-project-templates/fuzz-testing). +1. [Fork the project template](../user/project/repository/forking_workflow.md). +1. When forking the project template: + - Name the forked project `fuzz-testing-demo`. + - Select an appropriate [namespace](../user/namespace/index.md). + - Set [project visibility](../user/public_access.md) to **Private**. + +You have successfully forked the `fuzz-testing` project template. Before you can +start fuzz testing, remove the relationship between the project template and the fork: + +1. On the left sidebar, select **Settings > General**. +1. Expand **Advanced**. +1. In the **Remove fork relationship** section, select **Remove fork relationship**. + Enter the name of the project when prompted. + +Your project is ready and you can now create the fuzz test. Next you will create +the fuzz targets. + +## Create the fuzz targets + +Now you have a project for fuzz testing, you create the fuzz targets. A fuzz target +is a function or program that, given an input, makes a call to the application +being tested. + +In this tutorial, the fuzz targets call a function of the `my-tools.js` file using +a random buffer as a parameter. + +To create the two fuzz target files: + +1. On the top bar, select **Main menu > Projects** and select the `fuzz-testing-demo` project. +1. Create a file in the root directory of the project. +1. Name the file `fuzz-sayhello.js` and add the following code: + + ```javascript + let tools = require('./my-tools') + + function fuzz(buf) { + const text = buf.toString() + tools.sayHello(text) + } + + module.exports = { + fuzz + } + ``` + + You can also copy this code from the `fuzz-testing-demo/fuzzers/fuzz-sayhello.js` + project file. + +1. Name the **Target Branch** `add-fuzz-test` and write a descriptive commit message. + - Do not select the **Start a new merge request with these changes** checkbox yet. +1. Select **Commit changes**. +1. Return to the root directory of the project. +1. Make sure you are in the `add-fuzz-test` branch. +1. Create the second file named `fuzz-readme.js` and add the following code: + + ```javascript + let tools = require('./my-tools') + function fuzz(buf) { + const text = buf.toString() + tools.readmeContent(text) + } + module.exports = { + fuzz + } + ``` + + You can also copy this code from the `fuzz-testing-demo/fuzzers/fuzz-readme.js` + project file. + +1. Write a descriptive commit message. +1. Make sure the **Target Branch** is `add-fuzz-test`. +1. Select **Commit changes**. + +You now have two fuzz targets that can make calls to the application being tested. +Next you will enable the fuzz testing. + +## Enable coverage-guided fuzz testing + +To enable coverage-guided fuzz testing, create a CI/CD pipeline running +the `gitlab-cov-fuzz` CLI to execute the fuzz test on the two fuzz targets. + +To create the pipeline file: + +1. Make sure you are in the `add-fuzz-test` branch. +1. In the root directory of the `fuzz-testing-demo` project, create a new file. +1. Name the file `.gitlab-ci.yml` and add the following code: + + ```yaml + image: node:18 + + stages: + - fuzz + + include: + - template: Coverage-Fuzzing.gitlab-ci.yml + + readme_fuzz_target: + extends: .fuzz_base + tags: [saas-linux-large-amd64] # Optional + variables: + COVFUZZ_ADDITIONAL_ARGS: '--fuzzTime=60' + script: + - npm config set @gitlab-org:registry https://gitlab.com/api/v4/packages/npm/ && npm i -g @gitlab-org/jsfuzz + - ./gitlab-cov-fuzz run --engine jsfuzz -- fuzz-readme.js + + hello_fuzzing_target: + extends: .fuzz_base + tags: [saas-linux-large-amd64] # Optional + variables: + COVFUZZ_ADDITIONAL_ARGS: '--fuzzTime=60' + script: + - npm config set @gitlab-org:registry https://gitlab.com/api/v4/packages/npm/ && npm i -g @gitlab-org/jsfuzz + - ./gitlab-cov-fuzz run --engine jsfuzz -- fuzz-sayhello.js + ``` + + This step adds the following to your pipeline: + - A `fuzz` stage using a template. + - Two jobs, `readme_fuzz_target` and `hello_fuzzing_target`. Each job runs using + the `jsfuzz` engine, which reports unhandled exceptions as crashes. + + You can also copy this code from the `fuzz-testing-demo/fuzzers/fuzzers.yml` + project file. + +1. Write a descriptive commit message. +1. Make sure the **Target Branch** is `add-fuzz-test`. +1. Select **Commit changes**. + +You have successfully enabled coverage-guided fuzz testing. Next you will run the +fuzz test using the pipeline you've just created. + +## Run the fuzz test + +To run the fuzz test: + +1. On the left sidebar, select **Merge requests**. +1. Select **New merge request**. +1. In the **Source branch** section, select the `add-fuzz-test` branch. +1. In the **Target branch** section, make sure that your namespace and the `main` branch are selected. +1. Select **Compare branches and continue**. +1. [Create the merge request](../user/project/merge_requests/creating_merge_requests.md). + +Creating the merge request triggers a new pipeline, which runs the fuzz test. +When the pipeline is finished running, you should see a security vulnerability +alert on the merge request page. + +To see more information on each vulnerability, select the individual **Uncaught-exception** links. + +You have successfully run the fuzz test and identified vulnerabilities to fix. + +## Fix the vulnerabilities + +The fuzz test identified two security vulnerabilities. To fix those +vulnerabilities, you use the `my-tools.js` library. + +To create the `my-tools.js` file: + +1. Make sure you are in the `add-fuzz-test` branch of the project. +1. Go to the root directory of your project and open the `my-tools.js` file. +1. Replace the contents of this file with the following code: + + ```javascript + const fs = require('fs') + + function sayHello(name) { + if(name.includes("z")) { + //throw new Error("😡 error name: " + name) + console.log("😡 error name: " + name) + } else { + return "😀 hello " + name + } + } + + function readmeContent(name) { + + let fileName = name => { + if(name.includes("w")) { + return "./README.txt" + } else { + return "./README.md" + } + } + + //const data = fs.readFileSync(fileName(name), 'utf8') + try { + const data = fs.readFileSync(fileName(name), 'utf8') + return data + } catch (err) { + console.error(err.message) + return "" + } + + } + + module.exports = { + sayHello, readmeContent + } + ``` + + You can also copy the code from the `fuzz-testing-demo/javascript/my-tools.js` + project file. + +1. Select **Commit changes**. This triggers another pipeline to run another fuzz test. +1. When the pipeline is finished, check the merge request **Overview** page. You + should see that the security scan detected no new potential vulnerabilities. +1. Merge your changes. + +Congratulations, you've successfully run a fuzz test and fixed the identified +security vulnerabilities! + +For more information, see [coverage-guided fuzz testing](../user/application_security/coverage_fuzzing/index.md). diff --git a/doc/tutorials/index.md b/doc/tutorials/index.md index 7a94583ae69..aee3adba919 100644 --- a/doc/tutorials/index.md +++ b/doc/tutorials/index.md @@ -16,7 +16,7 @@ and running quickly. | Topic | Description | Good for beginners | |-------|-------------|--------------------| | <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Introduction to GitLab](https://youtu.be/_4SmIyQ5eis?t=90) (59m 51s) | Walk through recommended processes and example workflows for using GitLab. | **{star}** | -| [GitLab 101](https://levelup.gitlab.com/learn/course/gitlab101) | Learn the basics of GitLab in this certification course. | **{star}** | +| [GitLab with Git Essentials](https://levelup.gitlab.com/courses/gitlab-with-git-essentials) | Learn the basics of Git and GitLab in this self-paced course. | **{star}** | | <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Use GitLab for DevOps](https://www.youtube.com/watch?v=7q9Y1Cv-ib0) (12m 34s) | Use GitLab through the entire DevOps lifecycle, from planning to monitoring. | **{star}** | | [Use Markdown at GitLab](../user/markdown.md) | GitLab Flavored Markdown (GLFM) is used in many areas of GitLab, for example, in merge requests. | **{star}** | | <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Learn GitLab project walkthrough](https://www.youtube.com/watch?v=-oaI2WEKdI4&list=PL05JrBw4t0KofkHq4GZJ05FnNGa11PQ4d) (59m 2s) | Step through the tutorial-style issues in the **Learn GitLab** project. If you don't have this project, download [the export file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/project_templates/learn_gitlab_ultimate.tar.gz) and [import it to a new project](../user/project/settings/import_export.md#import-a-project-and-its-data). | | @@ -35,16 +35,18 @@ the most out of GitLab. | [Take advantage of Git rebase](https://about.gitlab.com/blog/2022/10/06/take-advantage-of-git-rebase/)| Learn how to use the `rebase` command in your workflow. | | | [Git cheat sheet](https://about.gitlab.com/images/press/git-cheat-sheet.pdf) | Download a PDF of common Git commands. | | -## Plan your work in projects +## Plan and track your work -Your work takes place in a project, from creating code, to planning, -collaborating, and more. +Create a project to host your code, and plan your work using +issues, epics, and more. | Topic | Description | Good for beginners | |-------|-------------|--------------------| +| [GitLab Agile Project Management](https://levelup.gitlab.com/courses/gitlab-agile-project-management) | Learn how to use planning features to manage your projects in this self-paced course. | **{star}** | | [Create a project from a template](https://gitlab.com/projects/new#create_from_template) | Choose a project template and create a project with files to get you started. | | | [Migrate to GitLab](../user/project/import/index.md) | If you are coming to GitLab from another platform, you can import or convert your projects. | | | [Run an agile iteration](agile_sprint.md) | Use group, projects, and iterations to run an agile development iteration. | +| <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Epics and Issue Boards](https://www.youtube.com/watch?v=I1bFIAQBHB8) | Find out how to use epics and issue boards for project management. | | ## Use CI/CD pipelines @@ -54,8 +56,8 @@ CI/CD pipelines are used to automatically build, test, and deploy your code. |-------|-------------|--------------------| | [Create and run your first GitLab CI/CD pipeline](../ci/quick_start/index.md) | Create a `.gitlab-ci.yml` file and start a pipeline. | **{star}** | | <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Get started: Learn about CI/CD](https://www.youtube.com/watch?v=sIegJaLy2ug) (9m 02s) | Learn about the `.gitlab-ci.yml` file and how it's used. | **{star}** | +| [GitLab CI/CD](https://levelup.gitlab.com/courses/continuous-integration-and-delivery-ci-cd-with-gitlab) | Learn about GitLab CI/CD and build a pipeline in this self-paced course. | **{star}** | | <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [CI deep dive](https://www.youtube.com/watch?v=ZVUbmVac-m8&list=PL05JrBw4t0KorkxIFgZGnzzxjZRCGROt_&index=27) (22m 51s) | Take a closer look at pipelines and continuous integration concepts. | | -| <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [CD deep dive](https://www.youtube.com/watch?v=Cn0rzND-Yjw&list=PL05JrBw4t0KorkxIFgZGnzzxjZRCGROt_&index=10) (47m 54s) | Learn about deploying in GitLab. | | | [Set up CI/CD in the cloud](../ci/examples/index.md#cicd-in-the-cloud) | Learn how to set up CI/CD in different cloud-based environments. | | | [Find CI/CD examples and templates](../ci/examples/index.md#cicd-examples) | Use these examples and templates to set up CI/CD for your use case. | | | <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Understand CI/CD rules](https://www.youtube.com/watch?v=QjQc-zeL16Q) (8m 56s) | Learn more about how to use CI/CD rules. | | @@ -79,14 +81,17 @@ Use GitLab Pages to publish a static website directly from your project. | [Create a Pages website from a CI/CD template](../user/project/pages/getting_started/pages_ci_cd_template.md) | Quickly generate a Pages website for your project using a CI/CD template for a popular Static Site Generator (SSG). | **{star}** | | [Create a Pages website from scratch](../user/project/pages/getting_started/pages_from_scratch.md) | Create all the components of a Pages website from a blank project. | | -## Secure your application +## Secure your application and check compliance -GitLab can check your application for security vulnerabilities. +GitLab can check your application for security vulnerabilities and that it meets compliance requirements. | Topic | Description | Good for beginners | |-------|-------------|--------------------| | [Set up dependency scanning](https://about.gitlab.com/blog/2021/01/14/try-dependency-scanning/) | Try out dependency scanning, which checks for known vulnerabilities in dependencies. | **{star}** | +| [Create a compliance pipeline](create_compliance_pipeline.md) | Learn how to create compliance pipelines for your groups. | **{star}** | +| [Set up a scan result policy](scan_result_policy.md) | Learn how to configure a scan result policy that takes action based on scan results. | **{star}** | | [Get started with GitLab application security](../user/application_security/get-started-security.md) | Follow recommended steps to set up security tools. | | +| [GitLab Security Essentials](https://levelup.gitlab.com/courses/security-essentials) | Learn about the essential security capabilities of GitLab in this self-paced course. | | ## Work with a self-managed instance @@ -113,7 +118,7 @@ enabling you to work with those services directly from GitLab. If you're learning about GitLab, here are some ways you can find more tutorial content: -- Find learning tracks and certification options at [GitLab Learn](https://about.gitlab.com/learn/). +- Find learning tracks and certification options at [Level Up](https://levelup.gitlab.com/). GitLab learning platform login required (email and password for non-GitLab team members). - Find recent tutorials on the GitLab blog by [searching by the `tutorial` tag](https://about.gitlab.com/blog/tags.html#tutorial). diff --git a/doc/tutorials/scan_result_policy.md b/doc/tutorials/scan_result_policy.md new file mode 100644 index 00000000000..e617f06d321 --- /dev/null +++ b/doc/tutorials/scan_result_policy.md @@ -0,0 +1,125 @@ +--- +stage: Govern +group: Security Policies +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Tutorial: Set up a scan result policy **(ULTIMATE)** + +This tutorial shows you how to create and configure a [scan result policy](../user/application_security/policies/scan-result-policies.md). These policies can be set to take action based on scan results. +For example, in this tutorial, you'll set up a policy that requires approval from two specified users if a vulnerability is detected in a merge request. + +Prerequisites: + +The namespace used for this tutorial must: + +- Contain a minimum of three users, including your own. If you don't have two other users, you must first + create them. For details, see [Creating users](../user/profile/account/create_accounts.md). + +To set up a scan result policy: + +1. [Create a test project](#create-a-test-project). +1. [Add a scan result policy](#add-a-scan-result-policy). +1. [Test the scan result policy](#test-the-scan-result-policy). + +## Create a test project + +1. On the top bar, select **Main menu > Projects**. +1. Select **New project**. +1. Select **Create blank project**. +1. Complete the fields. + - **Project name**: `sast-scan-result-policy`. + - Select the **Enable Static Application Security Testing (SAST)** checkbox. +1. Select **Create project**. + +## Add a scan result policy + +Next, you'll add a scan result policy to your test project: + +1. On the top bar, select **Main menu > Projects** and find the `sast-scan-result-policy` project. +1. On the left sidebar, go to **Security and Compliance > Policies**. +1. Select **New policy**. +1. In **Scan result policy**, select **Select policy**. +1. Complete the fields. + - **Name**: `sast-scan-result-policy` + - **Policy status**: **Enabled** +1. Add the following rule: + + ```plaintext + IF |Security Scan| from |SAST| find(s) more than |0| |All severity levels| |All vulnerability states| vulnerabilities in an open merge request targeting |All protected branches| + ``` + +1. Set **Actions** to the following: + + ```plaintext + THEN Require approval from | 2 | of the following approvers: + ``` + +1. Select two users. +1. Select **Configure with a merge request**. + + The application creates a new project to store the policies linked to it, and creates a merge request to define the policy. + +1. Select **Merge**. +1. On the top bar, select **Main menu > Projects** and select the `sast-scan-result-policy` project. +1. On the left sidebar, select **Security and Compliance > Policies**. + + You can see the list of policies added in the previous steps. + +## Test the scan result policy + +Nice work, you've created a scan result policy. To test it, create some vulnerabilities and check the result: + +1. On the top bar, select **Main menu > Projects** and select the `sast-scan-result-policy` project. +1. On the left sidebar, select **Repository > Files**. +1. From the **Add** (**{plus}**) dropdown list, select **New file**. +1. In the **Filename** field enter `main.ts`. +1. In the file's content, copy the following: + + ```typescript + // Non-literal require - tsr-detect-non-literal-require + var lib: String = 'fs' + require(lib) + + // Eval with variable - tsr-detect-eval-with-expression + var myeval: String = 'console.log("Hello.");'; + eval(myeval); + + // Unsafe Regexp - tsr-detect-unsafe-regexp + const regex: RegExp = /(x+x+)+y/; + + // Non-literal Regexp - tsr-detect-non-literal-regexp + var myregexpText: String = "/(x+x+)+y/"; + var myregexp: RegExp = new RegExp(myregexpText); + myregexp.test("(x+x+)+y"); + + // Markup escaping disabled - tsr-detect-disable-mustache-escape + var template: Object = new Object; + template.escapeMarkup = false; + + // Detects HTML injections - tsr-detect-html-injection + var element: Element = document.getElementById("mydiv"); + var content: String = "mycontent" + Element.innerHTML = content; + + // Timing attack - tsr-detect-possible-timing-attacks + var userInput: String = "Jane"; + var auth: String = "Jane"; + if (userInput == auth) { + console.log(userInput); + } + ``` + +1. In the **Commit message** field, enter `Add vulnerable file`. +1. In the **Target Branch** field, enter `test-branch`. +1. Select **Commit changes**. The **New merge request** form opens. +1. Select **Create merge request**. +1. In the new merge request, select `Create merge request`. + + Wait for the pipeline to complete. This could be a few minutes. + +The merge request security widget confirms that security scanning detected one potential +vulnerability. As defined in the scan result policy, the merge request is blocked and waiting for +approval. + +You now know how to set up and use scan result policies to catch vulnerabilities! diff --git a/doc/update/background_migrations.md b/doc/update/background_migrations.md index 47b786f2894..be6ad43fa8a 100644 --- a/doc/update/background_migrations.md +++ b/doc/update/background_migrations.md @@ -203,6 +203,47 @@ To disable it: Feature.disable(:optimize_batched_migrations) ``` +### Parallel execution + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104027) +> in GitLab 15.7, [behind a feature flag](../user/feature_flags.md), +> [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/372316). +> - Enabled on GitLab.com. +> - Recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to +> [disable it](#enable-or-disable-parallel-execution-for-batched-background-migrations). + +There can be [risks when disabling released features](../administration/feature_flags.md#risks-when-disabling-released-features). +Refer to this feature's version history for more details. + +To speed up the execution of batched background migrations, two migrations are executed at the same time. + +[GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) can change +the number of batched background migrations executed in parallel: + +```ruby +ApplicationSetting.update_all(database_max_running_batched_background_migrations: 4) +``` + +#### Enable or disable parallel execution for batched background migrations + +Parallel execution for batched background migrations is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) +can opt to disable it. + +To enable it: + +```ruby +Feature.enable(:batched_migrations_parallel_execution) +``` + +To disable it: + +```ruby +Feature.disable(:batched_migrations_parallel_execution) +``` + ## Troubleshooting ### Enable or disable background migrations diff --git a/doc/update/deprecations.md b/doc/update/deprecations.md index d7bbf254f2a..4eb34a5a4af 100644 --- a/doc/update/deprecations.md +++ b/doc/update/deprecations.md @@ -2,6 +2,7 @@ stage: none group: none info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +toc: false --- # Deprecations by version @@ -30,7 +31,7 @@ For deprecation reviewers (Technical Writers only): {::options parse_block_html="true" /} -In each release, GitLab announces features that are deprecated and no longer recommended for use. +These GitLab features are deprecated and no longer recommended for use. Each deprecated feature will be removed in a future release. Some features cause breaking changes when they are removed. @@ -39,1014 +40,1202 @@ add this URL to your RSS feed reader: `https://about.gitlab.com/breaking-changes You can also view [REST API](https://docs.gitlab.com/ee/api/rest/deprecations.html) and [GraphQL](https://docs.gitlab.com/ee/api/graphql/removed_items.html) deprecations/removals. + <div class="js-deprecation-filters"></div> +<div class="milestone-wrapper" data-milestone="17.0"> -<div class="announcement-milestone"> +## GitLab 17.0 -## Announced in 15.10 +<div class="deprecation breaking-change" data-milestone="17.0"> -<div class="deprecation removal-160 breaking-change"> +### Accessibility Testing is deprecated -### Deprecated Consul http metrics +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +Due to low customer usage, Accessibility Testing is deprecated and will be removed. There is no planned replacement and users should stop using Accessibility Testing before GitLab 17.0. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -The Consul provided in the GitLab Omnibus package will no longer provide older deprecated Consul metrics starting in GitLab 16.0. +<div class="deprecation breaking-change" data-milestone="17.0"> -In GitLab 14.0, [Consul was updated to 1.9.6](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5344), -which deprecated some telemetry metrics from being at the `consul.http` path. In GitLab 16.0, the `consul.http` path will be removed. +### Auto DevOps support for Herokuish is deprecated -If you have monitoring that consumes Consul metrics, update them to use `consul.api.http` instead of `consul.http`. -For more information, see [the deprecation notes for Consul 1.9.0](https://github.com/hashicorp/consul/releases/tag/v1.9.0). +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +Auto DevOps support for Herokuish is deprecated in favor of [Cloud Native Buildpacks](https://docs.gitlab.com/ee/topics/autodevops/stages.html#auto-build-using-cloud-native-buildpacks). You should [migrate your builds from Herokuish to Cloud Native Buildpacks](https://docs.gitlab.com/ee/topics/autodevops/stages.html#moving-from-herokuish-to-cloud-native-buildpacks). From GitLab 14.0, Auto Build uses Cloud Native Buildpacks by default. + +Because Cloud Native Buildpacks do not support automatic testing, the Auto Test feature of Auto DevOps is also deprecated. </div> -<div class="deprecation removal-170 breaking-change"> +<div class="deprecation breaking-change" data-milestone="17.0"> -### DingTalk OmniAuth provider +### Browser Performance Testing is deprecated -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +Due to limited customer usage, Browser Performance Testing is deprecated and will be removed. There is no planned replacement and users should stop using Browser Performance Testing before GitLab 17.0. + +</div> + +<div class="deprecation " data-milestone="17.0"> + +### Deprecate legacy shell escaping and quoting runner shell executor + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.11</span> +- End of Support: GitLab <span class="milestone">17.9</span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The runner's legacy escape sequence mechanism to handle variable expansion implements a sub-optimal implementation of Ansi-C quoting. This method means that the runner would expand arguments included in double quotes. As of 15.11, we are deprecating the legacy escaping and quoting methods in the runner shell executor. + +</div> + +<div class="deprecation breaking-change" data-milestone="17.0"> + +### DingTalk OmniAuth provider + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The `omniauth-dingtalk` gem that provides GitLab with the DingTalk OmniAuth provider will be removed in our next major release, GitLab 17.0. This gem sees very little use and is better suited for JiHu edition. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="17.0"> -### Environment search query requires at least three characters +### Filepath field in Releases and Release Links APIs -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +Support for specifying a `filepath` for a direct asset link in the [Releases API](https://docs.gitlab.com/ee/api/releases) +and [Release Links API](https://docs.gitlab.com/ee/api/releases/links.html) is deprecated in GitLab 15.9 and will be +removed in GitLab 17.0. GitLab introduced a new field called `direct_asset_path` in GitLab 15.9 to replace `filepath` +until it is finally removed. -From GitLab 16.0, when you search for environments with the API, you must use at least three characters. This change helps us ensure the scalability of the search operation. +To avoid any disruptions, you should replace `filepath` with `direct_asset_path` in your calls to the following endpoints: + +- Releases API: + - [Create a release](https://docs.gitlab.com/ee/api/releases/#create-a-release) + - [Download a release asset](https://docs.gitlab.com/ee/api/releases/#download-a-release-asset) +- Release Links API: + - [Create a release link](https://docs.gitlab.com/ee/api/releases/links.html#create-a-release-link) + - [Update a release link](https://docs.gitlab.com/ee/api/releases/links.html#update-a-release-link) </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="17.0"> -### Legacy Gitaly configuration method +### GitLab Helm chart values `gitlab.kas.privateApi.*` are deprecated -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +We introduced the `global.kas.tls.*` Helm values to facilitate TLS communication between KAS and your Helm chart components. +The old values `gitlab.kas.privateApi.tls.enabled` and `gitlab.kas.privateApi.tls.secretName` are deprecated and scheduled for removal in GitLab 17.0. -Gitaly configuration within Omnibus GitLab has been updated such that all Gitaly related configuration keys are in a single -configuration structure that matches the standard Gitaly configuration. As such, the previous configuration structure is deprecated. +Because the new values provide a streamlined, comprehensive method to enable TLS for KAS, you should use `global.kas.tls.*` instead of `gitlab.kas.privateApi.tls.*`. The `gitlab.kas.privateApi.tls.*` For more information, see: -The single configuration structure is available from GitLab 15.10, though backwards compatibility is maintained. Once removed, Gitaly must be configured using the single -configuration structure. You should update the configuration of Gitaly at your earliest convenience. +- The [merge request](https://gitlab.com/gitlab-org/charts/gitlab/-/merge_requests/2888) that introduces the `global.kas.tls.*` values. +- The [deprecated `gitlab.kas.privateApi.tls.*` documentation](https://docs.gitlab.com/charts/charts/gitlab/kas/index.html#enable-tls-communication-through-the-gitlabkasprivateapi-attributes-deprecated). +- The [new `global.kas.tls.*` documentation](https://docs.gitlab.com/charts/charts/globals.html#tls-settings-1). -The change improves consistency between Omnibus GitLab and source installs and enables us to provide better documentation and tooling for both. +</div> -You should update to the new configuration structure as soon as possible using -[the upgrade instructions](https://docs.gitlab.com/ee/update/#gitaly-omnibus-gitlab-configuration-structure-change). +<div class="deprecation breaking-change" data-milestone="17.0"> + +### GitLab Runner platforms and setup instructions in GraphQL API +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> + +The `runnerPlatforms` and `runnerSetup` queries to get GitLab Runner platforms and installation instructions +are deprecated and will be removed from the GraphQL API. For installation instructions, you should use the +[GitLab Runner documentation](https://docs.gitlab.com/runner/) + </div> -<div class="announcement-milestone"> +<div class="deprecation breaking-change" data-milestone="17.0"> -## Announced in 15.9 +### GitLab Runner registration token in Runner Operator -<div class="deprecation removal-170 breaking-change"> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.6</span> +- End of Support: GitLab <span class="milestone">17.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -### Accessibility Testing is deprecated +The [`runner-registration-token`](https://docs.gitlab.com/runner/install/operator.html#install-the-kubernetes-operator) parameter that uses the OpenShift and k8s Vanilla Operator to install a runner on Kubernetes is deprecated. +We plan to implement a new method to bind runners to a GitLab instance +as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). +The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="17.0"> -Due to low customer usage, Accessibility Testing is deprecated and will be removed. There is no planned replacement and users should stop using Accessibility Testing before GitLab 17.0. +### Load Performance Testing is deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> - -### Browser Performance Testing is deprecated +Due to low customer usage, Load Performance Testing is deprecated and will be removed. There is no planned replacement and users should stop using Load Performance Testing before GitLab 17.0. -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="17.0"> -Due to limited customer usage, Browser Performance Testing is deprecated and will be removed. There is no planned replacement and users should stop using Browser Performance Testing before GitLab 17.0. +### Queue selector for running Sidekiq is deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- End of Support: GitLab <span class="milestone">16.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +Running Sidekiq with a [queue selector](https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#queue-selectors) (having multiple processes listening to a set of queues) and [negate settings](https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#negate-settings) is deprecated and will be fully removed in 17.0. -### CI/CD jobs will fail when no secret is returned from Hashicorp Vault +You can migrate away from queue selectors to [listening to all queues in all processes](https://docs.gitlab.com/ee/administration/sidekiq/extra_sidekiq_processes.html#start-multiple-processes). For example, if Sidekiq is currently running with 4 processes (denoted by 4 elements in `sidekiq['queue_groups']` in `/etc/gitlab/gitlab.rb`) with queue selector (`sidekiq['queue_selector'] = true`), you can change Sidekiq to listen to all queues in all 4 processes,for example `sidekiq['queue_groups'] = ['*'] * 4`. This approach is also recommended in our [Reference Architecture](https://docs.gitlab.com/ee/administration/reference_architectures/5k_users.html#configure-sidekiq). Note that Sidekiq can effectively run as many processes as the number of CPUs in the machine. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +While the above approach is recommended for most instances, Sidekiq can also be run using [routing rules](https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#routing-rules) which is also being used on GitLab.com. You can follow the [migration guide from queue selectors to routing rules](https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#migrating-from-queue-selectors-to-routing-rules). You need to take care with the migration to avoid losing jobs entirely. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -When using the native HashiCorp Vault integration, CI/CD jobs will fail when no secret is returned from Vault. Make sure your configuration always return a secret, or update your pipeline to handle this change, before GitLab 16.0. +<div class="deprecation breaking-change" data-milestone="17.0"> +### Registration tokens and server-side runner arguments in `POST /api/v4/runners` endpoint + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.6</span> +- End of Support: GitLab <span class="milestone">17.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The support for registration tokens and certain runner configuration arguments in the `POST` method operation on the `/api/v4/runners` endpoint is deprecated. +This endpoint [registers](https://docs.gitlab.com/ee/api/runners.html#register-a-new-runner) a runner +with a GitLab instance at the instance, group, or project level through the API. We plan to remove the support for +registration tokens and certain configuration arguments in this endpoint in GitLab 17.0. -### Default CI/CD job token (`CI_JOB_TOKEN`) scope changed +We plan to implement a new method to bind runners to a GitLab instance +as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). +The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). +This new architecture introduces a new method for registering runners and will eliminate the legacy +[runner registration token](https://docs.gitlab.com/ee/security/token_overview.html#runner-registration-tokens). +From GitLab 17.0 and later, the runner registration methods implemented by the new GitLab Runner token architecture will be the only supported methods. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="17.0"> -In GitLab 14.4 we introduced the ability to limit the "outbound" scope of the CI/CD job token (`CI_JOB_TOKEN`) to make it more secure. You can prevent job tokens from your project's pipelines from being used to access other projects. If needed, you can list specific projects that you want to access with your project's job tokens. +### Registration tokens and server-side runner arguments in `gitlab-runner register` command -In 15.9 we extended this functionality with a better solution, an "inbound" scope limit. You can prevent the job tokens from _other_ projects from being used to access your project. With this feature, you can optionally list specific projects that you want to allow to access your project with _their_ job token. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.6</span> +- End of Support: GitLab <span class="milestone">17.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -In 16.0, this inbound scope limit will be the only option available for all projects, and the outbound limit setting will be removed. To prepare for this change, you can enable the ["inbound" CI/CD job token limit](https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html#configure-the-job-token-scope-limit) feature now, and list any projects that need to access your project. +The support for registration tokens and certain configuration arguments in the command to [register](https://docs.gitlab.com/runner/register/) a runner, `gitlab-runner register` is deprecated. +We plan to implement a new method to bind runners to a GitLab instance +as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). +The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). +The new method will involve creating the runner in the GitLab UI and passing the +[runner authentication token](https://docs.gitlab.com/ee/security/token_overview.html#runner-authentication-tokens-also-called-runner-tokens) +to the `gitlab-runner register` command. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="17.0"> -### Deprecation and planned removal for `CI_PRE_CLONE_SCRIPT` variable on GitLab SaaS +### Self-managed certificate-based integration with Kubernetes -End of Support: GitLab <span class="removal-milestone">16.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The certificate-based integration with Kubernetes [will be deprecated and removed](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/). -The [`CI_PRE_CLONE_SCRIPT` variable](https://docs.gitlab.com/ee/ci/runners/saas/linux_saas_runner.html#pre-clone-script) supported by GitLab SaaS Runners is deprecated as of GitLab 15.9 and will be removed in 16.0. The `CI_PRE_CLONE_SCRIPT` variable enables you to run commands in your CI/CD job prior to the runner executing Git init and get fetch. For more information about how this feature works, see [Pre-clone script](https://docs.gitlab.com/ee/ci/runners/saas/linux_saas_runner.html#pre-clone-script). As an alternative, you can use the [`pre_get_sources_script`](https://docs.gitlab.com/ee/ci/yaml/#hookspre_get_sources_script). +As a self-managed customer, we are introducing the [feature flag](../administration/feature_flags.md#enable-or-disable-the-feature) `certificate_based_clusters` in GitLab 15.0 so you can keep your certificate-based integration enabled. However, the feature flag will be disabled by default, so this change is a **breaking change**. -</div> +In GitLab 17.0 we will remove both the feature and its related code. Until the final removal in 17.0, features built on this integration will continue to work, if you enable the feature flag. Until the feature is removed, GitLab will continue to fix security and critical issues as they arise. -<div class="deprecation removal-160 breaking-change"> +For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the +[agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) -### Development dependencies reported for PHP and Python +Although an explicit removal date is set, we don't plan to remove this feature until the new solution has feature parity. +For more information about the blockers to removal, see [this issue](https://gitlab.com/gitlab-org/configure/general/-/issues/199). -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +For updates and details about this deprecation, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -In GitLab 16.0 the GitLab Dependency Scanning analyzer will begin reporting development dependencies for both Python/pipenv and PHP/composer projects. Users who do not wish to have these development dependencies reported should set `DS_INCLUDE_DEV_DEPENDENCIES: false` in their CI/CD file. +<div class="deprecation breaking-change" data-milestone="17.0"> + +### Single database connection is deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +Previously, [GitLab's database](https://docs.gitlab.com/omnibus/settings/database.html) +configuration had a single `main:` section. This is being deprecated. The new +configuration has both a `main:` and a `ci:` section. -### Embedding Grafana panels in Markdown is deprecated +This deprecation affects users compiling GitLab from source, who will need +to [add the `ci:` section](https://docs.gitlab.com/ee/install/installation.html#configure-gitlab-db-settings). +Omnibus, the Helm chart, and Operator will handle this configuration +automatically from GitLab 16.0 onwards. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="17.0"> -The ability to add Grafana panels in GitLab Flavored Markdown is deprecated in 15.9 and will be removed in 16.0. -We intend to replace this feature with the ability to [embed charts](https://gitlab.com/groups/gitlab-org/opstrace/-/epics/33) with the [GitLab Observability UI](https://gitlab.com/gitlab-org/opstrace/opstrace-ui). +### Slack notifications integration +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- End of Support: GitLab <span class="milestone">17.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +As we're consolidating all Slack capabilities into the +GitLab for Slack app, we're [deprecating the Slack notifications +integration](https://gitlab.com/gitlab-org/gitlab/-/issues/372411). +GitLab.com users can now use the GitLab for Slack app to manage notifications +to their Slack workspace. For self-managed users of the Slack notifications integration, +we'll be introducing support in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/1211). -### Enforced validation of CI/CD parameter character lengths +</div> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="17.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Support for REST API endpoints that reset runner registration tokens -While CI/CD [job names](https://docs.gitlab.com/ee/ci/jobs/index.html#job-name-limitations) have a strict 255 character limit, other CI/CD parameters do not yet have validations ensuring they also stay under the limit. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- End of Support: GitLab <span class="milestone">17.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -In GitLab 16.0, validation will be added to strictly limit the following to 255 characters as well: +The support for runner registration tokens is deprecated. As a consequence, the REST API endpoints to reset a registration token are also deprecated and will +be removed in GitLab 17.0. +The deprecated endpoints are: -- The `stage` keyword. -- The `ref`, which is the Git branch or tag name for the pipeline. -- The `description` and `target_url` parameter, used by external CI/CD integrations. +- `POST /runners/reset_registration_token` +- `POST /projects/:id/runners/reset_registration_token` +- `POST /groups/:id/runners/reset_registration_token` -Users on self-managed instances should update their pipelines to ensure they do not use parameters that exceed 255 characters. Users on GitLab.com do not need to make any changes, as these are already limited in that database. +We plan to implement a new method to bind runners to a GitLab instance +as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). +The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). +This new architecture introduces a new method for registering runners and will eliminate the legacy +[runner registration token](https://docs.gitlab.com/ee/security/token_overview.html#runner-registration-tokens). +From GitLab 17.0 and later, the runner registration methods implemented by the new GitLab Runner token architecture will be the only supported methods. </div> -<div class="deprecation removal-166 breaking-change"> +<div class="deprecation breaking-change" data-milestone="17.0"> -### Error Tracking UI in GitLab Rails is deprecated +### The GitLab legacy requirement IID is deprecated in favor of work item IID -Planned removal: GitLab <span class="removal-milestone">16.6</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +We will be transitioning to a new IID as a result of moving requirements to a [work item type](https://docs.gitlab.com/ee/development/work_items.html#work-items-and-work-item-types). Users should begin using the new IID as support for the legacy IID and existing formatting will end in GitLab 17.0. The legacy requirement IID remains available until its removal in GitLab 17.0. -The [Error Tracking UI](https://docs.gitlab.com/ee/operations/error_tracking.html) is deprecated in 15.9 and will be removed in 16.6 (milestone might change) once GitLab Observability UI is made available. In future versions, you should use the [GitLab Observability UI](https://gitlab.com/gitlab-org/opstrace/opstrace-ui/), which will gradually be made available on GitLab.com over the next few releases. +</div> -During the transition to the GitLab Observability UI, we will migrate the [GitLab Observability Backend](https://gitlab.com/gitlab-org/opstrace/opstrace) from a per-cluster deployment model to a per-tenant deployment model. Because [Integrated Error Tracking](https://docs.gitlab.com/ee/operations/error_tracking.html#integrated-error-tracking) is in Open Beta, we will not migrate any existing user data. For more details about the migration, see the direction pages for: +<div class="deprecation breaking-change" data-milestone="17.0"> -- [Observability](https://about.gitlab.com/direction/monitor/observability/data-visualization/). -- The [Observability Backend](https://about.gitlab.com/direction/monitor/observability/data-management/). -- [Data visualization](https://about.gitlab.com/direction/monitor/observability/data-visualization/). +### The Visual Reviews tool is deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### External field in GraphQL ReleaseAssetLink type +Due to limited customer usage and capabilities, the Visual Reviews feature for Review Apps is deprecated and will be removed. There is no planned replacement and users should stop using Visual Reviews before GitLab 17.0. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="17.0"> -In the [GraphQL API](https://docs.gitlab.com/ee/api/graphql), the `external` field of [`ReleaseAssetLink` type](https://docs.gitlab.com/ee/api/graphql/reference/index.html#releaseassetlink) was used to indicate whether a [release link](https://docs.gitlab.com/ee/user/project/releases/release_fields.html#links) is internal or external to your GitLab instance. -As of GitLab 15.9, we treat all release links as external, and therefore, this field is deprecated in GitLab 15.9, and will be removed in GitLab 16.0. -To avoid any disruptions to your workflow, please stop using the `external` field because it will be removed and will not be replaced. +### The `gitlab-runner exec` command is deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- End of Support: GitLab <span class="milestone">17.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### External field in Releases and Release Links APIs +The [`gitlab-runner exec`](https://docs.gitlab.com/runner/commands/#gitlab-runner-exec) command is deprecated and will be fully removed from GitLab Runner in 16.0. The `gitlab-runner exec` feature was initially developed to provide the ability to validate a GitLab CI pipeline on a local system without needing to commit the updates to a GitLab instance. However, with the continued evolution of GitLab CI, replicating all GitLab CI features into `gitlab-runner exec` was no longer viable. Pipeline syntax and validation [simulation](https://docs.gitlab.com/ee/ci/pipeline_editor/#simulate-a-cicd-pipeline) are available in the GitLab pipeline editor. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="17.0"> -In [Releases API](https://docs.gitlab.com/ee/api/releases) and [Release Links API](https://docs.gitlab.com/ee/api/releases/links.html), the `external` field was used to indicate whether a [release link](https://docs.gitlab.com/ee/user/project/releases/release_fields.html#links) is internal or external to your GitLab instance. -As of GitLab 15.9, we treat all release links as external, and therefore, this field is deprecated in GitLab 15.9, and will be removed in GitLab 16.0. -To avoid any disruptions to your workflow, please stop using the `external` field because it will be removed and will not be replaced. +### Trigger jobs can mirror downstream pipeline status exactly +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> +In some cases, like when a downstream pipeline had the `passed with warnings` status, trigger jobs that were using [`strategy: depend`](https://docs.gitlab.com/ee/ci/yaml/index.html#strategydepend) did not mirror the status of the downstream pipeline exactly. In GitLab 17.0 trigger jobs will show the exact same status as the the downstream pipeline. If your pipeline relied on this behavior, you should update your pipeline to handle the more accurate status. -### Filepath field in Releases and Release Links APIs +</div> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="17.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### `runnerRegistrationToken` parameter for GitLab Runner Helm Chart -Support for specifying a `filepath` for a direct asset link in the [Releases API](https://docs.gitlab.com/ee/api/releases) -and [Release Links API](https://docs.gitlab.com/ee/api/releases/links.html) is deprecated in GitLab 15.9 and will be -removed in GitLab 17.0. GitLab introduced a new field called `direct_asset_path` in GitLab 15.9 to replace `filepath` -until it is finally removed. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.6</span> +- End of Support: GitLab <span class="milestone">17.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -To avoid any disruptions, you should replace `filepath` with `direct_asset_path` in your calls to the following endpoints: +The [`runnerRegistrationToken`](https://docs.gitlab.com/runner/install/kubernetes.html#required-configuration) parameter to use the GitLab Helm Chart to install a runner on Kubernetes is deprecated. -- Releases API: - - [Create a release](https://docs.gitlab.com/ee/api/releases/#create-a-release) - - [Download a release asset](https://docs.gitlab.com/ee/api/releases/#download-a-release-asset) -- Release Links API: - - [Create a release link](https://docs.gitlab.com/ee/api/releases/links.html#create-a-release-link) - - [Update a release link](https://docs.gitlab.com/ee/api/releases/links.html#update-a-release-link) +We plan to implement a new method to bind runners to a GitLab instance leveraging `runnerToken` +as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). +The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). -</div> +From GitLab 17.0 and later, the methods to register runners introduced by the new GitLab Runner token architecture will be the only supported methods. -<div class="deprecation removal-170 breaking-change"> +</div> +</div> -### GitLab Runner platforms and setup instructions in GraphQL API +<div class="milestone-wrapper" data-milestone="16.6"> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +## GitLab 16.6 -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.6"> -The `runnerPlatforms` and `runnerSetup` queries to get GitLab Runner platforms and installation instructions -are deprecated and will be removed from the GraphQL API. For installation instructions, you should use the -[GitLab Runner documentation](https://docs.gitlab.com/runner/) +### Error Tracking UI in GitLab Rails is deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The [Error Tracking UI](https://docs.gitlab.com/ee/operations/error_tracking.html) is deprecated in 15.9 and will be removed in 16.6 (milestone might change) once GitLab Observability UI is made available. In future versions, you should use the [GitLab Observability UI](https://gitlab.com/gitlab-org/opstrace/opstrace-ui/), which will gradually be made available on GitLab.com over the next few releases. -### HashiCorp Vault integration will no longer use CI_JOB_JWT by default +During the transition to the GitLab Observability UI, we will migrate the [GitLab Observability Backend](https://gitlab.com/gitlab-org/opstrace/opstrace) from a per-cluster deployment model to a per-tenant deployment model. Because [Integrated Error Tracking](https://docs.gitlab.com/ee/operations/error_tracking.html#integrated-error-tracking) is in Open Beta, we will not migrate any existing user data. For more details about the migration, see the direction pages for: -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +- [Observability](https://about.gitlab.com/direction/monitor/observability/data-visualization/). +- The [Observability Backend](https://about.gitlab.com/direction/monitor/observability/data-management/). +- [Data visualization](https://about.gitlab.com/direction/monitor/observability/data-visualization/). -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> +</div> -As part of our effort to improve the security of your CI workflows using JWT and OIDC, the native HashiCorp integration is also being updated in GitLab 16.0. Any projects that use the [`secrets:vault`](https://docs.gitlab.com/ee/ci/yaml/#secretsvault) keyword to retrieve secrets from Vault will need to be [configured to use ID tokens](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html#configure-automatic-id-token-authentication). +<div class="milestone-wrapper" data-milestone="16.1"> -To be prepared for this change, you should do the following before GitLab 16.0: +## GitLab 16.1 -- [Disable the use of JSON web tokens](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html#enable-automatic-id-token-authentication) in the pipeline. -- Ensure the bound audience is prefixed with `https://`. -- Use the new [`id_tokens`](https://docs.gitlab.com/ee/ci/yaml/#id_tokens) keyword - and configure the `aud` claim. +<div class="deprecation " data-milestone="16.1"> +### GitLab Runner images based on Alpine 3.12, 3.13, 3.14 + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.11</span> +- End of Support: GitLab <span class="milestone">16.1</span> </div> -<div class="deprecation removal-160 breaking-change"> +We will stop publishing runner images based on the following, end-of-life Alpine versions: -### Legacy Praefect configuration method +- Alpine 3.12 +- Alpine 3.13 +- Alpine 3.14 (end-of-life on 2023-05-23) -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="milestone-wrapper" data-milestone="16.0"> -Previously, Praefect configuration keys were scattered throughout the configuration file. Now, these are in a single configuration structure that matches -Praefect configuration so the previous configuration method is deprecated. +## GitLab 16.0 -The single configuration structure available from GitLab 15.9, though backwards compatibility is maintained. Once removed, Praefect must be configured using the single -configuration structure. You should update your Praefect configuration as soon as possible using -[the upgrade instructions](https://docs.gitlab.com/ee/update/#praefect-omnibus-gitlab-configuration-structure-change). +<div class="deprecation breaking-change" data-milestone="16.0"> -This change brings Praefect configuration in Omnibus GitLab in line with the configuration structure of Praefect. Previously, the hierarchies and configuration keys -didn't match. The change improves consistency between Omnibus GitLab and source installs and enables us to provide better documentation and tooling for both. +### Atlassian Crowd OmniAuth provider +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.3</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The `omniauth_crowd` gem that provides GitLab with the Atlassian Crowd OmniAuth provider will be removed in our +next major release, GitLab 16.0. This gem sees very little use and its +[lack of compatibility](https://github.com/robdimarco/omniauth_crowd/issues/37) with OmniAuth 2.0 is +[blocking our upgrade](https://gitlab.com/gitlab-org/gitlab/-/issues/30073). -### Legacy URLs replaced or removed +</div> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Auto DevOps no longer provisions a PostgreSQL database by default -GitLab 16.0 removes legacy URLs from the GitLab application. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -When subgroups were introduced in GitLab 9.0, a `/-/` delimiter was added to URLs to signify the end of a group path. All GitLab URLs now use this delimiter for project, group, and instance level features. +Currently, Auto DevOps provisions an in-cluster PostgreSQL database by default. +In GitLab 16.0, databases will be provisioned only for users who opt in. This +change supports production deployments that require more robust database management. -URLs that do not use the `/-/` delimiter are planned for removal in GitLab 16.0. For the full list of these URLs, along with their replacements, see [issue 28848](https://gitlab.com/gitlab-org/gitlab/-/issues/28848#release-notes). +If you want Auto DevOps to provision an in-cluster database, +set the `POSTGRES_ENABLED` CI/CD variable to `true`. -Update any scripts or bookmarks that reference the legacy URLs. GitLab APIs are not affected by this change. +</div> + +<div class="deprecation breaking-change" data-milestone="16.0"> +### Azure Storage Driver defaults to the correct root prefix + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The Azure Storage Driver writes to `//` as the default root directory. This default root directory appears in some places within the Azure UI as `/<no-name>/`. We have maintained this legacy behavior to support older deployments using this storage driver. However, when moving to Azure from another storage driver, this behavior hides all your data until you configure the storage driver to build root paths without an extra leading slash by setting `trimlegacyrootprefix: true`. -### License Compliance CI Template +The new default configuration for the storage driver will set `trimlegacyrootprefix: true`, and `/` will be the default root directory. You can add `trimlegacyrootprefix: false` to your current configuration to avoid any disruptions. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +This breaking change will happen in GitLab 16.0. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -The GitLab [License Compliance](https://docs.gitlab.com/ee/user/compliance/license_compliance/) CI template is now deprecated and is scheduled for removal in the GitLab 16.0 release. Users who wish to continue using GitLab for License Compliance should remove the License Compliance template from their CI pipeline and add the [Dependency Scanning template](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuration). The Dependency Scanning template is now capable of gathering the required license information so it is no longer necessary to run a separate License Compliance job. The License Compliance CI template should not be removed prior to verifying that the `license_scanning_sbom_scanner` and `package_metadata_synchronization` flags are enabled for the instance and that the instance has been upgraded to a version that supports [the new method of license scanning](https://docs.gitlab.com/ee/user/compliance/license_scanning_of_cyclonedx_files/). +<div class="deprecation breaking-change" data-milestone="16.0"> -| CI Pipeline Includes | GitLab <= 15.8 | 15.9 <= GitLab < 16.0 | GitLab >= 16.0 | -| ------------- | ------------- | ------------- | ------------- | -| Both DS and LS templates | License data from LS job is used | License data from LS job is used | License data from DS job is used | -| DS template is included but LS template is not | No license data | License data from DS job is used | License data from DS job is used | -| LS template is included but DS template is not | License data from LS job is used | License data from LS job is used | No license data | +### Bundled Grafana Helm Chart is deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The Grafana Helm chart that is bundled with the GitLab Helm Chart is deprecated and will be removed in the GitLab Helm Chart 7.0 release (releasing along with GitLab 16.0). -### License-Check and the Policies tab on the License Compliance page +The bundled Grafana Helm chart is an optional service that can be turned on to provide the Grafana UI connected to the GitLab Helm Chart's Prometheus metrics. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +The version of Grafana that the GitLab Helm Chart is currently providing is no longer a supported Grafana version. +If you're using the bundled Grafana, you should switch to the [newer chart version from Grafana Labs](https://artifacthub.io/packages/helm/grafana/grafana) +or a Grafana Operator from a trusted provider. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +In your new Grafana instance, you can [configure the GitLab provided Prometheus as a data source](https://docs.gitlab.com/ee/administration/monitoring/performance/grafana_configuration.html#integration-with-gitlab-ui) +and [connect Grafana to the GitLab UI](https://docs.gitlab.com/ee/administration/monitoring/performance/grafana_configuration.html#integration-with-gitlab-ui). -The [License-Check feature](https://docs.gitlab.com/ee/user/compliance/license_check_rules.html) is now deprecated and is scheduled for removal in GitLab 16.0. Additionally, the Policies tab on the License Compliance page and all APIs related to the License-Check feature are deprecated and planned for removal in GitLab 16.0. Users who wish to continue to enforce approvals based on detected licenses are encouraged to create a new [License Approval policy](https://docs.gitlab.com/ee/user/compliance/license_approval_policies.html) instead. +</div> +<div class="deprecation breaking-change" data-milestone="16.0"> + +### CAS OmniAuth provider + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.3</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> +The `omniauth-cas3` gem that provides GitLab with the CAS OmniAuth provider will be removed in our next major +release, GitLab 16.0. This gem sees very little use and its lack of upstream maintenance is preventing GitLab's +[upgrade to OmniAuth 2.0](https://gitlab.com/gitlab-org/gitlab/-/issues/30073). -### Load Performance Testing is deprecated +</div> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### CI/CD jobs will fail when no secret is returned from Hashicorp Vault -Due to low customer usage, Load Performance Testing is deprecated and will be removed. There is no planned replacement and users should stop using Load Performance Testing before GitLab 17.0. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +When using the native HashiCorp Vault integration, CI/CD jobs will fail when no secret is returned from Vault. Make sure your configuration always return a secret, or update your pipeline to handle this change, before GitLab 16.0. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Managed Licenses API +### Changing merge request approvals with the `/approvals` API endpoint -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +To change the approvals required for a merge request, you should no longer use the `/approvals` API endpoint, which was deprecated in GitLab 14.0. -The [Managed Licenses API](https://docs.gitlab.com/ee/api/managed_licenses.html) is now deprecated and is scheduled for removal in GitLab 16.0. +Instead, use the [`/approval_rules` endpoint](https://docs.gitlab.com/ee/api/merge_request_approvals.html#merge-request-level-mr-approvals) to [create](https://docs.gitlab.com/ee/api/merge_request_approvals.html#create-merge-request-level-rule) or [update](https://docs.gitlab.com/ee/api/merge_request_approvals.html#update-merge-request-level-rule) the approval rules for a merge request. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Old versions of JSON web tokens are deprecated +### CiCdSettingsUpdate mutation renamed to ProjectCiCdSettingsUpdate -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The `CiCdSettingsUpdate` mutation was renamed to `ProjectCiCdSettingsUpdate` in GitLab 15.0. +The `CiCdSettingsUpdate` mutation will be removed in GitLab 16.0. +Any user scripts that use the `CiCdSettingsUpdate` mutation must be updated to use `ProjectCiCdSettingsUpdate` +instead. -Now that we have released [ID tokens](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html) -with OIDC support, the old JSON web tokens are deprecated and will be removed. -Both the `CI_JOB_JWT` and `CI_JOB_JWT_V2` tokens, exposed to jobs as predefined variables, -will no longer be available in GitLab 16.0. +</div> -To prepare for this change, you should: +<div class="deprecation breaking-change" data-milestone="16.0"> -- Configure your pipelines to use the fully configurable and more secure - [`id_token`](https://docs.gitlab.com/ee/ci/yaml/index.html#id_tokens) keyword instead. -- [Enable the **Limit JSON Web Token (JWT) access**](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html#enable-automatic-id-token-authentication) - setting, which prevents the old tokens from being exposed to any jobs. This setting - will be permanently enabled for all projects in GitLab 16.0. +### Conan project-level search endpoint returns project-specific results +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +You can use the GitLab Conan repository with [project-level](https://docs.gitlab.com/ee/user/packages/conan_repository/#add-a-remote-for-your-project) or [instance-level](https://docs.gitlab.com/ee/user/packages/conan_repository/#add-a-remote-for-your-instance) endpoints. Each level supports the conan search command. However, the search endpoint for the project level is also returning packages from outside the target project. -### Option to delete projects immediately is deprecated from deletion protection settings +This unintended functionality is deprecated in GitLab 15.8 and will be removed in GitLab 16.0. The search endpoint for the project level will only return packages from the target project. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -The group and project deletion protection setting in the Admin Area had an option to delete groups and projects immediately. Starting with 16.0, this option will no longer be available, and delayed group and project deletion will become the default behavior. +### Configuration fields in GitLab Runner Helm Chart -The option will no longer appear as a group setting. Self-managed users will still have the option to define the deletion delay period, and SaaS users have a non-adjustable default retention period of 7 days. Users can still immediately delete the project from the project settings, and the group from the group settings. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.6</span> +- End of Support: GitLab <span class="milestone">16.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -The option to delete groups and projects immediately by default was deprecated to prevent users from accidentally taking this action and permanently losing groups and projects. +From GitLab 13.6, users can [specify any runner configuration in the GitLab Runner Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html). When we implemented this feature, we deprecated values in the GitLab Helm Chart configuration that were specific to GitLab Runner. These fields are deprecated and we plan to remove them in v1.0 of the GitLab Runner Helm chart. </div> -<div class="deprecation removal-170 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Queue selector for running Sidekiq is deprecated +### Configuring Redis config file paths using environment variables is deprecated -End of Support: GitLab <span class="removal-milestone">16.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +You can no longer specify Redis configuration file locations +using the environment variables like `GITLAB_REDIS_CACHE_CONFIG_FILE` or +`GITLAB_REDIS_QUEUES_CONFIG_FILE`. Use the default +config file locations instead, for example `config/redis.cache.yml` or +`config/redis.queues.yml`. -Running Sidekiq with a [queue selector](https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#queue-selectors) (having multiple processes listening to a set of queues) and [negate settings](https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#negate-settings) is deprecated and will be fully removed in 17.0. +</div> -You can migrate away from queue selectors to [listening to all queues in all processes](https://docs.gitlab.com/ee/administration/sidekiq/extra_sidekiq_processes.html#start-multiple-processes). For example, if Sidekiq is currently running with 4 processes (denoted by 4 elements in `sidekiq['queue_groups']` in `/etc/gitlab/gitlab.rb`) with queue selector (`sidekiq['queue_selector'] = true`), you can change Sidekiq to listen to all queues in all 4 processes,for example `sidekiq['queue_groups'] = ['*'] * 4`. This approach is also recommended in our [Reference Architecture](https://docs.gitlab.com/ee/administration/reference_architectures/5k_users.html#configure-sidekiq). Note that Sidekiq can effectively run as many processes as the number of CPUs in the machine. +<div class="deprecation breaking-change" data-milestone="16.0"> -While the above approach is recommended for most instances, Sidekiq can also be run using [routing rules](https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#routing-rules) which is also being used on GitLab.com. You can follow the [migration guide from queue selectors to routing rules](https://docs.gitlab.com/ee/administration/sidekiq/processing_specific_job_classes.html#migrating-from-queue-selectors-to-routing-rules). You need to take care with the migration to avoid losing jobs entirely. +### Container Registry pull-through cache +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The Container Registry pull-through cache is deprecated in GitLab 15.8 and will be removed in GitLab 16.0. While the Container Registry pull-through cache functionality is useful, we have not made significant changes to this feature. You can use the upstream version of the container registry to achieve the same functionality. Removing the pull-through cache allows us also to remove the upstream client code without sacrificing functionality. -### Remove offset pagination from Jobs API +</div> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Container Scanning variables that reference Docker -A request to the API for `/api/v4/projects/:id/jobs` can return a paginated list of jobs. Projects can contain hundreds or thousands of jobs, so using an offset to paginate through them is slow. Users should instead use [`keyset-based pagination`](https://docs.gitlab.com/ee/api/rest/index.html#keyset-based-pagination) when requesting consecutive pages of results. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.4</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -In milestone 16.0 we will remove offset-based pagination. +All Container Scanning variables that are prefixed by `DOCKER_` in variable name are deprecated. This includes the `DOCKER_IMAGE`, `DOCKER_PASSWORD`, `DOCKER_USER`, and `DOCKERFILE_PATH` variables. Support for these variables will be removed in the GitLab 16.0 release. Use the [new variable names](https://docs.gitlab.com/ee/user/application_security/container_scanning/#available-cicd-variables) `CS_IMAGE`, `CS_REGISTRY_PASSWORD`, `CS_REGISTRY_USER`, and `CS_DOCKERFILE_PATH` in place of the deprecated names. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Required Pipeline Configuration is deprecated +### Cookie authorization in the GitLab for Jira Cloud app -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +Cookie authentication in the GitLab for Jira Cloud app is now deprecated in favor of OAuth authentication. +On self-managed, you must [set up OAuth authentication](https://docs.gitlab.com/ee/integration/jira/connect-app.html#set-up-oauth-authentication-for-self-managed-instances) +to continue to use the GitLab for Jira Cloud app. Without OAuth, you can't manage linked namespaces. -Required Pipeline Configuration will be removed in the 16.0 release. This impacts self-managed users on the Ultimate license. +</div> -We recommend replacing this with an alternative [compliance solution](https://docs.gitlab.com/ee/user/group/compliance_frameworks.html#compliance-pipelines) -that is available now. We recommend this alternative solution because it provides greater flexibility, allowing required pipelines to be assigned to specific compliance framework labels. +<div class="deprecation breaking-change" data-milestone="16.0"> +### DAST API scans using DAST template is deprecated + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +With the move to the new DAST API analyzer and the `DAST-API.gitlab-ci.yml` template for DAST API scans, we will be removing the ability to scan APIs with the DAST analyzer. Use of the `DAST.gitlab-ci.yml` or `DAST-latest.gitlab-ci.yml` templates for API scans is deprecated as of GitLab 15.7 and will no longer work in GitLab 16.0. Please use `DAST-API.gitlab-ci.yml` template and refer to the [DAST API analyzer](https://docs.gitlab.com/ee/user/application_security/dast_api/#configure-dast-api-with-an-openapi-specification) documentation for configuration details. -### SAST analyzer coverage changing in GitLab 16.0 +</div> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### DAST API variables -GitLab SAST uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/sast/analyzers/) to scan code for vulnerabilities. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -We're reducing the number of supported analyzers used by default in GitLab SAST. -This is part of our long-term strategy to deliver a faster, more consistent user experience across different programming languages. +With the switch to the new DAST API analyzer in GitLab 15.6, two legacy DAST API variables are being deprecated. The variables `DAST_API_HOST_OVERRIDE` and `DAST_API_SPECIFICATION` will no longer be used for DAST API scans. -Starting in GitLab 16.0, the GitLab SAST CI/CD template will no longer use the following analyzers, and they will enter End of Support status: +`DAST_API_HOST_OVERRIDE` has been deprecated in favor of using the `DAST_API_TARGET_URL` to automatically override the host in the OpenAPI specification. -- [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) (.NET) -- [PHPCS Security Audit](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) (PHP) +`DAST_API_SPECIFICATION` has been deprecated in favor of `DAST_API_OPENAPI`. To continue using an OpenAPI specification to guide the test, users must replace the `DAST_API_SPECIFICATION` variable with the `DAST_API_OPENAPI` variable. The value can remain the same, but the variable name must be replaced. -We'll remove these analyzers from the [SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) and replace them with GitLab-supported detection rules and the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). -Effective immediately, these analyzers will receive only security updates; other routine improvements or updates are not guaranteed. -After these analyzers reach End of Support, no further updates will be provided. -However, we won't delete container images previously published for these analyzers or remove the ability to run them by using a custom CI/CD pipeline job. +These two variables will be removed in GitLab 16.0. -We will also remove Scala from the scope of the [SpotBugs-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) and replace it with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). -This change will make it simpler to scan Scala code; compilation will no longer be required. -This change will be reflected in the automatic language detection portion of the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml). -Note that the SpotBugs-based analyzer will continue to cover Groovy and Kotlin. +</div> -If you've already dismissed a vulnerability finding from one of the deprecated analyzers, the replacement attempts to respect your previous dismissal. The system behavior depends on: +<div class="deprecation breaking-change" data-milestone="16.0"> -- whether you've excluded the Semgrep-based analyzer from running in the past. -- which analyzer first discovered the vulnerabilities shown in the project's Vulnerability Report. +### DAST ZAP advanced configuration variables deprecation -See [Vulnerability translation documentation](https://docs.gitlab.com/ee/user/application_security/sast/analyzers.html#vulnerability-translation) for further details. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -If you applied customizations to any of the affected analyzers or if you currently disable the Semgrep analyzer in your pipelines, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/390416#breaking-change). +With the new browser-based DAST analyzer GA in GitLab 15.7, we are working towards making it the default DAST analyzer at some point in the future. In preparation for this, the following legacy DAST variables are being deprecated and scheduled for removal in GitLab 16.0: `DAST_ZAP_CLI_OPTIONS` and `DAST_ZAP_LOG_CONFIGURATION`. These variables allowed for advanced configuration of the legacy DAST analyzer, which was based on OWASP ZAP. The new browser-based analyzer will not include the same functionality, as these were specific to how ZAP worked. + +These three variables will be removed in GitLab 16.0. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Secure analyzers major version update +### DAST report variables deprecation -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +With the new browser-based DAST analyzer GA in GitLab 15.7, we are working towards making it the default DAST analyzer at some point in the future. In preparation for this, the following legacy DAST variables are being deprecated and scheduled for removal in GitLab 16.0: `DAST_HTML_REPORT`, `DAST_XML_REPORT`, and `DAST_MARKDOWN_REPORT`. These reports relied on the legacy DAST analyzer and we do not plan to implement them in the new browser-based analyzer. As of GitLab 16.0, these report artifacts will no longer be generated. -The Secure stage will be bumping the major versions of its analyzers in tandem with the GitLab 16.0 release. This bump will enable a clear delineation for analyzers, between: +These three variables will be removed in GitLab 16.0. -- Those released prior to May 22, 2023 -- Those released after May 22, 2023 +</div> -If you are not using the default included templates, or have pinned your analyzer versions you will need to update your CI/CD job definition to either remove the pinned version or to update the latest major version. -Users of GitLab 13.0-15.10 will continue to experience analyzer updates as normal until the release of GitLab 16.0, following which all newly fixed bugs and released features will be released only in the new major version of the analyzers. We do not backport bugs and features to deprecated versions as per our [maintenance policy](https://docs.gitlab.com/ee/policy/maintenance.html). As required, security patches will be backported within the latest 3 minor releases. -Specifically, the following are being deprecated and will no longer be updated after 16.0 GitLab release: +<div class="deprecation breaking-change" data-milestone="16.0"> -- API Fuzzing: version 2 -- Container Scanning: version 5 -- Coverage-guided fuzz testing: version 3 -- Dependency Scanning: version 3 -- Dynamic Application Security Testing (DAST): version 3 -- DAST API: version 2 -- IaC Scanning: version 3 -- License Scanning: version 4 -- Secret Detection: version 4 -- Static Application Security Testing (SAST): version 3 of [all analyzers](https://docs.gitlab.com/ee/user/application_security/sast/#supported-languages-and-frameworks) - - `brakeman`: version 3 - - `flawfinder`: version 3 - - `kubesec`: version 3 - - `mobsf`: version 3 - - `nodejs-scan`: version 3 - - `phpcs-security-audit`: version 3 - - `pmd-apex`: version 3 - - `security-code-scan`: version 3 - - `semgrep`: version 3 - - `sobelow`: version 3 - - `spotbugs`: version 3 +### Default CI/CD job token (`CI_JOB_TOKEN`) scope changed +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +In GitLab 14.4 we introduced the ability to limit the "outbound" scope of the CI/CD job token (`CI_JOB_TOKEN`) to make it more secure. You can prevent job tokens from your project's pipelines from being used to access other projects. If needed, you can list specific projects that you want to access with your project's job tokens. -### Secure scanning CI/CD templates will use new job `rules` +In 15.9 we extended this functionality with a better solution, an "inbound" scope limit. You can prevent the job tokens from _other_ projects from being used to access your project. With this feature, you can optionally list specific projects that you want to allow to access your project with _their_ job token. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +In 16.0, this inbound scope limit will be the only option available for all projects, and the outbound limit setting will be disabled. To prepare for this change, you can enable the ["inbound" CI/CD job token limit](https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html#configure-the-job-token-scope-limit) feature now, and list any projects that need to access your project. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -GitLab-managed CI/CD templates for security scanning will be updated in the GitLab 16.0 release. -The updates will include improvements already released in the Latest versions of the CI/CD templates. -We released these changes in the Latest template versions because they have the potential to disrupt customized CI/CD pipeline configurations. +<div class="deprecation breaking-change" data-milestone="16.0"> -In all updated templates, we're: +### Dependency Scanning support for Java 13, 14, 15, and 16 -- Adding support for running scans in merge request (MR) pipelines. -- Updating the definition of variables like `SAST_DISABLED` and `DEPENDENCY_SCANNING_DISABLED` to disable scanning only if the value is `"true"`. Previously, even if the value were `"false"`, scanning would be disabled. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -The following templates will be updated: +GitLab has deprecated Dependency Scanning support for Java versions 13, 14, 15, and 16 and plans to remove that support in the upcoming GitLab 16.0 release. This is consistent with [Oracle's support policy](https://www.oracle.com/java/technologies/java-se-support-roadmap.html) as Oracle Premier and Extended Support for these versions has ended. This also allows GitLab to focus Dependency Scanning Java support on LTS versions moving forward. -- API Fuzzing: [`API-Fuzzing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) -- Container Scanning: [`Container-Scanning.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Container-Scanning.gitlab-ci.yml) -- Coverage-Guided Fuzzing: [`Coverage-Fuzzing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Coverage-Fuzzing.gitlab-ci.yml) -- DAST: [`DAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml) -- DAST API: [`DAST-API.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) -- Dependency Scanning: [`Dependency-Scanning.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml) -- IaC Scanning: [`SAST-IaC.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml) -- SAST: [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) -- Secret Detection: [`Secret-Detection.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Secret-Detction.gitlab-ci.yml) +</div> -We recommend that you test your pipelines before the 16.0 release if you use one of the templates listed above and you do any of the following: +<div class="deprecation breaking-change" data-milestone="16.0"> - 1. You override `rules` for your security scanning jobs. - 1. You use the `_DISABLED` variables but set a value other than `"true"`. +### Deployment API returns error when `updated_at` and `updated_at` are not used together +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> +The Deployment API will now return an error when `updated_at` filtering and `updated_at` sorting are not used together. Some users were using filtering by `updated_at` to fetch "latest" deployment without using `updated_at` sorting, which may produce wrong results. You should instead use them together, or migrate to filtering by `finished_at` and sorting by `finished_at` which will give you "latest deployments" in a consistent way. -### Single database connection is deprecated +</div> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Deprecate legacy Gitaly configuration methods -Previously, [GitLab's database](https://docs.gitlab.com/omnibus/settings/database.html) -configuration had a single `main:` section. This is being deprecated. The new -configuration has both a `main:` and a `ci:` section. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -This deprecation affects users compiling GitLab from source, who will need -to [add the `ci:` section](https://docs.gitlab.com/ee/install/installation.html#configure-gitlab-db-settings). -Omnibus, the Helm chart, and Operator will handle this configuration -automatically from GitLab 16.0 onwards. +Using environment variables `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configure Gitaly is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352609). +These variables are being replaced with standard [`config.toml` Gitaly configuration](https://docs.gitlab.com/ee/administration/gitaly/reference.html). + +GitLab instances that use `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configure Gitaly should switch to configuring using +`config.toml`. </div> -<div class="deprecation removal-170 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Slack notifications integration +### Deprecated Consul http metrics + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -End of Support: GitLab <span class="removal-milestone">17.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +The Consul provided in the GitLab Omnibus package will no longer provide older deprecated Consul metrics starting in GitLab 16.0. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +In GitLab 14.0, [Consul was updated to 1.9.6](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5344), +which deprecated some telemetry metrics from being at the `consul.http` path. In GitLab 16.0, the `consul.http` path will be removed. -As we're consolidating all Slack capabilities into the -GitLab for Slack app, we're [deprecating the Slack notifications -integration](https://gitlab.com/gitlab-org/gitlab/-/issues/372411). -GitLab.com users can now use the GitLab for Slack app to manage notifications -to their Slack workspace. For self-managed users of the Slack notifications integration, -we'll be introducing support in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/1211). +If you have monitoring that consumes Consul metrics, update them to use `consul.api.http` instead of `consul.http`. +For more information, see [the deprecation notes for Consul 1.9.0](https://github.com/hashicorp/consul/releases/tag/v1.9.0). </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Support for Praefect custom metrics endpoint configuration +### Deprecation and planned removal for `CI_PRE_CLONE_SCRIPT` variable on GitLab SaaS -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- End of Support: GitLab <span class="milestone">16.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The [`CI_PRE_CLONE_SCRIPT` variable](https://docs.gitlab.com/ee/ci/runners/saas/linux_saas_runner.html#pre-clone-script) supported by GitLab SaaS Runners is deprecated as of GitLab 15.9 and will be removed in 16.0. The `CI_PRE_CLONE_SCRIPT` variable enables you to run commands in your CI/CD job prior to the runner executing Git init and get fetch. For more information about how this feature works, see [Pre-clone script](https://docs.gitlab.com/ee/ci/runners/saas/linux_saas_runner.html#pre-clone-script). As an alternative, you can use the [`pre_get_sources_script`](https://docs.gitlab.com/ee/ci/yaml/#hookspre_get_sources_script). -Support for using the `prometheus_exclude_database_from_default_metrics` configuration value is deprecated in GitLab -15.9 and will be removed in GitLab 16.0. We are removing this configuration value because using it is non-performant. -This change means the following metrics will become unavailable on `/metrics`: +</div> -- `gitaly_praefect_unavailable_repositories`. -- `gitaly_praefect_verification_queue_depth`. -- `gitaly_praefect_replication_queue_depth`. +<div class="deprecation breaking-change" data-milestone="16.0"> -This may require updating your metrics collection targets to also scrape `/db_metrics`. +### Developer role providing the ability to import projects to a group +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> - -### The GitLab legacy requirement IID is deprecated in favor of work item IID +The ability for users with the Developer role for a group to import projects to that group is deprecated in GitLab +15.8 and will be removed in GitLab 16.0. From GitLab 16.0, only users with at least the Maintainer role for a group +will be able to import projects to that group. -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -We will be transitioning to a new IID as a result of moving requirements to a [work item type](https://docs.gitlab.com/ee/development/work_items.html#work-items-and-work-item-types). Users should begin using the new IID as support for the legacy IID and existing formatting will end in GitLab 17.0. The legacy requirement IID remains available until its removal in GitLab 17.0. +### Development dependencies reported for PHP and Python +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> +In GitLab 16.0 the GitLab Dependency Scanning analyzer will begin reporting development dependencies for both Python/pipenv and PHP/composer projects. Users who do not wish to have these development dependencies reported should set `DS_INCLUDE_DEV_DEPENDENCIES: false` in their CI/CD file. -### Trigger jobs can mirror downstream pipeline status exactly +</div> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Embedding Grafana panels in Markdown is deprecated -In some cases, like when a downstream pipeline had the `passed with warnings` status, trigger jobs that were using [`strategy: depend`](https://docs.gitlab.com/ee/ci/yaml/index.html#strategydepend) did not mirror the status of the downstream pipeline exactly. In GitLab 17.0 trigger jobs will show the exact same status as the the downstream pipeline. If your pipeline relied on this behavior, you should update your pipeline to handle the more accurate status. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +The ability to add Grafana panels in GitLab Flavored Markdown is deprecated in 15.9 and will be removed in 16.0. +We intend to replace this feature with the ability to [embed charts](https://gitlab.com/groups/gitlab-org/opstrace/-/epics/33) with the [GitLab Observability UI](https://gitlab.com/gitlab-org/opstrace/opstrace-ui). </div> + +<div class="deprecation breaking-change" data-milestone="16.0"> + +### Enforced validation of CI/CD parameter character lengths + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="announcement-milestone"> +While CI/CD [job names](https://docs.gitlab.com/ee/ci/jobs/index.html#job-name-limitations) have a strict 255 character limit, other CI/CD parameters do not yet have validations ensuring they also stay under the limit. -## Announced in 15.8 +In GitLab 16.0, validation will be added to strictly limit the following to 255 characters as well: -<div class="deprecation removal-160 breaking-change"> +- The `stage` keyword. +- The `ref`, which is the Git branch or tag name for the pipeline. +- The `description` and `target_url` parameter, used by external CI/CD integrations. -### Auto DevOps no longer provisions a PostgreSQL database by default +Users on self-managed instances should update their pipelines to ensure they do not use parameters that exceed 255 characters. Users on GitLab.com do not need to make any changes, as these are already limited in that database. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -Currently, Auto DevOps provisions an in-cluster PostgreSQL database by default. -In GitLab 16.0, databases will be provisioned only for users who opt in. This -change supports production deployments that require more robust database management. +### Environment search query requires at least three characters -If you want Auto DevOps to provision an in-cluster database, -set the `POSTGRES_ENABLED` CI/CD variable to `true`. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +From GitLab 16.0, when you search for environments with the API, you must use at least three characters. This change helps us ensure the scalability of the search operation. </div> -<div class="deprecation removal-170 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Auto DevOps support for Herokuish is deprecated +### External field in GraphQL ReleaseAssetLink type -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +In the [GraphQL API](https://docs.gitlab.com/ee/api/graphql/), the `external` field of [`ReleaseAssetLink` type](https://docs.gitlab.com/ee/api/graphql/reference/index.html#releaseassetlink) was used to indicate whether a [release link](https://docs.gitlab.com/ee/user/project/releases/release_fields.html#links) is internal or external to your GitLab instance. +As of GitLab 15.9, we treat all release links as external, and therefore, this field is deprecated in GitLab 15.9, and will be removed in GitLab 16.0. +To avoid any disruptions to your workflow, please stop using the `external` field because it will be removed and will not be replaced. -Auto DevOps support for Herokuish is deprecated in favor of [Cloud Native Buildpacks](https://docs.gitlab.com/ee/topics/autodevops/stages.html#auto-build-using-cloud-native-buildpacks). You should [migrate your builds from Herokuish to Cloud Native Buildpacks](https://docs.gitlab.com/ee/topics/autodevops/stages.html#moving-from-herokuish-to-cloud-native-buildpacks). From GitLab 14.0, Auto Build uses Cloud Native Buildpacks by default. +</div> -Because Cloud Native Buildpacks do not support automatic testing, the Auto Test feature of Auto DevOps is also deprecated. +<div class="deprecation breaking-change" data-milestone="16.0"> + +### External field in Releases and Release Links APIs +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-1510 breaking-change"> +In [Releases API](https://docs.gitlab.com/ee/api/releases/) and [Release Links API](https://docs.gitlab.com/ee/api/releases/links.html), the `external` field was used to indicate whether a [release link](https://docs.gitlab.com/ee/user/project/releases/release_fields.html#links) is internal or external to your GitLab instance. +As of GitLab 15.9, we treat all release links as external, and therefore, this field is deprecated in GitLab 15.9, and will be removed in GitLab 16.0. +To avoid any disruptions to your workflow, please stop using the `external` field because it will be removed and will not be replaced. -### Automatic backup upload using Openstack Swift and Rackspace APIs +</div> -End of Support: GitLab <span class="removal-milestone">15.10</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">15.10</span> <span class="removal-date"></span> +<div class="deprecation " data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Geo: Project repository redownload is deprecated -We are deprecating support for [uploading backups to remote storage](https://docs.gitlab.com/ee/raketasks/backup_gitlab.html#upload-backups-to-a-remote-cloud-storage) using Openstack Swift and Rackspace APIs. The support for these APIs depends on third-party libraries that are no longer actively maintained and have not been updated for Ruby 3. GitLab is switching over to Ruby 3 prior to EOL of Ruby 2 in order to stay up to date on security patches. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.11</span> +</div> -- If you're using OpenStack, you need to change you configuration to use the S3 API instead of Swift. -- If you're using Rackspace storage, you need to switch to a different provider or manually upload the backup file after the backup task is complete. +In secondary Geo sites, the button to "Redownload" a project repository is +deprecated. The redownload logic has inherent data consistency issues which +are difficult to resolve when encountered. The button will be removed in +GitLab 16.0. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Azure Storage Driver defaults to the correct root prefix +### GitLab self-monitoring project -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +GitLab self-monitoring gives administrators of self-hosted GitLab instances the tools to monitor the health of their instances. This feature is deprecated in GitLab 14.9, and is scheduled for removal in 16.0. -The Azure Storage Driver writes to `//` as the default root directory. This default root directory appears in some places within the Azure UI as `/<no-name>/`. We have maintained this legacy behavior to support older deployments using this storage driver. However, when moving to Azure from another storage driver, this behavior hides all your data until you configure the storage driver to build root paths without an extra leading slash by setting `trimlegacyrootprefix: true`. +</div> -The new default configuration for the storage driver will set `trimlegacyrootprefix: true`, and `/` will be the default root directory. You can add `trimlegacyrootprefix: false` to your current configuration to avoid any disruptions. +<div class="deprecation " data-milestone="16.0"> -This breaking change will happen in GitLab 16.0. +### GitLab.com importer +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> </div> -<div class="deprecation removal-160 breaking-change"> +The [GitLab.com importer](https://docs.gitlab.com/ee/user/project/import/gitlab_com.html) is deprecated in GitLab 15.8 and will be removed in GitLab 16.0. -### Conan project-level search endpoint returns project-specific results +The GitLab.com importer was introduced in 2015 for importing a project from GitLab.com to a self-managed GitLab instance through the UI. +This feature is available on self-managed instances only. [Migrating GitLab groups and projects by direct transfer](https://docs.gitlab.com/ee/user/group/import/#migrate-groups-by-direct-transfer-recommended) +supersedes the GitLab.com importer and provides a more cohesive importing functionality. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +See [migrated group items](https://docs.gitlab.com/ee/user/group/import/#migrated-group-items) and [migrated project items](https://docs.gitlab.com/ee/user/group/import/#migrated-project-items) for an overview. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -You can use the GitLab Conan repository with [project-level](https://docs.gitlab.com/ee/user/packages/conan_repository/#add-a-remote-for-your-project) or [instance-level](https://docs.gitlab.com/ee/user/packages/conan_repository/#add-a-remote-for-your-instance) endpoints. Each level supports the conan search command. However, the search endpoint for the project level is also returning packages from outside the target project. +<div class="deprecation breaking-change" data-milestone="16.0"> -This unintended functionality is deprecated in GitLab 15.8 and will be removed in GitLab 16.0. The search endpoint for the project level will only return packages from the target project. +### GraphQL API Runner status will not return `paused` +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The GitLab Runner GraphQL API endpoints will not return `paused` or `active` as a status in GitLab 16.0. +In a future v5 of the REST API, the endpoints for GitLab Runner will also not return `paused` or `active`. + +A runner's status will only relate to runner contact status, such as: +`online`, `offline`, or `not_connected`. Status `paused` or `active` will no longer appear. -### Configuring Redis config file paths using environment variables is deprecated +When checking if a runner is `paused`, API users are advised to check the boolean attribute +`paused` to be `true` instead. When checking if a runner is `active`, check if `paused` is `false`. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -You can no longer specify Redis configuration file locations -using the environment variables like `GITLAB_REDIS_CACHE_CONFIG_FILE` or -`GITLAB_REDIS_QUEUES_CONFIG_FILE`. Use the default -config file locations instead, for example `config/redis.cache.yml` or -`config/redis.queues.yml`. +### GraphQL API legacyMode argument for Runner status +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The `legacyMode` argument to the `status` field in `RunnerType` will be rendered non-functional in the 16.0 release +as part of the deprecations details in the [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). -### Container Registry pull-through cache +In GitLab 16.0 and later, the `status` field will act as if `legacyMode` is null. The `legacyMode` argument will +be present during the 16.x cycle to avoid breaking the API signature, and will be removed altogether in the +17.0 release. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -The Container Registry pull-through cache is deprecated in GitLab 15.8 and will be removed in GitLab 16.0. While the Container Registry pull-through cache functionality is useful, we have not made significant changes to this feature. You can use the upstream version of the container registry to achieve the same functionality. Removing the pull-through cache allows us also to remove the upstream client code without sacrificing functionality. +### GraphQL field `confidential` changed to `internal` on notes +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The `confidential` field for a `Note` will be deprecated and renamed to `internal`. -### Cookie authorization in the GitLab for Jira Cloud app +</div> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### GraphQL: The `DISABLED_WITH_OVERRIDE` value of the `SharedRunnersSetting` enum is deprecated. Use `DISABLED_AND_OVERRIDABLE` instead -Cookie authentication in the GitLab for Jira Cloud app is now deprecated in favor of OAuth authentication. -You must [set up OAuth authentication](https://docs.gitlab.com/ee/integration/jira/connect-app.html#set-up-oauth-authentication) -to continue to use the GitLab for Jira Cloud app. Without OAuth, you will not be able to manage linked namespaces. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +In GitLab 16.0, the `DISABLED_WITH_OVERRIDE` value of the `SharedRunnersSetting` GraphQL enum type will be replaced with the value, `DISABLED_AND_OVERRIDABLE`. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Dependency Scanning support for Java 13, 14, 15, and 16 +### HashiCorp Vault integration will no longer use CI_JOB_JWT by default -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +As part of our effort to improve the security of your CI workflows using JWT and OIDC, the native HashiCorp integration is also being updated in GitLab 16.0. Any projects that use the [`secrets:vault`](https://docs.gitlab.com/ee/ci/yaml/#secretsvault) keyword to retrieve secrets from Vault will need to be [configured to use ID tokens](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html#configure-automatic-id-token-authentication). -GitLab has deprecated Dependency Scanning support for Java versions 13, 14, 15, and 16 and plans to remove that support in the upcoming GitLab 16.0 release. This is consistent with [Oracle's support policy](https://www.oracle.com/java/technologies/java-se-support-roadmap.html) as Oracle Premier and Extended Support for these versions has ended. This also allows GitLab to focus Dependency Scanning Java support on LTS versions moving forward. +To be prepared for this change, you should do the following before GitLab 16.0: + +- [Disable the use of JSON web tokens](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html#enable-automatic-id-token-authentication) in the pipeline. +- Ensure the bound audience is prefixed with `https://`. +- Use the new [`id_tokens`](https://docs.gitlab.com/ee/ci/yaml/#id_tokens) keyword + and configure the `aud` claim. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Deployment API returns error when `updated_at` and `updated_at` are not used together +### Jira DVCS connector for Jira Cloud -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.1</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The [Jira DVCS connector](https://docs.gitlab.com/ee/integration/jira/dvcs/) for Jira Cloud has been deprecated and will be removed in GitLab 16.0. If you're using the Jira DVCS connector with Jira Cloud, migrate to the [GitLab for Jira Cloud app](https://docs.gitlab.com/ee/integration/jira/connect-app.html). -The Deployment API will now return an error when `updated_at` filtering and `updated_at` sorting are not used together. Some users were using filtering by `updated_at` to fetch "latest" deployment without using `updated_at` sorting, which may produce wrong results. You should instead use them together, or migrate to filtering by `finished_at` and sorting by `finished_at` which will give you "latest deployments" in a consistent way. +The Jira DVCS connector is also deprecated for Jira 8.13 and earlier. You can only use the Jira DVCS connector with Jira Server or Jira Data Center in Jira 8.14 and later. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Developer role providing the ability to import projects to a group +### KAS Metrics Port in GitLab Helm Chart -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- End of Support: GitLab <span class="milestone">16.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The `gitlab.kas.metrics.port` has been deprecated in favor of the new `gitlab.kas.observability.port` configuration field for the [GitLab Helm Chart](https://gitlab.com/gitlab-org/charts/gitlab/-/merge_requests/2839). +This port is used for much more than just metrics, which warranted this change to avoid confusion in configuration. -The ability for users with the Developer role for a group to import projects to that group is deprecated in GitLab -15.8 and will be removed in GitLab 16.0. From GitLab 16.0, only users with at least the Maintainer role for a group -will be able to import projects to that group. +</div> +<div class="deprecation breaking-change" data-milestone="16.0"> + +### Legacy Gitaly configuration method + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> +Gitaly configuration within Omnibus GitLab has been updated such that all Gitaly related configuration keys are in a single +configuration structure that matches the standard Gitaly configuration. As such, the previous configuration structure is deprecated. -### GitLab Helm chart values `gitlab.kas.privateApi.*` are deprecated +The single configuration structure is available from GitLab 15.10, though backwards compatibility is maintained. Once removed, Gitaly must be configured using the single +configuration structure. You should update the configuration of Gitaly at your earliest convenience. -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +The change improves consistency between Omnibus GitLab and source installs and enables us to provide better documentation and tooling for both. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +You should update to the new configuration structure as soon as possible using +[the upgrade instructions](https://docs.gitlab.com/ee/update/#gitaly-omnibus-gitlab-configuration-structure-change). -We introduced the `global.kas.tls.*` Helm values to facilitate TLS communication between KAS and your Helm chart components. -The old values `gitlab.kas.privateApi.tls.enabled` and `gitlab.kas.privateApi.tls.secretName` are deprecated and scheduled for removal in GitLab 17.0. +</div> -Because the new values provide a streamlined, comprehensive method to enable TLS for KAS, you should use `global.kas.tls.*` instead of `gitlab.kas.privateApi.tls.*`. The `gitlab.kas.privateApi.tls.*` For more information, see: +<div class="deprecation breaking-change" data-milestone="16.0"> -- The [merge request](https://gitlab.com/gitlab-org/charts/gitlab/-/merge_requests/2888) that introduces the `global.kas.tls.*` values. -- The [deprecated `gitlab.kas.privateApi.tls.*` documentation](https://docs.gitlab.com/charts/charts/gitlab/kas/index.html#enable-tls-communication-through-the-gitlabkasprivateapi-attributes-deprecated). -- The [new `global.kas.tls.*` documentation](https://docs.gitlab.com/charts/charts/globals.html#tls-settings-1). +### Legacy Praefect configuration method +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160"> +Previously, Praefect configuration keys were scattered throughout the configuration file. Now, these are in a single configuration structure that matches +Praefect configuration so the previous configuration method is deprecated. -### GitLab.com importer +The single configuration structure available from GitLab 15.9, though backwards compatibility is maintained. Once removed, Praefect must be configured using the single +configuration structure. You should update your Praefect configuration as soon as possible using +[the upgrade instructions](https://docs.gitlab.com/ee/update/#praefect-omnibus-gitlab-configuration-structure-change). -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +This change brings Praefect configuration in Omnibus GitLab in line with the configuration structure of Praefect. Previously, the hierarchies and configuration keys +didn't match. The change improves consistency between Omnibus GitLab and source installs and enables us to provide better documentation and tooling for both. -The [GitLab.com importer](https://docs.gitlab.com/ee/user/project/import/gitlab_com.html) is deprecated in GitLab 15.8 and will be removed in GitLab 16.0. +</div> -The GitLab.com importer was introduced in 2015 for importing a project from GitLab.com to a self-managed GitLab instance through the UI. -This feature is available on self-managed instances only. [Migrating GitLab groups and projects by direct transfer](https://docs.gitlab.com/ee/user/group/import/#migrate-groups-by-direct-transfer-recommended) -supersedes the GitLab.com importer and provides a more cohesive importing functionality. +<div class="deprecation breaking-change" data-milestone="16.0"> -See [migrated group items](https://docs.gitlab.com/ee/user/group/import/#migrated-group-items) and [migrated project items](https://docs.gitlab.com/ee/user/group/import/#migrated-project-items) for an overview. +### Legacy URLs replaced or removed +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +GitLab 16.0 removes legacy URLs from the GitLab application. -### GraphQL: The `DISABLED_WITH_OVERRIDE` value of the `SharedRunnersSetting` enum is deprecated. Use `DISABLED_AND_OVERRIDABLE` instead +When subgroups were introduced in GitLab 9.0, a `/-/` delimiter was added to URLs to signify the end of a group path. All GitLab URLs now use this delimiter for project, group, and instance level features. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +URLs that do not use the `/-/` delimiter are planned for removal in GitLab 16.0. For the full list of these URLs, along with their replacements, see [issue 28848](https://gitlab.com/gitlab-org/gitlab/-/issues/28848#release-notes). -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +Update any scripts or bookmarks that reference the legacy URLs. GitLab APIs are not affected by this change. -In GitLab 16.0, the `DISABLED_WITH_OVERRIDE` value of the `SharedRunnersSetting` GraphQL enum type will be replaced with the value, `DISABLED_AND_OVERRIDABLE`. +</div> + +<div class="deprecation breaking-change" data-milestone="16.0"> + +### License Compliance CI Template +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The GitLab [License Compliance](https://docs.gitlab.com/ee/user/compliance/license_compliance/) CI template is now deprecated and is scheduled for removal in the GitLab 16.0 release. Users who wish to continue using GitLab for License Compliance should remove the License Compliance template from their CI pipeline and add the [Dependency Scanning template](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuration). The Dependency Scanning template is now capable of gathering the required license information so it is no longer necessary to run a separate License Compliance job. The License Compliance CI template should not be removed prior to verifying that the `license_scanning_sbom_scanner` and `package_metadata_synchronization` flags are enabled for the instance and that the instance has been upgraded to a version that supports [the new method of license scanning](https://docs.gitlab.com/ee/user/compliance/license_scanning_of_cyclonedx_files/). -### Limit personal access token and deploy token's access with external authorization +| CI Pipeline Includes | GitLab <= 15.8 | 15.9 <= GitLab < 16.0 | GitLab >= 16.0 | +| ------------- | ------------- | ------------- | ------------- | +| Both DS and LS templates | License data from LS job is used | License data from LS job is used | License data from DS job is used | +| DS template is included but LS template is not | No license data | License data from DS job is used | License data from DS job is used | +| LS template is included but DS template is not | License data from LS job is used | License data from LS job is used | No license data | -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -With external authorization enabled, personal access tokens (PATs) and deploy tokens must no longer be able to access container or package registries. This defense-in-depth security measure will be deployed in 16.0. For users that use PATs and deploy tokens to access these registries, this measure breaks this use of these tokens. Disable external authorization to use tokens with container or package registries. +### License-Check and the Policies tab on the License Compliance page +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-159 breaking-change"> +The [License-Check feature](https://docs.gitlab.com/ee/user/compliance/license_check_rules.html) is now deprecated and is scheduled for removal in GitLab 16.0. Additionally, the Policies tab on the License Compliance page and all APIs related to the License-Check feature are deprecated and planned for removal in GitLab 16.0. Users who wish to continue to enforce approvals based on detected licenses are encouraged to create a new [License Approval policy](https://docs.gitlab.com/ee/user/compliance/license_approval_policies.html) instead. -### Live Preview no longer available in the Web IDE +</div> -Planned removal: GitLab <span class="removal-milestone">15.9</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Limit personal access token and deploy token's access with external authorization -The Live Preview feature of the Web IDE was intended to provide a client-side preview of static web applications. However, complex configuration steps and a narrow set of supported project types have limited its utility. With the introduction of the Web IDE Beta in GitLab 15.7, you can now connect to a full server-side runtime environment. With upcoming support for installing extensions in the Web IDE, we'll also support more advanced workflows than those available with Live Preview. As of GitLab 15.9, Live Preview is no longer available in the Web IDE. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +With external authorization enabled, personal access tokens (PATs) and deploy tokens must no longer be able to access container or package registries. This defense-in-depth security measure will be deployed in 16.0. For users that use PATs and deploy tokens to access these registries, this measure breaks this use of these tokens. Disable external authorization to use tokens with container or package registries. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> ### Maintainer role providing the ability to change Package settings using GraphQL API -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The ability for users with the Maintainer role to change the **Packages and registries** settings for a group using the GraphQL API is deprecated in GitLab 15.8 and will be removed in GitLab 16.0. These settings include: @@ -1060,657 +1249,666 @@ settings for the group using either the GitLab UI or GraphQL API. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Non-standard default Redis ports are deprecated +### Major bundled Helm Chart updates for the GitLab Helm Chart -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +To coincide with GitLab 16.0, the GitLab Helm Chart will release the 7.0 major version. The following major bundled chart updates will be included: -If GitLab starts without any Redis configuration file present, -GitLab assumes it can connect to three Redis servers at `localhost:6380`, -`localhost:6381` and `localhost:6382`. We are changing this behavior -so GitLab assumes there is one Redis server at `localhost:6379`. +- In GitLab 16.0, [PostgreSQL 12 support is being removed, and PostgreSQL 13 is becoming the new minimum](#postgresql-12-deprecated). + - Installs using production-ready external databases will need to complete their migration to a newer PostgreSQL version before upgrading. + - Installs using the [non-production bundled PostgreSQL 12 chart](https://docs.gitlab.com/charts/installation/tools.html#postgresql) will have the chart upgraded to the new version. For more information, [see issue 4118](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/4118) +- Installs using the [non-production bundled Redis chart](https://docs.gitlab.com/charts/installation/tools.html#redis) will have the chart upgraded to a newer version. For more information, [see issue 3375](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/3375) +- Installs using the [bundled cert-manager chart](https://docs.gitlab.com/charts/installation/tls.html#option-1-cert-manager-and-lets-encrypt) will have the chart upgraded to a newer version. For more information, [see issue 4313](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/4313) -Administrators who want to keep the three servers must configure -the Redis URLs by editing the `config/redis.cache.yml`,`config/redis.queues.yml` -and `config/redis.shared_state.yml` files. +The full GitLab Helm Chart 7.0 upgrade steps will be available in the [upgrade docs](https://docs.gitlab.com/charts/installation/upgrade.html). </div> -<div class="deprecation removal-160 breaking-change"> - -### Null value for `private_profile` attribute in User API is deprecated +<div class="deprecation breaking-change" data-milestone="16.0"> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +### Managed Licenses API -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -When creating and updating users through the API, `null` was a valid value for the `private_profile` attribute, which would internally be converted to the default value. Starting with 16.0, `null` will no longer be a valid value for this parameter, and the response will be a 400 if used. Now the only valid values are `true` and `false`. +The [Managed Licenses API](https://docs.gitlab.com/ee/api/managed_licenses.html) is now deprecated and is scheduled for removal in GitLab 16.0. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation " data-milestone="16.0"> -### Projects API field `operations_access_level` is deprecated +### Maximum number of active pipelines per project limit (`ci_active_pipelines`) -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.3</span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The [**Maximum number of active pipelines per project** limit](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#set-cicd-limits) was never enabled by default and will be removed in GitLab 16.0. This limit can also be configured in the Rails console under [`ci_active_pipelines`](https://docs.gitlab.com/ee/administration/instance_limits.html#number-of-pipelines-running-concurrently). Instead, use the other recommended rate limits that offer similar protection: -We are deprecating the `operations_access_level` field in the Projects API. This field has been replaced by fields to control specific features: `releases_access_level`, `environments_access_level`, `feature_flags_access_level`, `infrastructure_access_level`, and `monitor_access_level`. +- [**Pipelines rate limits**](https://docs.gitlab.com/ee/user/admin_area/settings/rate_limit_on_pipelines_creation.html). +- [**Total number of jobs in currently active pipelines**](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#set-cicd-limits). </div> -<div class="deprecation removal-160"> - -### Rake task for importing bare repositories - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -The [Rake task for importing bare repositories](https://docs.gitlab.com/ee/raketasks/import.html) `gitlab:import:repos` is deprecated in GitLab 15.8 and will be removed in GitLab 16.0. +### Monitor performance metrics through Prometheus -This Rake task imports a directory tree of repositories into a GitLab instance. These repositories must have been -managed by GitLab previously, because the Rake task relies on the specific directory structure or a specific custom Git setting in order to work (`gitlab.fullpath`). +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -Importing repositories using this Rake task has limitations. The Rake task: +By displaying data stored in a Prometheus instance, GitLab allows users to view performance metrics. GitLab also displays visualizations of these metrics in dashboards. The user can connect to a previously-configured external Prometheus instance, or set up Prometheus as a GitLab Managed App. +However, since certificate-based integration with Kubernetes clusters is deprecated in GitLab, the metrics functionality in GitLab that relies on Prometheus is also deprecated. This includes the metrics visualizations in dashboards. GitLab is working to develop a single user experience based on [Opstrace](https://about.gitlab.com/press/releases/2021-12-14-gitlab-acquires-opstrace-to-expand-its-devops-platform-with-open-source-observability-solution.html). An [issue exists](https://gitlab.com/groups/gitlab-org/-/epics/6976) for you to follow work on the Opstrace integration. -- Only knows about project and project wiki repositories and doesn't support repositories for designs, group wikis, or snippets. -- Permits you to import non-hashed storage projects even though these aren't supported. -- Relies on having Git config `gitlab.fullpath` set. [Epic 8953](https://gitlab.com/groups/gitlab-org/-/epics/8953) proposes removing support for this setting. +</div> -Alternatives to using the `gitlab:import:repos` Rake task include: +<div class="deprecation breaking-change" data-milestone="16.0"> -- Migrating projects using either [an export file](https://docs.gitlab.com/ee/user/project/settings/import_export.html) or - [direct transfer](https://docs.gitlab.com/ee/user/group/import/#migrate-groups-by-direct-transfer-recommended) migrate repositories as well. -- Importing a [repository by URL](https://docs.gitlab.com/ee/user/project/import/repo_by_url.html). -- Importing a [repositories from a non-GitLab source](https://docs.gitlab.com/ee/user/project/import/). +### Non-expiring access tokens +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.4</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +Access tokens that have no expiration date are valid indefinitely, which presents a security risk if the access token +is divulged. Because access tokens that have an exipiration date are better, from GitLab 15.3 we +[populate a default expiration date](https://gitlab.com/gitlab-org/gitlab/-/issues/348660). -### Support for third party registries +In GitLab 16.0, any [personal](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html), +[project](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html), or +[group](https://docs.gitlab.com/ee/user/group/settings/group_access_tokens.html) access token that does not have an +expiration date will automatically have an expiration date set at one year. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +We recommend giving your access tokens an expiration date in line with your company's security policies before the +default is applied: -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +- On GitLab.com during the 16.0 milestone. +- On GitLab self-managed instances when they are upgraded to 16.0. -Support for third-party container registries is deprecated in GitLab 15.8 and will be [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/376217) in GitLab 16.0. Supporting both GitLab's Container Registry and third-party container registries is challenging for maintenance, code quality, and backward compatibility. This hinders our ability to stay [efficient](https://about.gitlab.com/handbook/values/#efficiency). +</div> -Since we released the new [GitLab Container Registry](https://gitlab.com/groups/gitlab-org/-/epics/5523) version for GitLab.com, we've started to implement additional features that are not available in third-party container registries. These new features have allowed us to achieve significant performance improvements, such as [cleanup policies](https://gitlab.com/groups/gitlab-org/-/epics/8379). We are focusing on delivering [new features](https://gitlab.com/groups/gitlab-org/-/epics/5136), most of which will require functionalities only available on the GitLab Container Registry. This deprecation allows us to reduce fragmentation and user frustration in the long term by focusing on delivering a more robust integrated registry experience and feature set. +<div class="deprecation breaking-change" data-milestone="16.0"> -Moving forward, we'll continue to invest in developing and releasing new features that will only be available in the GitLab Container Registry. +### Non-standard default Redis ports are deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +If GitLab starts without any Redis configuration file present, +GitLab assumes it can connect to three Redis servers at `localhost:6380`, +`localhost:6381` and `localhost:6382`. We are changing this behavior +so GitLab assumes there is one Redis server at `localhost:6379`. -### Test system hook endpoint +Administrators who want to keep the three servers must configure +the Redis URLs by editing the `config/redis.cache.yml`,`config/redis.queues.yml` +and `config/redis.shared_state.yml` files. -End of Support: GitLab <span class="removal-milestone">16.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -The [test system hook](https://docs.gitlab.com/ee/api/system_hooks.html#test-system-hook) endpoint returns dummy data. -This endpoint is now deprecated and will be removed from the GitLab codebase. +### Null value for `private_profile` attribute in User API is deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +When creating and updating users through the API, `null` was a valid value for the `private_profile` attribute, which would internally be converted to the default value. Starting with 16.0, `null` will no longer be a valid value for this parameter, and the response will be a 400 if used. Now the only valid values are `true` and `false`. -### The API no longer returns revoked tokens for the agent for Kubernetes +</div> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Old versions of JSON web tokens are deprecated -Currently, GET requests to the [Cluster Agents API](https://docs.gitlab.com/ee/api/cluster_agents.html#list-tokens-for-an-agent) -endpoints can return revoked tokens. In GitLab 16.0, GET requests will not return revoked tokens. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -You should review your calls to these endpoints and ensure you do not use revoked tokens. +Now that we have released [ID tokens](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html) +with OIDC support, the old JSON web tokens are deprecated and will be removed. +Both the `CI_JOB_JWT` and `CI_JOB_JWT_V2` tokens, exposed to jobs as predefined variables, +will no longer be available in GitLab 16.0. -This change affects the following REST and GraphQL API endpoints: +To prepare for this change, you should: -- REST API: - - [List tokens](https://docs.gitlab.com/ee/api/cluster_agents.html#list-tokens-for-an-agent) - - [Get a single token](https://docs.gitlab.com/ee/api/cluster_agents.html#get-a-single-agent-token) -- GraphQL: - - [`ClusterAgent.tokens`](https://docs.gitlab.com/ee/api/graphql/reference/#clusteragenttokens) +- Configure your pipelines to use the fully configurable and more secure + [`id_token`](https://docs.gitlab.com/ee/ci/yaml/index.html#id_tokens) keyword instead. +- [Enable the **Limit JSON Web Token (JWT) access**](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html#enable-automatic-id-token-authentication) + setting, which prevents the old tokens from being exposed to any jobs. This setting + will be permanently enabled for all projects in GitLab 16.0. </div> -<div class="deprecation removal-170 breaking-change"> - -### The Visual Reviews tool is deprecated +<div class="deprecation breaking-change" data-milestone="16.0"> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -Due to limited customer usage and capabilities, the Visual Reviews feature for Review Apps is deprecated and will be removed. There is no planned replacement and users should stop using Visual Reviews before GitLab 17.0. +### Option to delete projects immediately is deprecated from deletion protection settings +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The group and project deletion protection setting in the Admin Area had an option to delete groups and projects immediately. Starting with 16.0, this option will no longer be available, and delayed group and project deletion will become the default behavior. -### The latest Terraform templates will overwrite current stable templates +The option will no longer appear as a group setting. Self-managed users will still have the option to define the deletion delay period, and SaaS users have a non-adjustable default retention period of 7 days. Users can still immediately delete the project from the project settings, and the group from the group settings. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +The option to delete groups and projects immediately by default was deprecated to prevent users from accidentally taking this action and permanently losing groups and projects. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -With every major GitLab version, we update the stable Terraform templates with the current latest templates. -This change affects the [quickstart](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml) -and the [base](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) templates. +<div class="deprecation breaking-change" data-milestone="16.0"> -Because the new templates ship with default rules, the update might break your Terraform pipelines. -For example, if your Terraform jobs are triggered as a downstream pipeline, the rules won't trigger your jobs -in GitLab 16.0. - -To accommodate the changes, you might need to adjust the [`rules`](https://docs.gitlab.com/ee/ci/yaml/#rules) in your -`.gitlab-ci.yml` file. +### Package pipelines in API payload is paginated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +A request to the API for `/api/v4/projects/:id/packages` returns a paginated result of packages. Each package lists all of its pipelines in this response. This is a performance concern, as it's possible for a package to have hundreds or thousands of associated pipelines. -### `environment_tier` parameter for DORA API +In milestone 16.0, we will remove the `pipelines` attribute from the API response. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -To avoid confusion and duplication, the `environment_tier` parameter is deprecated in favor of the `environment_tiers` parameter. The new `environment_tiers` parameter allows DORA APIs to return aggregated data for multiple tiers at the same time. The `environment_tier` parameter will be removed in GitLab 16.0. +### PipelineSecurityReportFinding name GraphQL field +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.1</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-1511"> +Previously, the [PipelineSecurityReportFinding GraphQL type was updated](https://gitlab.com/gitlab-org/gitlab/-/issues/335372) to include a new `title` field. This field is an alias for the current `name` field, making the less specific `name` field redundant. The `name` field will be removed from the PipelineSecurityReportFinding type in GitLab 16.0. -### openSUSE Leap 15.3 packages +</div> -Planned removal: GitLab <span class="removal-milestone">15.11</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -Distribution support and security updates for openSUSE Leap 15.3 [ended December 2022](https://en.opensuse.org/Lifetime#Discontinued_distributions). +### PipelineSecurityReportFinding projectFingerprint GraphQL field -Starting in GitLab 15.7 we started providing packages for openSUSE Leap 15.4, and will stop providing packages for openSUSE Leap 15.3 in the 15.11 milestone. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.1</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -- Switch from the openSUSE Leap 15.3 packages to the provided 15.4 packages. +The [`project_fingerprint`](https://gitlab.com/groups/gitlab-org/-/epics/2791) attribute of vulnerability findings is being deprecated in favor of a `uuid` attribute. By using UUIDv5 values to identify findings, we can easily associate any related entity with a finding. The `project_fingerprint` attribute is no longer being used to track findings, and will be removed in GitLab 16.0. </div> -</div> -<div class="announcement-milestone"> +<div class="deprecation breaking-change" data-milestone="16.0"> -## Announced in 15.7 - -<div class="deprecation removal-160 breaking-change"> +### PostgreSQL 12 deprecated -### DAST API scans using DAST template is deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +Support for PostgreSQL 12 is scheduled for removal in GitLab 16.0. +In GitLab 16.0, PostgreSQL 13 becomes the minimum required PostgreSQL version. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +PostgreSQL 12 will be supported for the full GitLab 15 release cycle. +PostgreSQL 13 will also be supported for instances that want to upgrade prior to GitLab 16.0. -With the move to the new DAST API analyzer and the `DAST-API.gitlab-ci.yml` template for DAST API scans, we will be removing the ability to scan APIs with the DAST analyzer. Use of the `DAST.gitlab-ci.yml` or `DAST-latest.gitlab-ci.yml` templates for API scans is deprecated as of GitLab 15.7 and will no longer work in GitLab 16.0. Please use `DAST-API.gitlab-ci.yml` template and refer to the [DAST API analyzer](https://docs.gitlab.com/ee/user/application_security/dast_api/#configure-dast-api-with-an-openapi-specification) documentation for configuration details. +Support for PostgreSQL 13 was added to Geo in GitLab 15.2. </div> -<div class="deprecation removal-160 breaking-change"> - -### DAST API variables +<div class="deprecation breaking-change" data-milestone="16.0"> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +### Projects API field `operations_access_level` is deprecated -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -With the switch to the new DAST API analyzer in GitLab 15.6, two legacy DAST API variables are being deprecated. The variables `DAST_API_HOST_OVERRIDE` and `DAST_API_SPECIFICATION` will no longer be used for DAST API scans. +We are deprecating the `operations_access_level` field in the Projects API. This field has been replaced by fields to control specific features: `releases_access_level`, `environments_access_level`, `feature_flags_access_level`, `infrastructure_access_level`, and `monitor_access_level`. -`DAST_API_HOST_OVERRIDE` has been deprecated in favor of using the `DAST_API_TARGET_URL` to automatically override the host in the OpenAPI specification. +</div> -`DAST_API_SPECIFICATION` has been deprecated in favor of `DAST_API_OPENAPI`. To continue using an OpenAPI specification to guide the test, users must replace the `DAST_API_SPECIFICATION` variable with the `DAST_API_OPENAPI` variable. The value can remain the same, but the variable name must be replaced. +<div class="deprecation " data-milestone="16.0"> -These two variables will be removed in GitLab 16.0. +### Rake task for importing bare repositories +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> </div> -<div class="deprecation removal-160 breaking-change"> +The [Rake task for importing bare repositories](https://docs.gitlab.com/ee/raketasks/import.html) `gitlab:import:repos` is deprecated in GitLab 15.8 and will be removed in GitLab 16.0. -### DAST ZAP advanced configuration variables deprecation +This Rake task imports a directory tree of repositories into a GitLab instance. These repositories must have been +managed by GitLab previously, because the Rake task relies on the specific directory structure or a specific custom Git setting in order to work (`gitlab.fullpath`). -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +Importing repositories using this Rake task has limitations. The Rake task: -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +- Only knows about project and project wiki repositories and doesn't support repositories for designs, group wikis, or snippets. +- Permits you to import non-hashed storage projects even though these aren't supported. +- Relies on having Git config `gitlab.fullpath` set. [Epic 8953](https://gitlab.com/groups/gitlab-org/-/epics/8953) proposes removing support for this setting. -With the new browser-based DAST analyzer GA in GitLab 15.7, we are working towards making it the default DAST analyzer at some point in the future. In preparation for this, the following legacy DAST variables are being deprecated and scheduled for removal in GitLab 16.0: `DAST_ZAP_CLI_OPTIONS` and `DAST_ZAP_LOG_CONFIGURATION`. These variables allowed for advanced configuration of the legacy DAST analyzer, which was based on OWASP ZAP. The new browser-based analyzer will not include the same functionality, as these were specific to how ZAP worked. +Alternatives to using the `gitlab:import:repos` Rake task include: -These three variables will be removed in GitLab 16.0. +- Migrating projects using either [an export file](https://docs.gitlab.com/ee/user/project/settings/import_export.html) or + [direct transfer](https://docs.gitlab.com/ee/user/group/import/#migrate-groups-by-direct-transfer-recommended) migrate repositories as well. +- Importing a [repository by URL](https://docs.gitlab.com/ee/user/project/import/repo_by_url.html). +- Importing a [repositories from a non-GitLab source](https://docs.gitlab.com/ee/user/project/import/). </div> -<div class="deprecation removal-160 breaking-change"> - -### DAST report variables deprecation - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="16.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Redis 5 deprecated -With the new browser-based DAST analyzer GA in GitLab 15.7, we are working towards making it the default DAST analyzer at some point in the future. In preparation for this, the following legacy DAST variables are being deprecated and scheduled for removal in GitLab 16.0: `DAST_HTML_REPORT`, `DAST_XML_REPORT`, and `DAST_MARKDOWN_REPORT`. These reports relied on the legacy DAST analyzer and we do not plan to implement them in the new browser-based analyzer. As of GitLab 16.0, these report artifacts will no longer be generated. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.3</span> +- End of Support: GitLab <span class="milestone">15.6</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -These three variables will be removed in GitLab 16.0. +With GitLab 13.9, in the Omnibus GitLab package and GitLab Helm chart 4.9, the Redis version [was updated to Redis 6](https://about.gitlab.com/releases/2021/02/22/gitlab-13-9-released/#omnibus-improvements). +Redis 5 has reached the end of life in April 2022 and will no longer be supported as of GitLab 15.6. +If you are using your own Redis 5.0 instance, you should upgrade it to Redis 6.0 or higher before upgrading to GitLab 16.0 or higher. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### KAS Metrics Port in GitLab Helm Chart +### Remove `job_age` parameter from `POST /jobs/request` Runner endpoint -End of Support: GitLab <span class="removal-milestone">16.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.2</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The `job_age` parameter, returned from the `POST /jobs/request` API endpoint used in communication with GitLab Runner, was never used by any GitLab or Runner feature. This parameter will be removed in GitLab 16.0. -The `gitlab.kas.metrics.port` has been deprecated in favor of the new `gitlab.kas.observability.port` configuration field for the [GitLab Helm Chart](https://gitlab.com/gitlab-org/charts/gitlab/-/merge_requests/2839). -This port is used for much more than just metrics, which warranted this change to avoid confusion in configuration. +This could be a breaking change for anyone that developed their own runner that relies on this parameter being returned by the endpoint. This is not a breaking change for anyone using an officially released version of GitLab Runner, including public shared runners on GitLab.com. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Shimo integration +### Required Pipeline Configuration is deprecated -End of Support: GitLab <span class="removal-milestone">16.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +Required Pipeline Configuration will be removed in the 16.0 release. This impacts self-managed users on the Ultimate license. -The [Shimo Workspace integration](https://docs.gitlab.com/ee/user/project/integrations/shimo.html) has been deprecated -and will be moved to the JiHu GitLab codebase. +We recommend replacing this with an alternative [compliance solution](https://docs.gitlab.com/ee/user/group/compliance_frameworks.html#compliance-pipelines) +that is available now. We recommend this alternative solution because it provides greater flexibility, allowing required pipelines to be assigned to specific compliance framework labels. </div> -<div class="deprecation removal-170 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Support for REST API endpoints that reset runner registration tokens - -End of Support: GitLab <span class="removal-milestone">17.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +### SAST analyzer coverage changing in GitLab 16.0 -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -The support for runner registration tokens is deprecated. As a consequence, the REST API endpoints to reset a registration token are also deprecated and will -be removed in GitLab 17.0. -The deprecated endpoints are: +GitLab SAST uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/sast/analyzers/) to scan code for vulnerabilities. -- `POST /runners/reset_registration_token` -- `POST /projects/:id/runners/reset_registration_token` -- `POST /groups/:id/runners/reset_registration_token` +We're reducing the number of supported analyzers used by default in GitLab SAST. +This is part of our long-term strategy to deliver a faster, more consistent user experience across different programming languages. -We plan to implement a new method to bind runners to a GitLab instance -as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). -The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). -This new architecture introduces a new method for registering runners and will eliminate the legacy -[runner registration token](https://docs.gitlab.com/ee/security/token_overview.html#runner-registration-tokens). -From GitLab 17.0 and later, the runner registration methods implemented by the new GitLab Runner token architecture will be the only supported methods. +Starting in GitLab 16.0, the GitLab SAST CI/CD template will no longer use the following analyzers, and they will enter End of Support status: -</div> +- [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) (.NET) +- [PHPCS Security Audit](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) (PHP) -<div class="deprecation removal-160 breaking-change"> +We'll remove these analyzers from the [SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) and replace them with GitLab-supported detection rules and the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). +Effective immediately, these analyzers will receive only security updates; other routine improvements or updates are not guaranteed. +After these analyzers reach End of Support, no further updates will be provided. +However, we won't delete container images previously published for these analyzers or remove the ability to run them by using a custom CI/CD pipeline job. -### Support for periods (`.`) in Terraform state names might break existing states +We will also remove Scala from the scope of the [SpotBugs-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) and replace it with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). +This change will make it simpler to scan Scala code; compilation will no longer be required. +This change will be reflected in the automatic language detection portion of the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml). +Note that the SpotBugs-based analyzer will continue to cover Groovy and Kotlin. -End of Support: GitLab <span class="removal-milestone">16.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +If you've already dismissed a vulnerability finding from one of the deprecated analyzers, the replacement attempts to respect your previous dismissal. The system behavior depends on: -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +- whether you've excluded the Semgrep-based analyzer from running in the past. +- which analyzer first discovered the vulnerabilities shown in the project's Vulnerability Report. -Previously, Terraform state names containing periods were not supported. However, you could still use state names with periods via a workaround. +See [Vulnerability translation documentation](https://docs.gitlab.com/ee/user/application_security/sast/analyzers.html#vulnerability-translation) for further details. -GitLab 15.7 [adds full support](https://docs.gitlab.com/ee/user/infrastructure/iac/troubleshooting.html#state-not-found-if-the-state-name-contains-a-period) for state names that contain periods. If you used a workaround to handle these state names, your jobs might fail, or it might look like you've run Terraform for the first time. +If you applied customizations to any of the affected analyzers or if you currently disable the Semgrep analyzer in your pipelines, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/390416#breaking-change). -To resolve the issue: +</div> - 1. Change any references to the state file by excluding the period and any characters that follow. - - For example, if your state name is `state.name`, change all references to `state`. - 1. Run your Terraform commands. +<div class="deprecation breaking-change" data-milestone="16.0"> -To use the full state name, including the period, [migrate to the full state file](https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html#migrate-to-a-gitlab-managed-terraform-state). +### Secure analyzers major version update +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### The Phabricator task importer is deprecated +The Secure stage will be bumping the major versions of its analyzers in tandem with the GitLab 16.0 release. This bump will enable a clear delineation for analyzers, between: -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +- Those released prior to May 22, 2023 +- Those released after May 22, 2023 -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +If you are not using the default included templates, or have pinned your analyzer versions you will need to update your CI/CD job definition to either remove the pinned version or to update the latest major version. +Users of GitLab 13.0-15.10 will continue to experience analyzer updates as normal until the release of GitLab 16.0, following which all newly fixed bugs and released features will be released only in the new major version of the analyzers. We do not backport bugs and features to deprecated versions as per our [maintenance policy](https://docs.gitlab.com/ee/policy/maintenance.html). As required, security patches will be backported within the latest 3 minor releases. +Specifically, the following are being deprecated and will no longer be updated after 16.0 GitLab release: -The [Phabricator task importer](https://docs.gitlab.com/ee/user/project/import/phabricator.html) is being deprecated. Phabricator itself as a project is no longer actively maintained since June 1, 2021. We haven't observed imports using this tool. There has been no activity on the open related issues on GitLab. +- API Fuzzing: version 2 +- Container Scanning: version 5 +- Coverage-guided fuzz testing: version 3 +- Dependency Scanning: version 3 +- Dynamic Application Security Testing (DAST): version 3 +- DAST API: version 2 +- IaC Scanning: version 3 +- License Scanning: version 4 +- Secret Detection: version 4 +- Static Application Security Testing (SAST): version 3 of [all analyzers](https://docs.gitlab.com/ee/user/application_security/sast/#supported-languages-and-frameworks) + - `brakeman`: version 3 + - `flawfinder`: version 3 + - `kubesec`: version 3 + - `mobsf`: version 3 + - `nodejs-scan`: version 3 + - `phpcs-security-audit`: version 3 + - `pmd-apex`: version 3 + - `security-code-scan`: version 3 + - `semgrep`: version 3 + - `sobelow`: version 3 + - `spotbugs`: version 3 </div> -<div class="deprecation removal-170 breaking-change"> - -### The `gitlab-runner exec` command is deprecated - -End of Support: GitLab <span class="removal-milestone">17.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -The [`gitlab-runner exec`](https://docs.gitlab.com/runner/commands/#gitlab-runner-exec) command is deprecated and will be fully removed from GitLab Runner in 16.0. The `gitlab-runner exec` feature was initially developed to provide the ability to validate a GitLab CI pipeline on a local system without needing to commit the updates to a GitLab instance. However, with the continued evolution of GitLab CI, replicating all GitLab CI features into `gitlab-runner exec` was no longer viable. Pipeline syntax and validation [simulation](https://docs.gitlab.com/ee/ci/pipeline_editor/#simulate-a-cicd-pipeline) are available in the GitLab pipeline editor. +### Secure scanning CI/CD templates will use new job `rules` +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### ZenTao integration +GitLab-managed CI/CD templates for security scanning will be updated in the GitLab 16.0 release. +The updates will include improvements already released in the Latest versions of the CI/CD templates. +We released these changes in the Latest template versions because they have the potential to disrupt customized CI/CD pipeline configurations. -End of Support: GitLab <span class="removal-milestone">16.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +In all updated templates, we're: -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +- Adding support for running scans in merge request (MR) pipelines. +- Updating the definition of variables like `SAST_DISABLED` and `DEPENDENCY_SCANNING_DISABLED` to disable scanning only if the value is `"true"`. Previously, even if the value were `"false"`, scanning would be disabled. -The [ZenTao product integration](https://docs.gitlab.com/ee/user/project/integrations/zentao.html) has been deprecated -and will be moved to the JiHu GitLab codebase. +The following templates will be updated: -</div> +- API Fuzzing: [`API-Fuzzing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) +- Container Scanning: [`Container-Scanning.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Container-Scanning.gitlab-ci.yml) +- Coverage-Guided Fuzzing: [`Coverage-Fuzzing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Coverage-Fuzzing.gitlab-ci.yml) +- DAST: [`DAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml) +- DAST API: [`DAST-API.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) +- Dependency Scanning: [`Dependency-Scanning.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml) +- IaC Scanning: [`SAST-IaC.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml) +- SAST: [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) +- Secret Detection: [`Secret-Detection.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Secret-Detction.gitlab-ci.yml) -<div class="deprecation removal-160 breaking-change"> +We recommend that you test your pipelines before the 16.0 release if you use one of the templates listed above and you do any of the following: -### `POST ci/lint` API endpoint deprecated + 1. You override `rules` for your security scanning jobs. + 1. You use the `_DISABLED` variables but set a value other than `"true"`. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -The `POST ci/lint` API endpoint is deprecated in 15.7, and will be removed in 16.0. This endpoint does not validate the full range of CI/CD configuration options. Instead, use [`POST /projects/:id/ci/lint`](https://docs.gitlab.com/ee/api/lint.html#validate-a-ci-yaml-configuration-with-a-namespace), which properly validates CI/CD configuration. +### Security report schemas version 14.x.x -</div> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.3</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="announcement-milestone"> +Version 14.x.x [security report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas) are deprecated. -## Announced in 15.6 +In GitLab 15.8 and later, [security report scanner integrations](https://docs.gitlab.com/ee/development/integrations/secure.html) that use schema version 14.x.x will display a deprecation warning in the pipeline's **Security** tab. -<div class="deprecation removal-160 breaking-change"> +In GitLab 16.0 and later, the feature will be removed. Security reports that use schema version 14.x.x will cause an error in the pipeline's **Security** tab. -### Configuration fields in GitLab Runner Helm Chart +For more information, refer to [security report validation](https://docs.gitlab.com/ee/user/application_security/#security-report-validation). -End of Support: GitLab <span class="removal-milestone">16.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -From GitLab 13.6, users can [specify any runner configuration in the GitLab Runner Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html). When we implemented this feature, we deprecated values in the GitLab Helm Chart configuration that were specific to GitLab Runner. These fields are deprecated and we plan to remove them in v1.0 of the GitLab Runner Helm chart. +### Shimo integration +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- End of Support: GitLab <span class="milestone">16.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> - -### GitLab Runner registration token in Runner Operator +The [Shimo Workspace integration](https://docs.gitlab.com/ee/user/project/integrations/shimo.html) has been deprecated +and will be moved to the JiHu GitLab codebase. -End of Support: GitLab <span class="removal-milestone">17.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -The [`runner-registration-token`](https://docs.gitlab.com/runner/install/operator.html#install-the-kubernetes-operator) parameter that uses the OpenShift and k8s Vanilla Operator to install a runner on Kubernetes is deprecated. -We plan to implement a new method to bind runners to a GitLab instance -as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). -The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). +### Starboard directive in the config for the GitLab Agent for Kubernetes +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.4</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> - -### Registration tokens and server-side runner arguments in `POST /api/v4/runners` endpoint - -End of Support: GitLab <span class="removal-milestone">17.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +GitLab's operational container scanning capabilities no longer require starboard to be installed. Consequently, use of the `starboard:` directive in the configuration file for the GitLab Agent for Kubernetes is now deprecated and is scheduled for removal in GitLab 16.0. Update your configuration file to use the `container_scanning:` directive. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -The support for registration tokens and certain runner configuration arguments in the `POST` method operation on the `/api/v4/runners` endpoint is deprecated. -This endpoint [registers](https://docs.gitlab.com/ee/api/runners.html#register-a-new-runner) a runner -with a GitLab instance at the instance, group, or project level through the API. We plan to remove the support for -registration tokens and certain configuration arguments in this endpoint in GitLab 17.0. +<div class="deprecation breaking-change" data-milestone="16.0"> -We plan to implement a new method to bind runners to a GitLab instance -as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). -The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). -This new architecture introduces a new method for registering runners and will eliminate the legacy -[runner registration token](https://docs.gitlab.com/ee/security/token_overview.html#runner-registration-tokens). -From GitLab 17.0 and later, the runner registration methods implemented by the new GitLab Runner token architecture will be the only supported methods. +### Support for Praefect custom metrics endpoint configuration +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-170 breaking-change"> - -### Registration tokens and server-side runner arguments in `gitlab-runner register` command - -End of Support: GitLab <span class="removal-milestone">17.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +Support for using the `prometheus_exclude_database_from_default_metrics` configuration value is deprecated in GitLab +15.9 and will be removed in GitLab 16.0. We are removing this configuration value because using it is non-performant. +This change means the following metrics will become unavailable on `/metrics`: -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +- `gitaly_praefect_unavailable_repositories`. +- `gitaly_praefect_verification_queue_depth`. +- `gitaly_praefect_replication_queue_depth`. -The support for registration tokens and certain configuration arguments in the command to [register](https://docs.gitlab.com/runner/register/) a runner, `gitlab-runner register` is deprecated. -We plan to implement a new method to bind runners to a GitLab instance -as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). -The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). -The new method will involve creating the runner in the GitLab UI and passing the -[runner authentication token](https://docs.gitlab.com/ee/security/token_overview.html#runner-authentication-tokens-also-called-runner-tokens) -to the `gitlab-runner register` command. +This may require updating your metrics collection targets to also scrape `/db_metrics`. </div> -<div class="deprecation removal-170 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### `runnerRegistrationToken` parameter for GitLab Runner Helm Chart +### Support for periods (`.`) in Terraform state names might break existing states -End of Support: GitLab <span class="removal-milestone">17.0</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- End of Support: GitLab <span class="milestone">16.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +Previously, Terraform state names containing periods were not supported. However, you could still use state names with periods via a workaround. -The [`runnerRegistrationToken`](https://docs.gitlab.com/runner/install/kubernetes.html#required-configuration) parameter to use the GitLab Helm Chart to install a runner on Kubernetes is deprecated. +GitLab 15.7 [adds full support](https://docs.gitlab.com/ee/user/infrastructure/iac/troubleshooting.html#state-not-found-if-the-state-name-contains-a-period) for state names that contain periods. If you used a workaround to handle these state names, your jobs might fail, or it might look like you've run Terraform for the first time. -We plan to implement a new method to bind runners to a GitLab instance leveraging `runnerToken` -as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/). -The work is planned in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7633). +To resolve the issue: -From GitLab 17.0 and later, the methods to register runners introduced by the new GitLab Runner token architecture will be the only supported methods. + 1. Change any references to the state file by excluding the period and any characters that follow. + - For example, if your state name is `state.name`, change all references to `state`. + 1. Run your Terraform commands. + +To use the full state name, including the period, [migrate to the full state file](https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html#migrate-to-a-gitlab-managed-terraform-state). -</div> </div> -<div class="announcement-milestone"> +<div class="deprecation breaking-change" data-milestone="16.0"> -## Announced in 15.5 +### Support for third party registries -<div class="deprecation removal-157 breaking-change"> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -### File Type variable expansion in `.gitlab-ci.yml` +Support for third-party container registries is deprecated in GitLab 15.8 and will be [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/376217) in GitLab 16.0. Supporting both GitLab's Container Registry and third-party container registries is challenging for maintenance, code quality, and backward compatibility. This hinders our ability to stay [efficient](https://about.gitlab.com/handbook/values/#efficiency). + +Since we released the new [GitLab Container Registry](https://gitlab.com/groups/gitlab-org/-/epics/5523) version for GitLab.com, we've started to implement additional features that are not available in third-party container registries. These new features have allowed us to achieve significant performance improvements, such as [cleanup policies](https://gitlab.com/groups/gitlab-org/-/epics/8379). We are focusing on delivering [new features](https://gitlab.com/groups/gitlab-org/-/epics/5136), most of which will require functionalities only available on the GitLab Container Registry. This deprecation allows us to reduce fragmentation and user frustration in the long term by focusing on delivering a more robust integrated registry experience and feature set. -Planned removal: GitLab <span class="removal-milestone">15.7</span> <span class="removal-date"></span> +Moving forward, we'll continue to invest in developing and releasing new features that will only be available in the GitLab Container Registry. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -Previously, variables that referenced or applied alias file variables expanded the value of the `File` type variable. For example, the file contents. This behavior was incorrect because it did not comply with typical shell variable expansion rules. To leak secrets or sensitive information stored in `File` type variables, a user could run an $echo command with the variable as an input parameter. +<div class="deprecation breaking-change" data-milestone="16.0"> -This breaking change fixes this issue but could disrupt user workflows that work around the behavior. With this change, job variable expansions that reference or apply alias file variables, expand to the file name or path of the `File` type variable, instead of its value, such as the file contents. +### Test system hook endpoint +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- End of Support: GitLab <span class="milestone">16.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### GraphQL field `confidential` changed to `internal` on notes +The [test system hook](https://docs.gitlab.com/ee/api/system_hooks.html#test-system-hook) endpoint returns dummy data. +This endpoint is now deprecated and will be removed from the GitLab codebase. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -The `confidential` field for a `Note` will be deprecated and renamed to `internal`. +### The API no longer returns revoked tokens for the agent for Kubernetes +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### vulnerabilityFindingDismiss GraphQL mutation +Currently, GET requests to the [Cluster Agents API](https://docs.gitlab.com/ee/api/cluster_agents.html#list-tokens-for-an-agent) +endpoints can return revoked tokens. In GitLab 16.0, GET requests will not return revoked tokens. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +You should review your calls to these endpoints and ensure you do not use revoked tokens. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +This change affects the following REST and GraphQL API endpoints: -The `VulnerabilityFindingDismiss` GraphQL mutation is being deprecated and will be removed in GitLab 16.0. This mutation was not used often as the Vulnerability Finding ID was not available to users (this field was [deprecated in 15.3](https://docs.gitlab.com/ee/update/deprecations.html#use-of-id-field-in-vulnerabilityfindingdismiss-mutation)). Users should instead use `VulnerabilityDismiss` to dismiss vulnerabilities in the Vulnerability Report or `SecurityFindingDismiss` for security findings in the CI Pipeline Security tab. +- REST API: + - [List tokens](https://docs.gitlab.com/ee/api/cluster_agents.html#list-tokens-for-an-agent) + - [Get a single token](https://docs.gitlab.com/ee/api/cluster_agents.html#get-a-single-agent-token) +- GraphQL: + - [`ClusterAgent.tokens`](https://docs.gitlab.com/ee/api/graphql/reference/#clusteragenttokens) </div> -</div> -<div class="announcement-milestone"> +<div class="deprecation breaking-change" data-milestone="16.0"> -## Announced in 15.4 +### The Phabricator task importer is deprecated -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -### Container Scanning variables that reference Docker +The [Phabricator task importer](https://docs.gitlab.com/ee/user/project/import/phabricator.html) is being deprecated. Phabricator itself as a project is no longer actively maintained since June 1, 2021. We haven't observed imports using this tool. There has been no activity on the open related issues on GitLab. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -All Container Scanning variables that are prefixed by `DOCKER_` in variable name are deprecated. This includes the `DOCKER_IMAGE`, `DOCKER_PASSWORD`, `DOCKER_USER`, and `DOCKERFILE_PATH` variables. Support for these variables will be removed in the GitLab 16.0 release. Use the [new variable names](https://docs.gitlab.com/ee/user/application_security/container_scanning/#available-cicd-variables) `CS_IMAGE`, `CS_REGISTRY_PASSWORD`, `CS_REGISTRY_USER`, and `CS_DOCKERFILE_PATH` in place of the deprecated names. +### The latest Terraform templates will overwrite current stable templates +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +With every major GitLab version, we update the stable Terraform templates with the current latest templates. +This change affects the [quickstart](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml) +and the [base](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) templates. -### Non-expiring access tokens +Because the new templates ship with default rules, the update might break your Terraform pipelines. +For example, if your Terraform jobs are triggered as a downstream pipeline, the rules won't trigger your jobs +in GitLab 16.0. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +To accommodate the changes, you might need to adjust the [`rules`](https://docs.gitlab.com/ee/ci/yaml/#rules) in your +`.gitlab-ci.yml` file. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -Access tokens that have no expiration date are valid indefinitely, which presents a security risk if the access token -is divulged. Because access tokens that have an exipiration date are better, from GitLab 15.3 we -[populate a default expiration date](https://gitlab.com/gitlab-org/gitlab/-/issues/348660). +<div class="deprecation breaking-change" data-milestone="16.0"> -In GitLab 16.0, any [personal](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html), -[project](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html), or -[group](https://docs.gitlab.com/ee/user/group/settings/group_access_tokens.html) access token that does not have an -expiration date will automatically have an expiration date set at one year. +### Toggle behavior of `/draft` quick action in merge requests -We recommend giving your access tokens an expiration date in line with your company's security policies before the -default is applied: +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.4</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -- On GitLab.com during the 16.0 milestone. -- On GitLab self-managed instances when they are upgraded to 16.0. +In order to make the behavior of toggling the draft status of a merge request more clear via a quick action, we're deprecating and removing the toggle behavior of the `/draft` quick action. Beginning with the 16.0 release of GitLab, `/draft` will only set a merge request to Draft and a new `/ready` quick action will be used to remove the draft status. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Starboard directive in the config for the GitLab Agent for Kubernetes - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +### Toggle notes confidentiality on APIs -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -GitLab's operational container scanning capabilities no longer require starboard to be installed. Consequently, use of the `starboard:` directive in the configuration file for the GitLab Agent for Kubernetes is now deprecated and is scheduled for removal in GitLab 16.0. Update your configuration file to use the `container_scanning:` directive. +Toggling notes confidentiality with REST and GraphQL APIs is being deprecated. Updating notes confidential attribute is no longer supported by any means. We are changing this to simplify the experience and prevent private information from being unintentionally exposed. </div> -<div class="deprecation removal-160 breaking-change"> - -### Toggle behavior of `/draft` quick action in merge requests +<div class="deprecation breaking-change" data-milestone="16.0"> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +### Use of `id` field in vulnerabilityFindingDismiss mutation -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.3</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -In order to make the behavior of toggling the draft status of a merge request more clear via a quick action, we're deprecating and removing the toggle behavior of the `/draft` quick action. Beginning with the 16.0 release of GitLab, `/draft` will only set a merge request to Draft and a new `/ready` quick action will be used to remove the draft status. +You can use the vulnerabilityFindingDismiss GraphQL mutation to set the status of a vulnerability finding to `Dismissed`. Previously, this mutation used the `id` field to identify findings uniquely. However, this did not work for dismissing findings from the pipeline security tab. Therefore, using the `id` field as an identifier has been dropped in favor of the `uuid` field. Using the 'uuid' field as an identifier allows you to dismiss the finding from the pipeline security tab. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> ### Vulnerability confidence field -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.4</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> In GitLab 15.3, [security report schemas below version 15 were deprecated](https://docs.gitlab.com/ee/update/deprecations.html#security-report-schemas-version-14xx). The `confidence` attribute on vulnerability findings exists only in schema versions before `15-0-0`, and therefore is effectively deprecated since GitLab 15.4 supports schema version `15-0-0`. To maintain consistency @@ -1718,528 +1916,538 @@ between the reports and our public APIs, the `confidence` attribute on any vulne removed in 16.0. </div> -</div> -<div class="announcement-milestone"> +<div class="deprecation breaking-change" data-milestone="16.0"> -## Announced in 15.3 +### Work items path with global ID at the end of the path is deprecated -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -### Atlassian Crowd OmniAuth provider +Usage of global IDs in work item URLs is deprecated. In the future, only internal IDs (IID) will be supported. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +Because GitLab supports multiple work item types, a path such as `https://gitlab.com/gitlab-org/gitlab/-/work_items/<global_id>` can display, for example, a [task](https://docs.gitlab.com/ee/user/tasks.html) or an [OKR](https://docs.gitlab.com/ee/user/okrs.html). -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +In GitLab 15.10 we added support for using internal IDs (IID) in that path by appending a query param at +the end (`iid_path`) in the following format: `https://gitlab.com/gitlab-org/gitlab/-/work_items/<iid>?iid_path=true`. -The `omniauth_crowd` gem that provides GitLab with the Atlassian Crowd OmniAuth provider will be removed in our -next major release, GitLab 16.0. This gem sees very little use and its -[lack of compatibility](https://github.com/robdimarco/omniauth_crowd/issues/37) with OmniAuth 2.0 is -[blocking our upgrade](https://gitlab.com/gitlab-org/gitlab/-/issues/30073). +In GitLab 16.0 we will remove the ability to use a global ID in the work items path. The number at the end of the path will be considered an internal ID (IID) without the need of adding a query param at the end. Only the following format will be supported: `https://gitlab.com/gitlab-org/gitlab/-/work_items/<iid>`. </div> -<div class="deprecation removal-154"> - -### Bundled Grafana deprecated - -Planned removal: GitLab <span class="removal-milestone">15.4</span> <span class="removal-date"></span> - -In GitLab 15.4, we will be swapping the bundled Grafana to a fork of Grafana maintained by GitLab. - -There was an [identified CVE for Grafana](https://nvd.nist.gov/vuln/detail/CVE-2022-31107), and to mitigate this security vulnerability, we must swap to our own fork because the older version of Grafana we were bundling is no longer receiving long-term support. +<div class="deprecation breaking-change" data-milestone="16.0"> -This is not expected to cause any incompatibilities with the previous version of Grafana. Neither when using our bundled version, nor when using an external instance of Grafana. +### ZenTao integration +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- End of Support: GitLab <span class="milestone">16.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### CAS OmniAuth provider +The [ZenTao product integration](https://docs.gitlab.com/ee/user/project/integrations/zentao.html) has been deprecated +and will be moved to the JiHu GitLab codebase. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -The `omniauth-cas3` gem that provides GitLab with the CAS OmniAuth provider will be removed in our next major -release, GitLab 16.0. This gem sees very little use and its lack of upstream maintenance is preventing GitLab's -[upgrade to OmniAuth 2.0](https://gitlab.com/gitlab-org/gitlab/-/issues/30073). +### `CI_BUILD_*` predefined variables +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160"> +The predefined CI/CD variables that start with `CI_BUILD_*` were deprecated in GitLab 9.0, and will be removed in GitLab 16.0. If you still use these variables, be sure to change to the replacement [predefined variables](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) which are functionally identical: -### Maximum number of active pipelines per project limit (`ci_active_pipelines`) +| Removed variable | Replacement variable | +| --------------------- |------------------------ | +| `CI_BUILD_BEFORE_SHA` | `CI_COMMIT_BEFORE_SHA` | +| `CI_BUILD_ID` | `CI_JOB_ID` | +| `CI_BUILD_MANUAL` | `CI_JOB_MANUAL` | +| `CI_BUILD_NAME` | `CI_JOB_NAME` | +| `CI_BUILD_REF` | `CI_COMMIT_SHA` | +| `CI_BUILD_REF_NAME` | `CI_COMMIT_REF_NAME` | +| `CI_BUILD_REF_SLUG` | `CI_COMMIT_REF_SLUG` | +| `CI_BUILD_REPO` | `CI_REPOSITORY_URL` | +| `CI_BUILD_STAGE` | `CI_JOB_STAGE` | +| `CI_BUILD_TAG` | `CI_COMMIT_TAG` | +| `CI_BUILD_TOKEN` | `CI_JOB_TOKEN` | +| `CI_BUILD_TRIGGERED` | `CI_PIPELINE_TRIGGERED` | -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -The [**Maximum number of active pipelines per project** limit](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#set-cicd-limits) was never enabled by default and will be removed in GitLab 16.0. This limit can also be configured in the Rails console under [`ci_active_pipelines`](https://docs.gitlab.com/ee/administration/instance_limits.html#number-of-pipelines-running-concurrently). Instead, use the other recommended rate limits that offer similar protection: +<div class="deprecation breaking-change" data-milestone="16.0"> -- [**Pipelines rate limits**](https://docs.gitlab.com/ee/user/admin_area/settings/rate_limit_on_pipelines_creation.html). -- [**Total number of jobs in currently active pipelines**](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#set-cicd-limits). +### `POST ci/lint` API endpoint deprecated +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### Redis 5 deprecated +The `POST ci/lint` API endpoint is deprecated in 15.7, and will be removed in 16.0. This endpoint does not validate the full range of CI/CD configuration options. Instead, use [`POST /projects/:id/ci/lint`](https://docs.gitlab.com/ee/api/lint.html#validate-a-ci-yaml-configuration-with-a-namespace), which properly validates CI/CD configuration. -End of Support: GitLab <span class="removal-milestone">15.6</span> <span class="support-end-date"></span><br /> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="16.0"> -With GitLab 13.9, in the Omnibus GitLab package and GitLab Helm chart 4.9, the Redis version [was updated to Redis 6](https://about.gitlab.com/releases/2021/02/22/gitlab-13-9-released/#omnibus-improvements). -Redis 5 has reached the end of life in April 2022 and will no longer be supported as of GitLab 15.6. -If you are using your own Redis 5.0 instance, you should upgrade it to Redis 6.0 or higher before upgrading to GitLab 16.0 or higher. +### `environment_tier` parameter for DORA API +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### Security report schemas version 14.x.x - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +To avoid confusion and duplication, the `environment_tier` parameter is deprecated in favor of the `environment_tiers` parameter. The new `environment_tiers` parameter allows DORA APIs to return aggregated data for multiple tiers at the same time. The `environment_tier` parameter will be removed in GitLab 16.0. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -Version 14.x.x [security report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas) are deprecated. +<div class="deprecation breaking-change" data-milestone="16.0"> -In GitLab 15.8 and later, [security report scanner integrations](https://docs.gitlab.com/ee/development/integrations/secure.html) that use schema version 14.x.x will display a deprecation warning in the pipeline's **Security** tab. +### project.pipeline.securityReportFindings GraphQL query -In GitLab 16.0 and later, the feature will be removed. Security reports that use schema version 14.x.x will cause an error in the pipeline's **Security** tab. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.1</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -For more information, refer to [security report validation](https://docs.gitlab.com/ee/user/application_security/#security-report-validation). +Previous work helped [align the vulnerabilities calls for pipeline security tabs](https://gitlab.com/gitlab-org/gitlab/-/issues/343469) to match the vulnerabilities calls for project-level and group-level vulnerability reports. This helped the frontend have a more consistent interface. The old `project.pipeline.securityReportFindings` query was formatted differently than other vulnerability data calls. Now that it has been replaced with the new `project.pipeline.vulnerabilities` field, the old `project.pipeline.securityReportFindings` is being deprecated and will be removed in GitLab 16.0. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="16.0"> -### Use of `id` field in vulnerabilityFindingDismiss mutation - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +### vulnerabilityFindingDismiss GraphQL mutation -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -You can use the vulnerabilityFindingDismiss GraphQL mutation to set the status of a vulnerability finding to `Dismissed`. Previously, this mutation used the `id` field to identify findings uniquely. However, this did not work for dismissing findings from the pipeline security tab. Therefore, using the `id` field as an identifier has been dropped in favor of the `uuid` field. Using the 'uuid' field as an identifier allows you to dismiss the finding from the pipeline security tab. +The `VulnerabilityFindingDismiss` GraphQL mutation is being deprecated and will be removed in GitLab 16.0. This mutation was not used often as the Vulnerability Finding ID was not available to users (this field was [deprecated in 15.3](https://docs.gitlab.com/ee/update/deprecations.html#use-of-id-field-in-vulnerabilityfindingdismiss-mutation)). Users should instead use `VulnerabilityDismiss` to dismiss vulnerabilities in the Vulnerability Report or `SecurityFindingDismiss` for security findings in the CI Pipeline Security tab. </div> </div> -<div class="announcement-milestone"> +<div class="milestone-wrapper" data-milestone="15.11"> -## Announced in 15.2 +## GitLab 15.11 -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation " data-milestone="15.11"> -### Remove `job_age` parameter from `POST /jobs/request` Runner endpoint +### openSUSE Leap 15.3 packages -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +Distribution support and security updates for openSUSE Leap 15.3 [ended December 2022](https://en.opensuse.org/Lifetime#Discontinued_distributions). -The `job_age` parameter, returned from the `POST /jobs/request` API endpoint used in communication with GitLab Runner, was never used by any GitLab or Runner feature. This parameter will be removed in GitLab 16.0. +Starting in GitLab 15.7 we started providing packages for openSUSE Leap 15.4, and will stop providing packages for openSUSE Leap 15.3 in the 15.11 milestone. -This could be a breaking change for anyone that developed their own runner that relies on this parameter being returned by the endpoint. This is not a breaking change for anyone using an officially released version of GitLab Runner, including public shared runners on GitLab.com. +- Switch from the openSUSE Leap 15.3 packages to the provided 15.4 packages. </div> </div> -<div class="announcement-milestone"> - -## Announced in 15.1 +<div class="milestone-wrapper" data-milestone="15.10"> -<div class="deprecation removal-160 breaking-change"> +## GitLab 15.10 -### Jira GitHub Enterprise DVCS integration +<div class="deprecation breaking-change" data-milestone="15.10"> -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The [Jira DVCS Connector](https://docs.gitlab.com/ee/integration/jira/dvcs/) (which enables the [Jira Development Panel](https://support.atlassian.com/jira-software-cloud/docs/view-development-information-for-an-issue/)), will no longer support Jira Cloud users starting with GitLab 16.0. The [GitLab for Jira App](https://docs.gitlab.com/ee/integration/jira/connect-app.html) has always been recommended for Jira Cloud users, and it will be required instead of the DVCS connector. If you are a Jira Cloud user, we recommended you begin migrating to the GitLab for Jira App. -Any Jira Server and Jira Data Center users will need to confirm they are not using the GitHub Enterprise Connector to enable the GitLab DVCS integration, but they may continue to use the [native GitLab DVCS integration](https://docs.gitlab.com/ee/integration/jira/dvcs/) (supported in Jira 8.14 and later). +### Automatic backup upload using Openstack Swift and Rackspace APIs +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- End of Support: GitLab <span class="milestone">15.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### PipelineSecurityReportFinding name GraphQL field - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +We are deprecating support for [uploading backups to remote storage](https://docs.gitlab.com/ee/raketasks/backup_gitlab.html#upload-backups-to-a-remote-cloud-storage) using Openstack Swift and Rackspace APIs. The support for these APIs depends on third-party libraries that are no longer actively maintained and have not been updated for Ruby 3. GitLab is switching over to Ruby 3 prior to EOL of Ruby 2 in order to stay up to date on security patches. -Previously, the [PipelineSecurityReportFinding GraphQL type was updated](https://gitlab.com/gitlab-org/gitlab/-/issues/335372) to include a new `title` field. This field is an alias for the current `name` field, making the less specific `name` field redundant. The `name` field will be removed from the PipelineSecurityReportFinding type in GitLab 16.0. +- If you're using OpenStack, you need to change you configuration to use the S3 API instead of Swift. +- If you're using Rackspace storage, you need to switch to a different provider or manually upload the backup file after the backup task is complete. </div> +</div> -<div class="deprecation removal-160 breaking-change"> +<div class="milestone-wrapper" data-milestone="15.9"> -### PipelineSecurityReportFinding projectFingerprint GraphQL field +## GitLab 15.9 -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.9"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The [`project_fingerprint`](https://gitlab.com/groups/gitlab-org/-/epics/2791) attribute of vulnerability findings is being deprecated in favor of a `uuid` attribute. By using UUIDv5 values to identify findings, we can easily associate any related entity with a finding. The `project_fingerprint` attribute is no longer being used to track findings, and will be removed in GitLab 16.0. +### Live Preview no longer available in the Web IDE +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-153"> +The Live Preview feature of the Web IDE was intended to provide a client-side preview of static web applications. However, complex configuration steps and a narrow set of supported project types have limited its utility. With the introduction of the Web IDE Beta in GitLab 15.7, you can now connect to a full server-side runtime environment. With upcoming support for installing extensions in the Web IDE, we'll also support more advanced workflows than those available with Live Preview. As of GitLab 15.9, Live Preview is no longer available in the Web IDE. -### Vulnerability Report sort by Tool +</div> -Planned removal: GitLab <span class="removal-milestone">15.3</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.9"> -The ability to sort the Vulnerability Report by the `Tool` column (scan type) was disabled and put behind a feature flag in GitLab 14.10 due to a refactor -of the underlying data model. The feature flag has remained off by default as further refactoring will be required to ensure sorting -by this value remains performant. Due to very low usage of the `Tool` column for sorting, the feature flag will instead be removed in -GitLab 15.3 to simplify the codebase and prevent any unwanted performance degradation. +### SaaS certificate-based integration with Kubernetes +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +The certificate-based integration with Kubernetes will be [deprecated and removed](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/). As a GitLab SaaS customer, on new namespaces, you will no longer be able to integrate GitLab and your cluster using the certificate-based approach as of GitLab 15.0. The integration for current users will be enabled per namespace. -### project.pipeline.securityReportFindings GraphQL query +For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the +[agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +Although an explicit removal date is set, we don't plan to remove this feature until the new solution has feature parity. +For more information about the blockers to removal, see [this issue](https://gitlab.com/gitlab-org/configure/general/-/issues/199). -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +For updates and details about this deprecation, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). -Previous work helped [align the vulnerabilities calls for pipeline security tabs](https://gitlab.com/gitlab-org/gitlab/-/issues/343469) to match the vulnerabilities calls for project-level and group-level vulnerability reports. This helped the frontend have a more consistent interface. The old `project.pipeline.securityReportFindings` query was formatted differently than other vulnerability data calls. Now that it has been replaced with the new `project.pipeline.vulnerabilities` field, the old `project.pipeline.securityReportFindings` is being deprecated and will be removed in GitLab 16.0. +GitLab self-managed customers can still use the feature [with a feature flag](https://docs.gitlab.com/ee/update/deprecations.html#self-managed-certificate-based-integration-with-kubernetes). </div> </div> -<div class="announcement-milestone"> +<div class="milestone-wrapper" data-milestone="15.7"> -## Announced in 15.0 +## GitLab 15.7 -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.7"> -### CiCdSettingsUpdate mutation renamed to ProjectCiCdSettingsUpdate +### File Type variable expansion in `.gitlab-ci.yml` -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +Previously, variables that referenced or applied alias file variables expanded the value of the `File` type variable. For example, the file contents. This behavior was incorrect because it did not comply with typical shell variable expansion rules. To leak secrets or sensitive information stored in `File` type variables, a user could run an $echo command with the variable as an input parameter. -The `CiCdSettingsUpdate` mutation was renamed to `ProjectCiCdSettingsUpdate` in GitLab 15.0. -The `CiCdSettingsUpdate` mutation will be removed in GitLab 16.0. -Any user scripts that use the `CiCdSettingsUpdate` mutation must be updated to use `ProjectCiCdSettingsUpdate` -instead. +This breaking change fixes this issue but could disrupt user workflows that work around the behavior. With this change, job variable expansions that reference or apply alias file variables, expand to the file name or path of the `File` type variable, instead of its value, such as the file contents. </div> +</div> -<div class="deprecation removal-160 breaking-change"> - -### GraphQL API legacyMode argument for Runner status - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="milestone-wrapper" data-milestone="15.6"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +## GitLab 15.6 -The `legacyMode` argument to the `status` field in `RunnerType` will be rendered non-functional in the 16.0 release -as part of the deprecations details in the [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). +<div class="deprecation " data-milestone="15.6"> -In GitLab 16.0 and later, the `status` field will act as if `legacyMode` is null. The `legacyMode` argument will -be present during the 16.x cycle to avoid breaking the API signature, and will be removed altogether in the -17.0 release. +### NFS for Git repository storage +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.0</span> </div> -<div class="deprecation removal-160 breaking-change"> - -### PostgreSQL 12 deprecated - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +With the general availability of Gitaly Cluster ([introduced in GitLab 13.0](https://about.gitlab.com/releases/2020/05/22/gitlab-13-0-released/)), we have deprecated development (bugfixes, performance improvements, etc) for NFS for Git repository storage in GitLab 14.0. We will continue to provide technical support for NFS for Git repositories throughout 14.x, but we will remove all support for NFS on November 22, 2022. This was originally planned for May 22, 2022, but in an effort to allow continued maturity of Gitaly Cluster, we have chosen to extend our deprecation of support date. Please see our official [Statement of Support](https://about.gitlab.com/support/statement-of-support/#gitaly-and-nfs) for further information. -Support for PostgreSQL 12 is scheduled for removal in GitLab 16.0. -In GitLab 16.0, PostgreSQL 13 becomes the minimum required PostgreSQL version. +Gitaly Cluster offers tremendous benefits for our customers such as: -PostgreSQL 12 will be supported for the full GitLab 15 release cycle. -PostgreSQL 13 will also be supported for instances that want to upgrade prior to GitLab 16.0. +- [Variable replication factors](https://docs.gitlab.com/ee/administration/gitaly/index.html#replication-factor). +- [Strong consistency](https://docs.gitlab.com/ee/administration/gitaly/index.html#strong-consistency). +- [Distributed read capabilities](https://docs.gitlab.com/ee/administration/gitaly/index.html#distributed-reads). -Upgrading to PostgreSQL 13 is not yet supported for GitLab instances with Geo enabled. Geo support for PostgreSQL 13 will be announced in a minor release version of GitLab 15, after the process is fully supported and validated. For more information, read the Geo related verifications on the [support epic for PostgreSQL 13](https://gitlab.com/groups/gitlab-org/-/epics/3832). +We encourage customers currently using NFS for Git repositories to plan their migration by reviewing our documentation on [migrating to Gitaly Cluster](https://docs.gitlab.com/ee/administration/gitaly/index.html#migrate-to-gitaly-cluster). </div> +</div> -<div class="deprecation removal-153"> +<div class="milestone-wrapper" data-milestone="15.4"> -### Vulnerability Report sort by State +## GitLab 15.4 -Planned removal: GitLab <span class="removal-milestone">15.3</span> <span class="removal-date"></span> +<div class="deprecation " data-milestone="15.4"> -The ability to sort the Vulnerability Report by the `State` column was disabled and put behind a feature flag in GitLab 14.10 due to a refactor -of the underlying data model. The feature flag has remained off by default as further refactoring will be required to ensure sorting -by this value remains performant. Due to very low usage of the `State` column for sorting, the feature flag will instead be removed to simplify the codebase and prevent any unwanted performance degradation. +### Bundled Grafana deprecated -</div> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.3</span> </div> -<div class="announcement-milestone"> - -## Announced in 14.10 +In GitLab 15.4, we will be swapping the bundled Grafana to a fork of Grafana maintained by GitLab. -<div class="deprecation removal-150 breaking-change"> +There was an [identified CVE for Grafana](https://nvd.nist.gov/vuln/detail/CVE-2022-31107), and to mitigate this security vulnerability, we must swap to our own fork because the older version of Grafana we were bundling is no longer receiving long-term support. -### Dependency Scanning default Java version changed to 17 +This is not expected to cause any incompatibilities with the previous version of Grafana. Neither when using our bundled version, nor when using an external instance of Grafana. -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="15.4"> -In GitLab 15.0, for Dependency Scanning, the default version of Java that the scanner expects will be updated from 11 to 17. Java 17 is [the most up-to-date Long Term Support (LTS) version](https://en.wikipedia.org/wiki/Java_version_history). Dependency scanning continues to support the same [range of versions (8, 11, 13, 14, 15, 16, 17)](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#supported-languages-and-package-managers), only the default version is changing. If your project uses the previous default of Java 11, be sure to [set the `DS_Java_Version` variable to match](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuring-specific-analyzers-used-by-dependency-scanning). +### SAST analyzer consolidation and CI/CD template changes +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> +GitLab SAST uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/sast/analyzers/) to scan code for vulnerabilities. -### Outdated indices of Advanced Search migrations +We are reducing the number of analyzers used in GitLab SAST as part of our long-term strategy to deliver a better and more consistent user experience. +Streamlining the set of analyzers will also enable faster [iteration](https://about.gitlab.com/handbook/values/#iteration), better [results](https://about.gitlab.com/handbook/values/#results), and greater [efficiency](https://about.gitlab.com/handbook/values/#efficiency) (including a reduction in CI runner usage in most cases). -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +In GitLab 15.4, GitLab SAST will no longer use the following analyzers: -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +- [ESLint](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) (JavaScript, TypeScript, React) +- [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) (Go) +- [Bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) (Python) -As Advanced Search migrations usually require support multiple code paths for a long period of time, it’s important to clean those up when we safely can. We use GitLab major version upgrades as a safe time to remove backward compatibility for indices that have not been fully migrated. See the [upgrade documentation](https://docs.gitlab.com/ee/update/index.html#upgrading-to-a-new-major-version) for details. +NOTE: +This change was originally planned for GitLab 15.0 and was postponed to GitLab 15.4. +See [the removal notice](./removals.md#sast-analyzer-consolidation-and-cicd-template-changes) for further details. -</div> +These analyzers will be removed from the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) and replaced with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). +Effective immediately, they will receive only security updates; other routine improvements or updates are not guaranteed. +After these analyzers reach End of Support, no further updates will be provided. +We will not delete container images previously published for these analyzers; any such change would be announced as a [deprecation, removal, or breaking change announcement](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes). -<div class="deprecation removal-160 breaking-change"> +We will also remove Java from the scope of the [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) analyzer and replace it with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). +This change will make it simpler to scan Java code; compilation will no longer be required. +This change will be reflected in the automatic language detection portion of the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml). Note that the SpotBugs-based analyzer will continue to cover Groovy, Kotlin, and Scala. -### Toggle notes confidentiality on APIs +If you've already dismissed a vulnerability finding from one of the deprecated analyzers, the replacement attempts to respect your previous dismissal. The system behavior depends on: -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +- whether you’ve excluded the Semgrep-based analyzer from running in the past. +- which analyzer first discovered the vulnerabilities shown in the project’s Vulnerability Report. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +See [Vulnerability translation documentation](https://docs.gitlab.com/ee/user/application_security/sast/analyzers.html#vulnerability-translation) for further details. -Toggling notes confidentiality with REST and GraphQL APIs is being deprecated. Updating notes confidential attribute is no longer supported by any means. We are changing this to simplify the experience and prevent private information from being unintentionally exposed. +If you applied customizations to any of the affected analyzers or if you currently disable the Semgrep analyzer in your pipelines, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352554#breaking-change). </div> </div> -<div class="announcement-milestone"> +<div class="milestone-wrapper" data-milestone="15.3"> -## Announced in 14.9 +## GitLab 15.3 -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation " data-milestone="15.3"> -### Background upload for object storage +### Vulnerability Report sort by State -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.0</span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The ability to sort the Vulnerability Report by the `State` column was disabled and put behind a feature flag in GitLab 14.10 due to a refactor +of the underlying data model. The feature flag has remained off by default as further refactoring will be required to ensure sorting +by this value remains performant. Due to very low usage of the `State` column for sorting, the feature flag will instead be removed to simplify the codebase and prevent any unwanted performance degradation. -To reduce the overall complexity and maintenance burden of GitLab's [object storage feature](https://docs.gitlab.com/ee/administration/object_storage.html), support for using `background_upload` to upload files is deprecated and will be fully removed in GitLab 15.0. Review the [15.0 specific changes](https://docs.gitlab.com/omnibus/update/gitlab_15_changes.html) for the [removed background uploads settings for object storage](https://docs.gitlab.com/omnibus/update/gitlab_15_changes.html#removed-background-uploads-settings-for-object-storage). +</div> -This impacts a small subset of object storage providers: +<div class="deprecation " data-milestone="15.3"> -- **OpenStack** Customers using OpenStack need to change their configuration to use the S3 API instead of Swift. -- **RackSpace** Customers using RackSpace-based object storage need to migrate data to a different provider. +### Vulnerability Report sort by Tool -GitLab will publish additional guidance to assist affected customers in migrating. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.1</span> +</div> +The ability to sort the Vulnerability Report by the `Tool` column (scan type) was disabled and put behind a feature flag in GitLab 14.10 due to a refactor +of the underlying data model. The feature flag has remained off by default as further refactoring will be required to ensure sorting +by this value remains performant. Due to very low usage of the `Tool` column for sorting, the feature flag will instead be removed in +GitLab 15.3 to simplify the codebase and prevent any unwanted performance degradation. + +</div> </div> -<div class="deprecation removal-151"> +<div class="milestone-wrapper" data-milestone="15.1"> + +## GitLab 15.1 + +<div class="deprecation " data-milestone="15.1"> ### Deprecate support for Debian 9 -Planned removal: GitLab <span class="removal-milestone">15.1</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.9</span> +</div> Long term service and support (LTSS) for [Debian 9 Stretch ends in July 2022](https://wiki.debian.org/LTS). Therefore, we will no longer support the Debian 9 distribution for the GitLab package. Users can upgrade to Debian 10 or Debian 11. </div> +</div> -<div class="deprecation removal-150"> +<div class="milestone-wrapper" data-milestone="15.0"> -### GitLab Pages running as daemon +## GitLab 15.0 -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.0"> -In 15.0, support for daemon mode for GitLab Pages will be removed. +### Audit events for repository push events +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.3</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> +Audit events for [repository events](https://docs.gitlab.com/ee/administration/audit_events.html#removed-events) are now deprecated and will be removed in GitLab 15.0. -### GitLab self-monitoring project +These events have always been disabled by default and had to be manually enabled with a +feature flag. Enabling them can cause too many events to be generated which can +dramatically slow down GitLab instances. For this reason, they are being removed. -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="15.0"> -GitLab self-monitoring gives administrators of self-hosted GitLab instances the tools to monitor the health of their instances. This feature is deprecated in GitLab 14.9, and is scheduled for removal in 16.0. +### Background upload for object storage +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> +To reduce the overall complexity and maintenance burden of GitLab's [object storage feature](https://docs.gitlab.com/ee/administration/object_storage.html), support for using `background_upload` to upload files is deprecated and will be fully removed in GitLab 15.0. Review the [15.0 specific changes](https://docs.gitlab.com/omnibus/update/gitlab_15_changes.html) for the [removed background uploads settings for object storage](https://docs.gitlab.com/omnibus/update/gitlab_15_changes.html#removed-background-uploads-settings-for-object-storage). -### GraphQL permissions change for Package settings +This impacts a small subset of object storage providers: -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +- **OpenStack** Customers using OpenStack need to change their configuration to use the S3 API instead of Swift. +- **RackSpace** Customers using RackSpace-based object storage need to migrate data to a different provider. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +GitLab will publish additional guidance to assist affected customers in migrating. -The GitLab Package stage offers a Package Registry, Container Registry, and Dependency Proxy to help you manage all of your dependencies using GitLab. Each of these product categories has a variety of settings that can be adjusted using the API. +</div> -The permissions model for GraphQL is being updated. After 15.0, users with the Guest, Reporter, and Developer role can no longer update these settings: +<div class="deprecation breaking-change" data-milestone="15.0"> -- [Package Registry settings](https://docs.gitlab.com/ee/api/graphql/reference/#packagesettings) -- [Container Registry cleanup policy](https://docs.gitlab.com/ee/api/graphql/reference/#containerexpirationpolicy) -- [Dependency Proxy time-to-live policy](https://docs.gitlab.com/ee/api/graphql/reference/#dependencyproxyimagettlgrouppolicy) -- [Enabling the Dependency Proxy for your group](https://docs.gitlab.com/ee/api/graphql/reference/#dependencyproxysetting) +### CI/CD job name length limit +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.6</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150"> +In GitLab 15.0 we are going to limit the number of characters in CI/CD job names to 255. Any pipeline with job names that exceed the 255 character limit will stop working after the 15.0 release. -### Move `custom_hooks_dir` setting from GitLab Shell to Gitaly +</div> -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.0"> -The [`custom_hooks_dir`](https://docs.gitlab.com/ee/administration/server_hooks.html#create-a-global-server-hook-for-all-repositories) setting is now configured in Gitaly, and will be removed from GitLab Shell in GitLab 15.0. +### Changing an instance (shared) runner to a project (specific) runner +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-1410 breaking-change"> +In GitLab 15.0, you can no longer change an instance (shared) runner to a project (specific) runner. -### Permissions change for downloading Composer dependencies +Users often accidentally change instance runners to project runners, and they're unable to change them back. GitLab does not allow you to change a project runner to a shared runner because of the security implications. A runner meant for one project could be set to run jobs for an entire instance. -Planned removal: GitLab <span class="removal-milestone">14.10</span> <span class="removal-date"></span> +Administrators who need to add runners for multiple projects can register a runner for one project, then go to the Admin view and choose additional projects. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +</div> -The GitLab Composer repository can be used to push, search, fetch metadata about, and download PHP dependencies. All these actions require authentication, except for downloading dependencies. +<div class="deprecation breaking-change" data-milestone="15.0"> -Downloading Composer dependencies without authentication is deprecated in GitLab 14.9, and will be removed in GitLab 15.0. Starting with GitLab 15.0, you must authenticate to download Composer dependencies. +### Container Network and Host Security +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### htpasswd Authentication for the Container Registry - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +All functionality related to GitLab's Container Network Security and Container Host Security categories is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. Users who need a replacement for this functionality are encouraged to evaluate the following open source projects as potential solutions that can be installed and managed outside of GitLab: [AppArmor](https://gitlab.com/apparmor/apparmor), [Cilium](https://github.com/cilium/cilium), [Falco](https://github.com/falcosecurity/falco), [FluentD](https://github.com/fluent/fluentd), [Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/). To integrate these technologies into GitLab, add the desired Helm charts into your copy of the [Cluster Management Project Template](https://docs.gitlab.com/ee/user/clusters/management_project_template.html). Deploy these Helm charts in production by calling commands through GitLab [CI/CD](https://docs.gitlab.com/ee/user/clusters/agent/ci_cd_workflow.html). -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +As part of this change, the following specific capabilities within GitLab are now deprecated, and are scheduled for removal in GitLab 15.0: -The Container Registry supports [authentication](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs/configuration.md#auth) with `htpasswd`. It relies on an [Apache `htpasswd` file](https://httpd.apache.org/docs/2.4/programs/htpasswd.html), with passwords hashed using `bcrypt`. +- The **Security & Compliance > Threat Monitoring** page. +- The `Network Policy` security policy type, as found on the **Security & Compliance > Policies** page. +- The ability to manage integrations with the following technologies through GitLab: AppArmor, Cilium, Falco, FluentD, and Pod Security Policies. +- All APIs related to the above functionality. -Since it isn't used in the context of GitLab (the product), `htpasswd` authentication will be deprecated in GitLab 14.9 and removed in GitLab 15.0. +For additional context, or to provide feedback regarding this change, please reference our open [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/7476). </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation " data-milestone="15.0"> -### user_email_lookup_limit API field +### Container scanning schemas below 14.0.0 -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +[Container scanning report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) +versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation +against the schema version declared in the report will also no longer be supported in GitLab 15.0. -The `user_email_lookup_limit` [API field](https://docs.gitlab.com/ee/api/settings.html) is deprecated and will be removed in GitLab 15.0. Until GitLab 15.0, `user_email_lookup_limit` is aliased to `search_rate_limit` and existing workflows will continue to work. +Third-party tools that [integrate with GitLab by outputting a container scanning security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) +as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate against the declared schema version will not be processed, and vulnerability findings will not display in MRs, pipelines, or Vulnerability Reports. -Any API calls attempting to change the rate limits for `user_email_lookup_limit` should use `search_rate_limit` instead. +To help with the transition, from GitLab 14.10, non-compliant reports will display a +[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) +in the Vulnerability Report. </div> -</div> -<div class="announcement-milestone"> +<div class="deprecation " data-milestone="15.0"> -## Announced in 14.8 - -<div class="deprecation removal-149"> +### Coverage guided fuzzing schemas below 14.0.0 -### Configurable Gitaly `per_repository` election strategy +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +</div> -Planned removal: GitLab <span class="removal-milestone">14.9</span> <span class="removal-date"></span> +[Coverage guided fuzzing report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) +below version 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation +against the schema version declared in the report will also no longer be supported in GitLab 15.0. -Configuring the `per_repository` Gitaly election strategy is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352612). -`per_repository` has been the only option since GitLab 14.0. +Third-party tools that [integrate with GitLab by outputting a coverage guided fuzzing security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) +as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct +schema with a minimum version of 14.0.0. Any reports with a lower version or that fail to validate +against the declared schema version will not be processed, and vulnerability +findings will not display in MRs, pipelines, or Vulnerability Reports. -This change is part of regular maintenance to keep our codebase clean. +To help with the transition, from GitLab 14.10, non-compliant reports will display a +[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) +in the Vulnerability Report. </div> -<div class="deprecation removal-150 breaking-change"> - -### Container Network and Host Security - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation " data-milestone="15.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### DAST schemas below 14.0.0 -All functionality related to GitLab's Container Network Security and Container Host Security categories is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. Users who need a replacement for this functionality are encouraged to evaluate the following open source projects as potential solutions that can be installed and managed outside of GitLab: [AppArmor](https://gitlab.com/apparmor/apparmor), [Cilium](https://github.com/cilium/cilium), [Falco](https://github.com/falcosecurity/falco), [FluentD](https://github.com/fluent/fluentd), [Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/). To integrate these technologies into GitLab, add the desired Helm charts into your copy of the [Cluster Management Project Template](https://docs.gitlab.com/ee/user/clusters/management_project_template.html). Deploy these Helm charts in production by calling commands through GitLab [CI/CD](https://docs.gitlab.com/ee/user/clusters/agent/ci_cd_workflow.html). +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +</div> -As part of this change, the following specific capabilities within GitLab are now deprecated, and are scheduled for removal in GitLab 15.0: +[DAST report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) +versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation +against the schema version declared in the report will also no longer be supported as of GitLab 15.0. -- The **Security & Compliance > Threat Monitoring** page. -- The `Network Policy` security policy type, as found on the **Security & Compliance > Policies** page. -- The ability to manage integrations with the following technologies through GitLab: AppArmor, Cilium, Falco, FluentD, and Pod Security Policies. -- All APIs related to the above functionality. +Third-party tools that [integrate with GitLab by outputting a DAST security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) +as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct +schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate +against the declared schema version will not be processed, and vulnerability +findings will not display in MRs, pipelines, or Vulnerability Reports. -For additional context, or to provide feedback regarding this change, please reference our open [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/7476). +To help with the transition, from GitLab 14.10, non-compliant reports will cause a +[warning to be displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) +in the Vulnerability Report. </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### Dependency Scanning Python 3.9 and 3.6 image deprecation -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> For those using Dependency Scanning for Python projects, we are deprecating the default `gemnasium-python:2` image which uses Python 3.6 as well as the custom `gemnasium-python:2-python-3.9` image which uses Python 3.9. The new default image as of GitLab 15.0 will be for Python 3.9 as it is a [supported version](https://endoflife.date/python) and 3.6 [is no longer supported](https://endoflife.date/python). @@ -2263,21 +2471,62 @@ gemnasium-python-dependency_scanning: </div> -<div class="deprecation removal-150"> +<div class="deprecation breaking-change" data-milestone="15.0"> + +### Dependency Scanning default Java version changed to 17 + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +In GitLab 15.0, for Dependency Scanning, the default version of Java that the scanner expects will be updated from 11 to 17. Java 17 is [the most up-to-date Long Term Support (LTS) version](https://en.wikipedia.org/wiki/Java_version_history). Dependency scanning continues to support the same [range of versions (8, 11, 13, 14, 15, 16, 17)](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#supported-languages-and-package-managers), only the default version is changing. If your project uses the previous default of Java 11, be sure to [set the `DS_Java_Version` variable to match](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuring-specific-analyzers-used-by-dependency-scanning). + +</div> + +<div class="deprecation " data-milestone="15.0"> + +### Dependency scanning schemas below 14.0.0 + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +</div> + +[Dependency scanning report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) +versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation +against the schema version declared in the report will also no longer be supported as of GitLab 15.0. + +Third-party tools that [integrate with GitLab by outputting a Dependency scanning security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) +as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct +schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate +against the declared schema version will not be processed, and vulnerability +findings will not display in MRs, pipelines, or Vulnerability Reports. + +To help with the transition, from GitLab 14.10, non-compliant reports will cause a +[warning to be displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) +in the Vulnerability Report. + +</div> + +<div class="deprecation " data-milestone="15.0"> ### Deprecate Geo Admin UI Routes -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +</div> In GitLab 13.0, we introduced new project and design replication details routes in the Geo Admin UI. These routes are `/admin/geo/replication/projects` and `/admin/geo/replication/designs`. We kept the legacy routes and redirected them to the new routes. In GitLab 15.0, we will remove support for the legacy routes `/admin/geo/projects` and `/admin/geo/designs`. Please update any bookmarks or scripts that may use the legacy routes. </div> -<div class="deprecation removal-150"> +<div class="deprecation " data-milestone="15.0"> ### Deprecate custom Geo:db:* Rake tasks -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +</div> In GitLab 14.8, we are [replacing the `geo:db:*` Rake tasks with built-in tasks](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77269/diffs) that are now possible after [switching the Geo tracking database to use Rails' 6 support of multiple databases](https://gitlab.com/groups/gitlab-org/-/epics/6458). The following `geo:db:*` tasks will be replaced with their corresponding `db:*:geo` tasks: @@ -2302,65 +2551,68 @@ The following `geo:db:*` tasks will be replaced with their corresponding `db:*:g </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### Deprecate feature flag PUSH_RULES_SUPERSEDE_CODE_OWNERS -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The feature flag `PUSH_RULES_SUPERSEDE_CODE_OWNERS` is being removed in GitLab 15.0. Upon its removal, push rules will supersede Code Owners. Even if Code Owner approval is required, a push rule that explicitly allows a specific user to push code supersedes the Code Owners setting. </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> -### Deprecate legacy Gitaly configuration methods - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +### Elasticsearch 6.8 -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -Using environment variables `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configure Gitaly is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352609). -These variables are being replaced with standard [`config.toml` Gitaly configuration](https://docs.gitlab.com/ee/administration/gitaly/reference.html). +Elasticsearch 6.8 is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. +Customers using Elasticsearch 6.8 need to upgrade their Elasticsearch version to 7.x prior to upgrading to GitLab 15.0. +We recommend using the latest version of Elasticsearch 7 to benefit from all Elasticsearch improvements. -GitLab instances that use `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configure Gitaly should switch to configuring using -`config.toml`. +Elasticsearch 6.8 is also incompatible with Amazon OpenSearch, which we [plan to support in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/327560). </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation " data-milestone="15.0"> -### Elasticsearch 6.8 +### Enforced validation of security report schemas -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +[Security report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) +versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation +against the schema version declared in the report will also no longer be supported in GitLab 15.0. -Elasticsearch 6.8 is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. -Customers using Elasticsearch 6.8 need to upgrade their Elasticsearch version to 7.x prior to upgrading to GitLab 15.0. -We recommend using the latest version of Elasticsearch 7 to benefit from all Elasticsearch improvements. +Security tools that [integrate with GitLab by outputting security reports](https://docs.gitlab.com/ee/development/integrations/secure.html#report) +as pipeline job artifacts are affected. You must ensure that all output reports adhere to the correct +schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate +against the declared schema version will not be processed, and vulnerability +findings will not display in MRs, pipelines, or Vulnerability Reports. -Elasticsearch 6.8 is also incompatible with Amazon OpenSearch, which we [plan to support in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/327560). +To help with the transition, from GitLab 14.10, non-compliant reports will display a +[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) +in the Vulnerability Report. </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### External status check API breaking changes -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The [external status check API](https://docs.gitlab.com/ee/api/status_checks.html) was originally implemented to support pass-by-default requests to mark a status check as passing. Pass-by-default requests are now deprecated. @@ -2380,15 +2632,56 @@ To align with this change, API calls to list external status checks will also re </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation " data-milestone="15.0"> -### GraphQL ID and GlobalID compatibility +### GitLab Pages running as daemon -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.9</span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +In 15.0, support for daemon mode for GitLab Pages will be removed. + +</div> + +<div class="deprecation breaking-change" data-milestone="15.0"> + +### GitLab Serverless + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.3</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +GitLab Serverless is a feature set to support Knative-based serverless development with automatic deployments and monitoring. + +We decided to remove the GitLab Serverless features as they never really resonated with our users. Besides, given the continuous development of Kubernetes and Knative, our current implementations do not even work with recent versions. + +</div> + +<div class="deprecation " data-milestone="15.0"> + +### Godep support in License Compliance + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +</div> + +The Godep dependency manager for Golang was deprecated in 2020 by Go and +has been replaced with Go modules. +To reduce our maintenance cost we are deprecating License Compliance for Godep projects as of 14.7 +and will remove it in GitLab 15.0 + +</div> + +<div class="deprecation breaking-change" data-milestone="15.0"> + +### GraphQL ID and GlobalID compatibility + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> We are removing a non-standard extension to our GraphQL processor, which we added for backwards compatibility. This extension modifies the validation of GraphQL queries, allowing the use of the `ID` type for arguments where it would normally be rejected. Some arguments originally had the type `ID`. These were changed to specific @@ -2444,15 +2737,119 @@ an inline argument expression). </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> -### OAuth tokens without expiration +### GraphQL permissions change for Package settings -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +The GitLab Package stage offers a Package Registry, Container Registry, and Dependency Proxy to help you manage all of your dependencies using GitLab. Each of these product categories has a variety of settings that can be adjusted using the API. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The permissions model for GraphQL is being updated. After 15.0, users with the Guest, Reporter, and Developer role can no longer update these settings: + +- [Package Registry settings](https://docs.gitlab.com/ee/api/graphql/reference/#packagesettings) +- [Container Registry cleanup policy](https://docs.gitlab.com/ee/api/graphql/reference/#containerexpirationpolicy) +- [Dependency Proxy time-to-live policy](https://docs.gitlab.com/ee/api/graphql/reference/#dependencyproxyimagettlgrouppolicy) +- [Enabling the Dependency Proxy for your group](https://docs.gitlab.com/ee/api/graphql/reference/#dependencyproxysetting) + +</div> + +<div class="deprecation breaking-change" data-milestone="15.0"> + +### Known host required for GitLab Runner SSH executor + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +In [GitLab 14.3](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3074), we added a configuration setting in the GitLab Runner `config.toml` file. This setting, [`[runners.ssh.disable_strict_host_key_checking]`](https://docs.gitlab.com/runner/executors/ssh.html#security), controls whether or not to use strict host key checking with the SSH executor. + +In GitLab 15.0 and later, the default value for this configuration option will change from `true` to `false`. This means that strict host key checking will be enforced when using the GitLab Runner SSH executor. + +</div> + +<div class="deprecation breaking-change" data-milestone="15.0"> + +### Legacy approval status names from License Compliance API + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.6</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +We deprecated legacy names for approval status of license policy (blacklisted, approved) in the `managed_licenses` API but they are still used in our API queries and responses. They will be removed in 15.0. + +If you are using our License Compliance API you should stop using the `approved` and `blacklisted` query parameters, they are now `allowed` and `denied`. In 15.0 the responses will also stop using `approved` and `blacklisted` so you need to adjust any of your custom tools to use the old and new values so they do not break with the 15.0 release. + +</div> + +<div class="deprecation breaking-change" data-milestone="15.0"> + +### Legacy database configuration + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.3</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +The syntax of [GitLabs database](https://docs.gitlab.com/omnibus/settings/database.html) +configuration located in `database.yml` is changing and the legacy format is deprecated. The legacy format +supported using a single PostgreSQL adapter, whereas the new format is changing to support multiple databases. The `main:` database needs to be defined as a first configuration item. + +This deprecation mainly impacts users compiling GitLab from source because Omnibus will handle this configuration automatically. + +</div> + +<div class="deprecation breaking-change" data-milestone="15.0"> + +### Logging in GitLab + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +The logging features in GitLab allow users to install the ELK stack (Elasticsearch, Logstash, and Kibana) to aggregate and manage application logs. Users can search for relevant logs in GitLab. However, since deprecating certificate-based integration with Kubernetes clusters and GitLab Managed Apps, we don't have a recommended solution for logging within GitLab. For more information, you can follow the issue for [integrating Opstrace with GitLab](https://gitlab.com/groups/gitlab-org/-/epics/6976). + +</div> + +<div class="deprecation " data-milestone="15.0"> + +### Move `custom_hooks_dir` setting from GitLab Shell to Gitaly + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.9</span> +</div> + +The [`custom_hooks_dir`](https://docs.gitlab.com/ee/administration/server_hooks.html#create-a-global-server-hook-for-all-repositories) setting is now configured in Gitaly, and will be removed from GitLab Shell in GitLab 15.0. + +</div> + +<div class="deprecation breaking-change" data-milestone="15.0"> + +### OAuth implicit grant + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.0</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +The OAuth implicit grant authorization flow will be removed in our next major release, GitLab 15.0. Any applications that use OAuth implicit grant should switch to alternative [supported OAuth flows](https://docs.gitlab.com/ee/api/oauth2.html). + +</div> + +<div class="deprecation breaking-change" data-milestone="15.0"> + +### OAuth tokens without expiration + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> By default, all new applications expire access tokens after 2 hours. In GitLab 14.2 and earlier, OAuth access tokens had no expiration. In GitLab 15.0, an expiry will be automatically generated for any existing token that does not @@ -2466,15 +2863,31 @@ tokens before GitLab 15.0 is released: </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> -### Optional enforcement of PAT expiration +### OmniAuth Kerberos gem + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.3</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +The `omniauth-kerberos` gem will be removed in our next major release, GitLab 15.0. + +This gem has not been maintained and has very little usage. We therefore plan to remove support for this authentication method and recommend using the Kerberos [SPNEGO](https://en.wikipedia.org/wiki/SPNEGO) integration instead. You can follow the [upgrade instructions](https://docs.gitlab.com/ee/integration/kerberos.html#upgrading-from-password-based-to-ticket-based-kerberos-sign-ins) to upgrade from the `omniauth-kerberos` integration to the supported one. -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +Note that we are not deprecating the Kerberos SPNEGO integration, only the old password-based Kerberos integration. + +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="15.0"> + +### Optional enforcement of PAT expiration + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The feature to disable enforcement of PAT expiration is unusual from a security perspective. We have become concerned that this unusual feature could create unexpected behavior for users. @@ -2482,15 +2895,14 @@ Unexpected behavior in a security feature is inherently dangerous, so we have de </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### Optional enforcement of SSH expiration -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The feature to disable enforcement of SSH expiration is unusual from a security perspective. We have become concerned that this unusual feature could create unexpected behavior for users. @@ -2498,15 +2910,14 @@ Unexpected behavior in a security feature is inherently dangerous, so we have de </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### Out-of-the-box SAST support for Java 8 -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The [GitLab SAST SpotBugs analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) scans [Java, Scala, Groovy, and Kotlin code](https://docs.gitlab.com/ee/user/application_security/sast/#supported-languages-and-frameworks) for security vulnerabilities. For technical reasons, the analyzer must first compile the code before scanning. @@ -2523,29 +2934,55 @@ If you rely on Java 8 being present in the analyzer environment, you must take a </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> -### Querying Usage Trends via the `instanceStatisticsMeasurements` GraphQL node +### Outdated indices of Advanced Search migrations + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.10</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> + +As Advanced Search migrations usually require support multiple code paths for a long period of time, it’s important to clean those up when we safely can. We use GitLab major version upgrades as a safe time to remove backward compatibility for indices that have not been fully migrated. See the [upgrade documentation](https://docs.gitlab.com/ee/update/index.html#upgrading-to-a-new-major-version) for details. + +</div> + +<div class="deprecation " data-milestone="15.0"> + +### Pseudonymizer + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +</div> + +The Pseudonymizer feature is generally unused, +can cause production issues with large databases, +and can interfere with object storage development. +It is now considered deprecated, and will be removed in GitLab 15.0. + +</div> + +<div class="deprecation breaking-change" data-milestone="15.0"> -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +### Querying Usage Trends via the `instanceStatisticsMeasurements` GraphQL node -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The `instanceStatisticsMeasurements` GraphQL node has been renamed to `usageTrendsMeasurements` in 13.10 and the old field name has been marked as deprecated. To fix the existing GraphQL queries, replace `instanceStatisticsMeasurements` with `usageTrendsMeasurements`. </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### Request profiling -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> [Request profiling](https://docs.gitlab.com/ee/administration/monitoring/performance/index.html) is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. @@ -2557,15 +2994,14 @@ For more information, check the [summary section of the deprecation issue](https </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### Required pipeline configurations in Premium tier -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The [required pipeline configuration](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#required-pipeline-configuration) feature is deprecated in GitLab 14.8 for Premium customers and is scheduled for removal in GitLab 15.0. This feature is not deprecated for GitLab Ultimate customers. @@ -2576,15 +3012,14 @@ This change will also help GitLab remain consistent in its tiering strategy with </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### Retire-JS Dependency Scanning tool -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> As of 14.8 the retire.js job is being deprecated from Dependency Scanning. It will continue to be included in our CI/CD template while deprecated. We are removing retire.js from Dependency Scanning on May 22, 2022 in GitLab 15.0. JavaScript scanning functionality will not be affected as it is still being covered by Gemnasium. @@ -2592,60 +3027,38 @@ If you have explicitly excluded retire.js using DS_EXCLUDED_ANALYZERS you will n </div> -<div class="deprecation removal-154 breaking-change"> - -### SAST analyzer consolidation and CI/CD template changes - -Planned removal: GitLab <span class="removal-milestone">15.4</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -GitLab SAST uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/sast/analyzers/) to scan code for vulnerabilities. - -We are reducing the number of analyzers used in GitLab SAST as part of our long-term strategy to deliver a better and more consistent user experience. -Streamlining the set of analyzers will also enable faster [iteration](https://about.gitlab.com/handbook/values/#iteration), better [results](https://about.gitlab.com/handbook/values/#results), and greater [efficiency](https://about.gitlab.com/handbook/values/#efficiency) (including a reduction in CI runner usage in most cases). - -In GitLab 15.4, GitLab SAST will no longer use the following analyzers: - -- [ESLint](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) (JavaScript, TypeScript, React) -- [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) (Go) -- [Bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) (Python) - -NOTE: -This change was originally planned for GitLab 15.0 and was postponed to GitLab 15.4. -See [the removal notice](./removals.md#sast-analyzer-consolidation-and-cicd-template-changes) for further details. - -These analyzers will be removed from the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) and replaced with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). -Effective immediately, they will receive only security updates; other routine improvements or updates are not guaranteed. -After these analyzers reach End of Support, no further updates will be provided. -We will not delete container images previously published for these analyzers; any such change would be announced as a [deprecation, removal, or breaking change announcement](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes). +<div class="deprecation " data-milestone="15.0"> -We will also remove Java from the scope of the [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) analyzer and replace it with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). -This change will make it simpler to scan Java code; compilation will no longer be required. -This change will be reflected in the automatic language detection portion of the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml). Note that the SpotBugs-based analyzer will continue to cover Groovy, Kotlin, and Scala. +### SAST schemas below 14.0.0 -If you've already dismissed a vulnerability finding from one of the deprecated analyzers, the replacement attempts to respect your previous dismissal. The system behavior depends on: +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +</div> -- whether you’ve excluded the Semgrep-based analyzer from running in the past. -- which analyzer first discovered the vulnerabilities shown in the project’s Vulnerability Report. +[SAST report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) +versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation +against the schema version declared in the report will also no longer be supported as of GitLab 15.0. -See [Vulnerability translation documentation](https://docs.gitlab.com/ee/user/application_security/sast/analyzers.html#vulnerability-translation) for further details. +Third-party tools that [integrate with GitLab by outputting a SAST security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) +as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct +schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate +against the declared schema version will not be processed, and vulnerability +findings will not display in MRs, pipelines, or Vulnerability Reports. -If you applied customizations to any of the affected analyzers or if you currently disable the Semgrep analyzer in your pipelines, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352554#breaking-change). +To help with the transition, from GitLab 14.10, non-compliant reports will display a +[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) +in the Vulnerability Report. </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### SAST support for .NET 2.1 -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The GitLab SAST Security Code Scan analyzer scans .NET code for security vulnerabilities. For technical reasons, the analyzer must first build the code to scan it. @@ -2669,11 +3082,13 @@ If you rely on .NET 2.1 support being present in the analyzer image by default, </div> -<div class="deprecation removal-150"> +<div class="deprecation " data-milestone="15.0"> ### Secret Detection configuration variables deprecated -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +</div> To make it simpler and more reliable to [customize GitLab Secret Detection](https://docs.gitlab.com/ee/user/application_security/secret_detection/#customizing-settings), we're deprecating some of the variables that you could previously set in your CI/CD configuration. @@ -2694,15 +3109,38 @@ For further details, see [the deprecation issue for this change](https://gitlab. </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation " data-milestone="15.0"> -### Secure and Protect analyzer images published in new location +### Secret detection schemas below 14.0.0 + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +</div> + +[Secret detection report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) +versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation +against the schema version declared in the report will also no longer be supported as of GitLab 15.0. + +Third-party tools that [integrate with GitLab by outputting a Secret detection security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) +as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct +schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate +against the declared schema version will not be processed, and vulnerability +findings will not display in MRs, pipelines, or Vulnerability Reports. + +To help with the transition, from GitLab 14.10, non-compliant reports will display a +[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) +in the Vulnerability Report. -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="15.0"> + +### Secure and Protect analyzer images published in new location + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> GitLab uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/terminology/#analyzer) to [scan for security vulnerabilities](https://docs.gitlab.com/ee/user/application_security/). Each analyzer is distributed as a container image. @@ -2722,15 +3160,14 @@ See the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352564 </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### Secure and Protect analyzer major version update -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> The Secure and Protect stages will be bumping the major versions of their analyzers in tandem with the GitLab 15.0 release. This major bump will enable a clear delineation for analyzers, between: @@ -2767,408 +3204,191 @@ Specifically, the following are being deprecated and will no longer be updated a </div> -<div class="deprecation removal-150 breaking-change"> - -### Support for gRPC-aware proxy deployed between Gitaly and rest of GitLab +<div class="deprecation breaking-change" data-milestone="15.0"> -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -Although not recommended or documented, it was possible to deploy a gRPC-aware proxy between Gitaly and -the rest of GitLab. For example, NGINX and Envoy. The ability to deploy a gRPC-aware proxy is -[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352517). If you currently use a gRPC-aware proxy for -Gitaly connections, you should change your proxy configuration to use TCP or TLS proxying (OSI layer 4) instead. - -Gitaly Cluster became incompatible with gRPC-aware proxies in GitLab 13.12. Now all GitLab installations will be incompatible with -gRPC-aware proxies, even without Gitaly Cluster. - -By sending some of our internal RPC traffic through a custom protocol (instead of gRPC) we -increase throughput and reduce Go garbage collection latency. For more information, see -the [relevant epic](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/463). +### Sidekiq metrics and health checks configuration +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### Test coverage project CI/CD setting - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +Exporting Sidekiq metrics and health checks using a single process and port is deprecated. +Support will be removed in 15.0. -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +We have updated Sidekiq to export [metrics and health checks from two separate processes](https://gitlab.com/groups/gitlab-org/-/epics/6409) +to improve stability and availability and prevent data loss in edge cases. +As those are two separate servers, a configuration change will be required in 15.0 +to explicitly set separate ports for metrics and health-checks. +The newly introduced settings for `sidekiq['health_checks_*']` +should always be set in `gitlab.rb`. +For more information, check the documentation for [configuring Sidekiq](https://docs.gitlab.com/ee/administration/sidekiq/index.html). -To simplify setting a test coverage pattern, in GitLab 15.0 the -[project setting for test coverage parsing](https://docs.gitlab.com/ee/ci/pipelines/settings.html#add-test-coverage-results-using-project-settings-removed) -is being removed. +These changes also require updates in either Prometheus to scrape the new endpoint or k8s health-checks to target the new +health-check port to work properly, otherwise either metrics or health-checks will disappear. -Instead, using the project’s `.gitlab-ci.yml`, provide a regular expression with the `coverage` keyword to set -testing coverage results in merge requests. +For the deprecation period those settings are optional +and GitLab will default the Sidekiq health-checks port to the same port as `sidekiq_exporter` +and only run one server (not changing the current behaviour). +Only if they are both set and a different port is provided, a separate metrics server will spin up +to serve the Sidekiq metrics, similar to the way Sidekiq will behave in 15.0. </div> -<div class="deprecation removal-150 breaking-change"> - -### Vulnerability Check - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation " data-milestone="15.0"> -The vulnerability check feature is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. We encourage you to migrate to the new security approvals feature instead. You can do so by navigating to **Security & Compliance > Policies** and creating a new Scan Result Policy. - -The new security approvals feature is similar to vulnerability check. For example, both can require approvals for MRs that contain security vulnerabilities. However, security approvals improve the previous experience in several ways: - -- Users can choose who is allowed to edit security approval rules. An independent security or compliance team can therefore manage rules in a way that prevents development project maintainers from modifying the rules. -- Multiple rules can be created and chained together to allow for filtering on different severity thresholds for each scanner type. -- A two-step approval process can be enforced for any desired changes to security approval rules. -- A single set of security policies can be applied to multiple development projects to allow for ease in maintaining a single, centralized ruleset. +### Static Site Editor +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> </div> -<div class="deprecation removal-160 breaking-change"> - -### `CI_BUILD_*` predefined variables - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The predefined CI/CD variables that start with `CI_BUILD_*` were deprecated in GitLab 9.0, and will be removed in GitLab 16.0. If you still use these variables, be sure to change to the replacement [predefined variables](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) which are functionally identical: +The Static Site Editor will no longer be available starting in GitLab 15.0. Improvements to the Markdown editing experience across GitLab will deliver smiliar benefit but with a wider reach. Incoming requests to the Static Site Editor will be redirected to the [Web IDE](https://docs.gitlab.com/ee/user/project/web_ide/index.html). -| Removed variable | Replacement variable | -| --------------------- |------------------------ | -| `CI_BUILD_BEFORE_SHA` | `CI_COMMIT_BEFORE_SHA` | -| `CI_BUILD_ID` | `CI_JOB_ID` | -| `CI_BUILD_MANUAL` | `CI_JOB_MANUAL` | -| `CI_BUILD_NAME` | `CI_JOB_NAME` | -| `CI_BUILD_REF` | `CI_COMMIT_SHA` | -| `CI_BUILD_REF_NAME` | `CI_COMMIT_REF_NAME` | -| `CI_BUILD_REF_SLUG` | `CI_COMMIT_REF_SLUG` | -| `CI_BUILD_REPO` | `CI_REPOSITORY_URL` | -| `CI_BUILD_STAGE` | `CI_JOB_STAGE` | -| `CI_BUILD_TAG` | `CI_COMMIT_TAG` | -| `CI_BUILD_TOKEN` | `CI_JOB_TOKEN` | -| `CI_BUILD_TRIGGERED` | `CI_PIPELINE_TRIGGERED` | +Current users of the Static Site Editor can view the [documentation](https://docs.gitlab.com/ee/user/project/web_ide/index.html) for more information, including how to remove the configuration files from existing projects. </div> -<div class="deprecation removal-150 breaking-change"> - -### `projectFingerprint` in `PipelineSecurityReportFinding` GraphQL +<div class="deprecation breaking-change" data-milestone="15.0"> -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The `projectFingerprint` field in the [PipelineSecurityReportFinding](https://docs.gitlab.com/ee/api/graphql/reference/index.html#pipelinesecurityreportfinding) -GraphQL object is being deprecated. This field contains a "fingerprint" of security findings used to determine uniqueness. -The method for calculating fingerprints has changed, resulting in different values. Going forward, the new values will be -exposed in the UUID field. Data previously available in the projectFingerprint field will eventually be removed entirely. +### Support for SLES 12 SP2 +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### `started` iterations API field - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The `started` field in the [iterations API](https://docs.gitlab.com/ee/api/iterations.html#list-project-iterations) is being deprecated and will be removed in GitLab 15.0. This field is being replaced with the `current` field (already available) which aligns with the naming for other time-based entities, such as milestones. +Long term service and support (LTSS) for SUSE Linux Enterprise Server (SLES) 12 SP2 [ended on March 31, 2021](https://www.suse.com/lifecycle/). The CA certificates on SP2 include the expired DST root certificate, and it's not getting new CA certificate package updates. We have implemented some [workarounds](https://gitlab.com/gitlab-org/gitlab-omnibus-builder/-/merge_requests/191), but we will not be able to continue to keep the build running properly. </div> -</div> - -<div class="announcement-milestone"> -## Announced in 14.7 +<div class="deprecation breaking-change" data-milestone="15.0"> -<div class="deprecation removal-150"> - -### Container scanning schemas below 14.0.0 - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -[Container scanning report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) -versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation -against the schema version declared in the report will also no longer be supported in GitLab 15.0. - -Third-party tools that [integrate with GitLab by outputting a container scanning security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) -as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate against the declared schema version will not be processed, and vulnerability findings will not display in MRs, pipelines, or Vulnerability Reports. - -To help with the transition, from GitLab 14.10, non-compliant reports will display a -[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) -in the Vulnerability Report. +### Support for gRPC-aware proxy deployed between Gitaly and rest of GitLab +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150"> - -### Coverage guided fuzzing schemas below 14.0.0 - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -[Coverage guided fuzzing report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) -below version 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation -against the schema version declared in the report will also no longer be supported in GitLab 15.0. +Although not recommended or documented, it was possible to deploy a gRPC-aware proxy between Gitaly and +the rest of GitLab. For example, NGINX and Envoy. The ability to deploy a gRPC-aware proxy is +[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352517). If you currently use a gRPC-aware proxy for +Gitaly connections, you should change your proxy configuration to use TCP or TLS proxying (OSI layer 4) instead. -Third-party tools that [integrate with GitLab by outputting a coverage guided fuzzing security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) -as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct -schema with a minimum version of 14.0.0. Any reports with a lower version or that fail to validate -against the declared schema version will not be processed, and vulnerability -findings will not display in MRs, pipelines, or Vulnerability Reports. +Gitaly Cluster became incompatible with gRPC-aware proxies in GitLab 13.12. Now all GitLab installations will be incompatible with +gRPC-aware proxies, even without Gitaly Cluster. -To help with the transition, from GitLab 14.10, non-compliant reports will display a -[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) -in the Vulnerability Report. +By sending some of our internal RPC traffic through a custom protocol (instead of gRPC) we +increase throughput and reduce Go garbage collection latency. For more information, see +the [relevant epic](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/463). </div> -<div class="deprecation removal-150"> - -### DAST schemas below 14.0.0 - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.0"> -[DAST report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) -versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation -against the schema version declared in the report will also no longer be supported as of GitLab 15.0. - -Third-party tools that [integrate with GitLab by outputting a DAST security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) -as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct -schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate -against the declared schema version will not be processed, and vulnerability -findings will not display in MRs, pipelines, or Vulnerability Reports. - -To help with the transition, from GitLab 14.10, non-compliant reports will cause a -[warning to be displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) -in the Vulnerability Report. +### Test coverage project CI/CD setting +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150"> - -### Dependency scanning schemas below 14.0.0 - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -[Dependency scanning report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) -versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation -against the schema version declared in the report will also no longer be supported as of GitLab 15.0. - -Third-party tools that [integrate with GitLab by outputting a Dependency scanning security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) -as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct -schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate -against the declared schema version will not be processed, and vulnerability -findings will not display in MRs, pipelines, or Vulnerability Reports. +To simplify setting a test coverage pattern, in GitLab 15.0 the +[project setting for test coverage parsing](https://docs.gitlab.com/ee/ci/pipelines/settings.html#add-test-coverage-results-using-project-settings-removed) +is being removed. -To help with the transition, from GitLab 14.10, non-compliant reports will cause a -[warning to be displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) -in the Vulnerability Report. +Instead, using the project’s `.gitlab-ci.yml`, provide a regular expression with the `coverage` keyword to set +testing coverage results in merge requests. </div> -<div class="deprecation removal-150"> - -### Enforced validation of security report schemas +<div class="deprecation breaking-change" data-milestone="15.0"> -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -[Security report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) -versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation -against the schema version declared in the report will also no longer be supported in GitLab 15.0. - -Security tools that [integrate with GitLab by outputting security reports](https://docs.gitlab.com/ee/development/integrations/secure.html#report) -as pipeline job artifacts are affected. You must ensure that all output reports adhere to the correct -schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate -against the declared schema version will not be processed, and vulnerability -findings will not display in MRs, pipelines, or Vulnerability Reports. - -To help with the transition, from GitLab 14.10, non-compliant reports will display a -[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) -in the Vulnerability Report. +### Tracing in GitLab +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150"> - -### Godep support in License Compliance - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -The Godep dependency manager for Golang was deprecated in 2020 by Go and -has been replaced with Go modules. -To reduce our maintenance cost we are deprecating License Compliance for Godep projects as of 14.7 -and will remove it in GitLab 15.0 +Tracing in GitLab is an integration with Jaeger, an open-source end-to-end distributed tracing system. GitLab users can navigate to their Jaeger instance to gain insight into the performance of a deployed application, tracking each function or microservice that handles a given request. Tracing in GitLab is deprecated in GitLab 14.7, and scheduled for removal in 15.0. To track work on a possible replacement, see the issue for [Opstrace integration with GitLab](https://gitlab.com/groups/gitlab-org/-/epics/6976). </div> -<div class="deprecation removal-150 breaking-change"> - -### Logging in GitLab - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The logging features in GitLab allow users to install the ELK stack (Elasticsearch, Logstash, and Kibana) to aggregate and manage application logs. Users can search for relevant logs in GitLab. However, since deprecating certificate-based integration with Kubernetes clusters and GitLab Managed Apps, we don't have a recommended solution for logging within GitLab. For more information, you can follow the issue for [integrating Opstrace with GitLab](https://gitlab.com/groups/gitlab-org/-/epics/6976). +### Update to the Container Registry group-level API +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-160 breaking-change"> - -### Monitor performance metrics through Prometheus - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +In milestone 15.0, support for the `tags` and `tags_count` parameters will be removed from the Container Registry API that [gets registry repositories from a group](../api/container_registry.md#within-a-group). -By displaying data stored in a Prometheus instance, GitLab allows users to view performance metrics. GitLab also displays visualizations of these metrics in dashboards. The user can connect to a previously-configured external Prometheus instance, or set up Prometheus as a GitLab Managed App. -However, since certificate-based integration with Kubernetes clusters is deprecated in GitLab, the metrics functionality in GitLab that relies on Prometheus is also deprecated. This includes the metrics visualizations in dashboards. GitLab is working to develop a single user experience based on [Opstrace](https://about.gitlab.com/press/releases/2021-12-14-gitlab-acquires-opstrace-to-expand-its-devops-platform-with-open-source-observability-solution.html). An [issue exists](https://gitlab.com/groups/gitlab-org/-/epics/6976) for you to follow work on the Opstrace integration. +The `GET /groups/:id/registry/repositories` endpoint will remain, but won't return any info about tags. To get the info about tags, you can use the existing `GET /registry/repositories/:id` endpoint, which will continue to support the `tags` and `tag_count` options as it does today. The latter must be called once per image repository. </div> -<div class="deprecation removal-150"> - -### Pseudonymizer - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.0"> -The Pseudonymizer feature is generally unused, -can cause production issues with large databases, -and can interfere with object storage development. -It is now considered deprecated, and will be removed in GitLab 15.0. +### Value Stream Analytics filtering calculation change +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150"> - -### SAST schemas below 14.0.0 - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -[SAST report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) -versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation -against the schema version declared in the report will also no longer be supported as of GitLab 15.0. - -Third-party tools that [integrate with GitLab by outputting a SAST security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) -as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct -schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate -against the declared schema version will not be processed, and vulnerability -findings will not display in MRs, pipelines, or Vulnerability Reports. +We are changing how the date filter works in Value Stream Analytics. Instead of filtering by the time that the issue or merge request was created, the date filter will filter by the end event time of the given stage. This will result in completely different figures after this change has rolled out. -To help with the transition, from GitLab 14.10, non-compliant reports will display a -[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) -in the Vulnerability Report. +If you monitor Value Stream Analytics metrics and rely on the date filter, to avoid losing data, you must save the data prior to this change. </div> -<div class="deprecation removal-150"> +<div class="deprecation breaking-change" data-milestone="15.0"> -### Secret detection schemas below 14.0.0 - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -[Secret detection report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) -versions earlier than 14.0.0 will no longer be supported in GitLab 15.0. Reports that do not pass validation -against the schema version declared in the report will also no longer be supported as of GitLab 15.0. - -Third-party tools that [integrate with GitLab by outputting a Secret detection security report](https://docs.gitlab.com/ee/development/integrations/secure.html#report) -as a pipeline job artifact are affected. You must ensure that all output reports adhere to the correct -schema with a minimum version of 14.0.0. Reports with a lower version or that fail to validate -against the declared schema version will not be processed, and vulnerability -findings will not display in MRs, pipelines, or Vulnerability Reports. - -To help with the transition, from GitLab 14.10, non-compliant reports will display a -[warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) -in the Vulnerability Report. +### Vulnerability Check +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### Sidekiq metrics and health checks configuration - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -Exporting Sidekiq metrics and health checks using a single process and port is deprecated. -Support will be removed in 15.0. - -We have updated Sidekiq to export [metrics and health checks from two separate processes](https://gitlab.com/groups/gitlab-org/-/epics/6409) -to improve stability and availability and prevent data loss in edge cases. -As those are two separate servers, a configuration change will be required in 15.0 -to explicitly set separate ports for metrics and health-checks. -The newly introduced settings for `sidekiq['health_checks_*']` -should always be set in `gitlab.rb`. -For more information, check the documentation for [configuring Sidekiq](https://docs.gitlab.com/ee/administration/sidekiq.html). +The vulnerability check feature is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. We encourage you to migrate to the new security approvals feature instead. You can do so by navigating to **Security & Compliance > Policies** and creating a new Scan Result Policy. -These changes also require updates in either Prometheus to scrape the new endpoint or k8s health-checks to target the new -health-check port to work properly, otherwise either metrics or health-checks will disappear. +The new security approvals feature is similar to vulnerability check. For example, both can require approvals for MRs that contain security vulnerabilities. However, security approvals improve the previous experience in several ways: -For the deprecation period those settings are optional -and GitLab will default the Sidekiq health-checks port to the same port as `sidekiq_exporter` -and only run one server (not changing the current behaviour). -Only if they are both set and a different port is provided, a separate metrics server will spin up -to serve the Sidekiq metrics, similar to the way Sidekiq will behave in 15.0. +- Users can choose who is allowed to edit security approval rules. An independent security or compliance team can therefore manage rules in a way that prevents development project maintainers from modifying the rules. +- Multiple rules can be created and chained together to allow for filtering on different severity thresholds for each scanner type. +- A two-step approval process can be enforced for any desired changes to security approval rules. +- A single set of security policies can be applied to multiple development projects to allow for ease in maintaining a single, centralized ruleset. </div> -<div class="deprecation removal-150"> - -### Static Site Editor - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.0"> -The Static Site Editor will no longer be available starting in GitLab 15.0. Improvements to the Markdown editing experience across GitLab will deliver smiliar benefit but with a wider reach. Incoming requests to the Static Site Editor will be redirected to the [Web IDE](https://docs.gitlab.com/ee/user/project/web_ide/index.html). - -Current users of the Static Site Editor can view the [documentation](https://docs.gitlab.com/ee/user/project/web_ide/index.html) for more information, including how to remove the configuration files from existing projects. +### `Versions` on base `PackageType` +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### Tracing in GitLab - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +As part of the work to create a [Package Registry GraphQL API](https://gitlab.com/groups/gitlab-org/-/epics/6318), the Package group deprecated the `Version` type for the basic `PackageType` type and moved it to [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/index.html#packagedetailstype). -Tracing in GitLab is an integration with Jaeger, an open-source end-to-end distributed tracing system. GitLab users can navigate to their Jaeger instance to gain insight into the performance of a deployed application, tracking each function or microservice that handles a given request. Tracing in GitLab is deprecated in GitLab 14.7, and scheduled for removal in 15.0. To track work on a possible replacement, see the issue for [Opstrace integration with GitLab](https://gitlab.com/groups/gitlab-org/-/epics/6976). +In milestone 15.0, we will completely remove `Version` from `PackageType`. </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> ### `artifacts:reports:cobertura` keyword -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.7</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> Currently, test coverage visualizations in GitLab only support Cobertura reports. Starting 15.0, the `artifacts:reports:cobertura` keyword will be replaced by @@ -3176,511 +3396,273 @@ Currently, test coverage visualizations in GitLab only support Cobertura reports only supported report file in 15.0, but this is the first step towards GitLab supporting other report types. </div> -</div> - -<div class="announcement-milestone"> -## Announced in 14.6 +<div class="deprecation breaking-change" data-milestone="15.0"> -<div class="deprecation removal-150 breaking-change"> - -### CI/CD job name length limit - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -In GitLab 15.0 we are going to limit the number of characters in CI/CD job names to 255. Any pipeline with job names that exceed the 255 character limit will stop working after the 15.0 release. +### `defaultMergeCommitMessageWithDescription` GraphQL API field +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### Legacy approval status names from License Compliance API - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -We deprecated legacy names for approval status of license policy (blacklisted, approved) in the `managed_licenses` API but they are still used in our API queries and responses. They will be removed in 15.0. - -If you are using our License Compliance API you should stop using the `approved` and `blacklisted` query parameters, they are now `allowed` and `denied`. In 15.0 the responses will also stop using `approved` and `blacklisted` so you need to adjust any of your custom tools to use the old and new values so they do not break with the 15.0 release. +The GraphQL API field `defaultMergeCommitMessageWithDescription` has been deprecated and will be removed in GitLab 15.0. For projects with a commit message template set, it will ignore the template. </div> -<div class="deprecation removal-150 breaking-change"> - -### `type` and `types` keyword in CI/CD configuration - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="15.0"> -The `type` and `types` CI/CD keywords will be removed in GitLab 15.0. Pipelines that use these keywords will stop working, so you must switch to `stage` and `stages`, which have the same behavior. +### `dependency_proxy_for_private_groups` feature flag +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### apiFuzzingCiConfigurationCreate GraphQL mutation - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +We added a feature flag because [GitLab-#11582](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) changed how public groups use the Dependency Proxy. Prior to this change, you could use the Dependency Proxy without authentication. The change requires authentication to use the Dependency Proxy. -The API Fuzzing configuration snippet is now being generated client-side and does not require an -API request anymore. We are therefore deprecating the `apiFuzzingCiConfigurationCreate` mutation -which isn't being used in GitLab anymore. +In milestone 15.0, we will remove the feature flag entirely. Moving forward, you must authenticate when using the Dependency Proxy. </div> -<div class="deprecation removal-150 breaking-change"> - -### bundler-audit Dependency Scanning tool - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +<div class="deprecation breaking-change" data-milestone="15.0"> -As of 14.6 bundler-audit is being deprecated from Dependency Scanning. It will continue to be in our CI/CD template while deprecated. We are removing bundler-audit from Dependency Scanning on May 22, 2022 in 15.0. After this removal Ruby scanning functionality will not be affected as it is still being covered by Gemnasium. - -If you have explicitly excluded bundler-audit using DS_EXCLUDED_ANALYZERS you will need to clean up (remove the reference) in 15.0. If you have customized your pipeline's Dependency Scanning configuration, for example to edit the `bundler-audit-dependency_scanning` job, you will want to switch to gemnasium-dependency_scanning before removal in 15.0, to prevent your pipeline from failing. If you have not used the DS_EXCLUDED_ANALYZERS to reference bundler-audit, or customized your template specifically for bundler-audit, you will not need to take action. +### `pipelines` field from the `version` field +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -</div> - -<div class="announcement-milestone"> - -## Announced in 14.5 -<div class="deprecation removal-150 breaking-change"> - -### Changing an instance (shared) runner to a project (specific) runner - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -In GitLab 15.0, you can no longer change an instance (shared) runner to a project (specific) runner. +In GraphQL, there are two `pipelines` fields that you can use in a [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/#packagedetailstype) to get the pipelines for package versions: -Users often accidentally change instance runners to project runners, and they're unable to change them back. GitLab does not allow you to change a project runner to a shared runner because of the security implications. A runner meant for one project could be set to run jobs for an entire instance. +- The `versions` field's `pipelines` field. This returns all the pipelines associated with all the package's versions, which can pull an unbounded number of objects in memory and create performance concerns. +- The `pipelines` field of a specific `version`. This returns only the pipelines associated with that single package version. -Administrators who need to add runners for multiple projects can register a runner for one project, then go to the Admin view and choose additional projects. +To mitigate possible performance problems, we will remove the `versions` field's `pipelines` field in milestone 15.0. Although you will no longer be able to get all pipelines for all versions of a package, you can still get the pipelines of a single version through the remaining `pipelines` field for that version. </div> -<div class="deprecation removal-160 breaking-change"> - -### GraphQL API Runner status will not return `paused` - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The GitLab Runner GraphQL API endpoints will not return `paused` or `active` as a status in GitLab 16.0. -In a future v5 of the REST API, the endpoints for GitLab Runner will also not return `paused` or `active`. - -A runner's status will only relate to runner contact status, such as: -`online`, `offline`, or `not_connected`. Status `paused` or `active` will no longer appear. - -When checking if a runner is `paused`, API users are advised to check the boolean attribute -`paused` to be `true` instead. When checking if a runner is `active`, check if `paused` is `false`. +### `projectFingerprint` in `PipelineSecurityReportFinding` GraphQL +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### Known host required for GitLab Runner SSH executor - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -In [GitLab 14.3](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3074), we added a configuration setting in the GitLab Runner `config.toml` file. This setting, [`[runners.ssh.disable_strict_host_key_checking]`](https://docs.gitlab.com/runner/executors/ssh.html#security), controls whether or not to use strict host key checking with the SSH executor. - -In GitLab 15.0 and later, the default value for this configuration option will change from `true` to `false`. This means that strict host key checking will be enforced when using the GitLab Runner SSH executor. +The `projectFingerprint` field in the [PipelineSecurityReportFinding](https://docs.gitlab.com/ee/api/graphql/reference/index.html#pipelinesecurityreportfinding) +GraphQL object is being deprecated. This field contains a "fingerprint" of security findings used to determine uniqueness. +The method for calculating fingerprints has changed, resulting in different values. Going forward, the new values will be +exposed in the UUID field. Data previously available in the projectFingerprint field will eventually be removed entirely. </div> -<div class="deprecation removal-160 breaking-change"> - -### Package pipelines in API payload is paginated - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -A request to the API for `/api/v4/projects/:id/packages` returns a paginated result of packages. Each package lists all of its pipelines in this response. This is a performance concern, as it's possible for a package to have hundreds or thousands of associated pipelines. +<div class="deprecation breaking-change" data-milestone="15.0"> -In milestone 16.0, we will remove the `pipelines` attribute from the API response. +### `promote-db` command from `gitlab-ctl` +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-159 breaking-change"> - -### SaaS certificate-based integration with Kubernetes - -Planned removal: GitLab <span class="removal-milestone">15.9</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The certificate-based integration with Kubernetes will be [deprecated and removed](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/). As a GitLab SaaS customer, on new namespaces, you will no longer be able to integrate GitLab and your cluster using the certificate-based approach as of GitLab 15.0. The integration for current users will be enabled per namespace. - -For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the -[agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) - -For updates and details about this deprecation, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). - -GitLab self-managed customers can still use the feature [with a feature flag](https://docs.gitlab.com/ee/update/deprecations.html#self-managed-certificate-based-integration-with-kubernetes). +In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-db` which is used to promote database nodes in multi-node Geo secondary sites. `gitlab-ctl promote-db` will continue to function as-is and be available until GitLab 15.0. We recommend that Geo customers begin testing the new `gitlab-ctl promote` command in their staging environments and incorporating the new command in their failover procedures. </div> -<div class="deprecation removal-170 breaking-change"> - -### Self-managed certificate-based integration with Kubernetes - -Planned removal: GitLab <span class="removal-milestone">17.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The certificate-based integration with Kubernetes [will be deprecated and removed](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/). - -As a self-managed customer, we are introducing the [feature flag](../administration/feature_flags.md#enable-or-disable-the-feature) `certificate_based_clusters` in GitLab 15.0 so you can keep your certificate-based integration enabled. However, the feature flag will be disabled by default, so this change is a **breaking change**. - -In GitLab 17.0 we will remove both the feature and its related code. Until the final removal in 17.0, features built on this integration will continue to work, if you enable the feature flag. Until the feature is removed, GitLab will continue to fix security and critical issues as they arise. - -For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the -[agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) +<div class="deprecation breaking-change" data-milestone="15.0"> -For updates and details about this deprecation, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). +### `promote-to-primary-node` command from `gitlab-ctl` +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### Support for SLES 12 SP2 - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -Long term service and support (LTSS) for SUSE Linux Enterprise Server (SLES) 12 SP2 [ended on March 31, 2021](https://www.suse.com/lifecycle/). The CA certificates on SP2 include the expired DST root certificate, and it's not getting new CA certificate package updates. We have implemented some [workarounds](https://gitlab.com/gitlab-org/gitlab-omnibus-builder/-/merge_requests/191), but we will not be able to continue to keep the build running properly. +In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-to-primary-node` which was only usable for single-node Geo sites. `gitlab-ctl promote-to-primary-node` will continue to function as-is and be available until GitLab 15.0. We recommend that Geo customers begin testing the new `gitlab-ctl promote` command in their staging environments and incorporating the new command in their failover procedures. </div> -<div class="deprecation removal-150 breaking-change"> - -### Update to the Container Registry group-level API - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -In milestone 15.0, support for the `tags` and `tags_count` parameters will be removed from the Container Registry API that [gets registry repositories from a group](../api/container_registry.md#within-a-group). - -The `GET /groups/:id/registry/repositories` endpoint will remain, but won't return any info about tags. To get the info about tags, you can use the existing `GET /registry/repositories/:id` endpoint, which will continue to support the `tags` and `tag_count` options as it does today. The latter must be called once per image repository. +### `started` iterations API field +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### Value Stream Analytics filtering calculation change - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -We are changing how the date filter works in Value Stream Analytics. Instead of filtering by the time that the issue or merge request was created, the date filter will filter by the end event time of the given stage. This will result in completely different figures after this change has rolled out. - -If you monitor Value Stream Analytics metrics and rely on the date filter, to avoid losing data, you must save the data prior to this change. +The `started` field in the [iterations API](https://docs.gitlab.com/ee/api/iterations.html#list-project-iterations) is being deprecated and will be removed in GitLab 15.0. This field is being replaced with the `current` field (already available) which aligns with the naming for other time-based entities, such as milestones. </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> -### `Versions` on base `PackageType` - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -As part of the work to create a [Package Registry GraphQL API](https://gitlab.com/groups/gitlab-org/-/epics/6318), the Package group deprecated the `Version` type for the basic `PackageType` type and moved it to [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/index.html#packagedetailstype). - -In milestone 15.0, we will completely remove `Version` from `PackageType`. +### `type` and `types` keyword in CI/CD configuration +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.6</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### `defaultMergeCommitMessageWithDescription` GraphQL API field - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The GraphQL API field `defaultMergeCommitMessageWithDescription` has been deprecated and will be removed in GitLab 15.0. For projects with a commit message template set, it will ignore the template. +The `type` and `types` CI/CD keywords will be removed in GitLab 15.0. Pipelines that use these keywords will stop working, so you must switch to `stage` and `stages`, which have the same behavior. </div> -<div class="deprecation removal-150 breaking-change"> - -### `dependency_proxy_for_private_groups` feature flag - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="15.0"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -We added a feature flag because [GitLab-#11582](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) changed how public groups use the Dependency Proxy. Prior to this change, you could use the Dependency Proxy without authentication. The change requires authentication to use the Dependency Proxy. - -In milestone 15.0, we will remove the feature flag entirely. Moving forward, you must authenticate when using the Dependency Proxy. +### apiFuzzingCiConfigurationCreate GraphQL mutation +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.6</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### `pipelines` field from the `version` field - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -In GraphQL, there are two `pipelines` fields that you can use in a [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/#packagedetailstype) to get the pipelines for package versions: - -- The `versions` field's `pipelines` field. This returns all the pipelines associated with all the package's versions, which can pull an unbounded number of objects in memory and create performance concerns. -- The `pipelines` field of a specific `version`. This returns only the pipelines associated with that single package version. - -To mitigate possible performance problems, we will remove the `versions` field's `pipelines` field in milestone 15.0. Although you will no longer be able to get all pipelines for all versions of a package, you can still get the pipelines of a single version through the remaining `pipelines` field for that version. +The API Fuzzing configuration snippet is now being generated client-side and does not require an +API request anymore. We are therefore deprecating the `apiFuzzingCiConfigurationCreate` mutation +which isn't being used in GitLab anymore. </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> -### `promote-db` command from `gitlab-ctl` +### bundler-audit Dependency Scanning tool -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.6</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +As of 14.6 bundler-audit is being deprecated from Dependency Scanning. It will continue to be in our CI/CD template while deprecated. We are removing bundler-audit from Dependency Scanning on May 22, 2022 in 15.0. After this removal Ruby scanning functionality will not be affected as it is still being covered by Gemnasium. -In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-db` which is used to promote database nodes in multi-node Geo secondary sites. `gitlab-ctl promote-db` will continue to function as-is and be available until GitLab 15.0. We recommend that Geo customers begin testing the new `gitlab-ctl promote` command in their staging environments and incorporating the new command in their failover procedures. +If you have explicitly excluded bundler-audit using DS_EXCLUDED_ANALYZERS you will need to clean up (remove the reference) in 15.0. If you have customized your pipeline's Dependency Scanning configuration, for example to edit the `bundler-audit-dependency_scanning` job, you will want to switch to gemnasium-dependency_scanning before removal in 15.0, to prevent your pipeline from failing. If you have not used the DS_EXCLUDED_ANALYZERS to reference bundler-audit, or customized your template specifically for bundler-audit, you will not need to take action. </div> -<div class="deprecation removal-150 breaking-change"> +<div class="deprecation breaking-change" data-milestone="15.0"> -### `promote-to-primary-node` command from `gitlab-ctl` +### htpasswd Authentication for the Container Registry -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The Container Registry supports [authentication](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs/configuration.md#auth) with `htpasswd`. It relies on an [Apache `htpasswd` file](https://httpd.apache.org/docs/2.4/programs/htpasswd.html), with passwords hashed using `bcrypt`. -In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-to-primary-node` which was only usable for single-node Geo sites. `gitlab-ctl promote-to-primary-node` will continue to function as-is and be available until GitLab 15.0. We recommend that Geo customers begin testing the new `gitlab-ctl promote` command in their staging environments and incorporating the new command in their failover procedures. +Since it isn't used in the context of GitLab (the product), `htpasswd` authentication will be deprecated in GitLab 14.9 and removed in GitLab 15.0. </div> -<div class="deprecation removal-148"> +<div class="deprecation breaking-change" data-milestone="15.0"> -### openSUSE Leap 15.2 packages +### user_email_lookup_limit API field -Planned removal: GitLab <span class="removal-milestone">14.8</span> <span class="removal-date"></span> +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) +</div> -Distribution support and security updates for openSUSE Leap 15.2 are [ending December 2021](https://en.opensuse.org/Lifetime#openSUSE_Leap). +The `user_email_lookup_limit` [API field](https://docs.gitlab.com/ee/api/settings.html) is deprecated and will be removed in GitLab 15.0. Until GitLab 15.0, `user_email_lookup_limit` is aliased to `search_rate_limit` and existing workflows will continue to work. -Starting in 14.5 we are providing packages for openSUSE Leap 15.3, and will stop providing packages for openSUSE Leap 15.2 in the 14.8 milestone. +Any API calls attempting to change the rate limits for `user_email_lookup_limit` should use `search_rate_limit` instead. </div> </div> -<div class="announcement-milestone"> - -## Announced in 14.3 +<div class="milestone-wrapper" data-milestone="14.10"> -<div class="deprecation removal-150 breaking-change"> - -### Audit events for repository push events +## GitLab 14.10 -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation breaking-change" data-milestone="14.10"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -Audit events for [repository events](https://docs.gitlab.com/ee/administration/audit_events.html#removed-events) are now deprecated and will be removed in GitLab 15.0. - -These events have always been disabled by default and had to be manually enabled with a -feature flag. Enabling them can cause too many events to be generated which can -dramatically slow down GitLab instances. For this reason, they are being removed. +### Permissions change for downloading Composer dependencies +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.9</span> +- [Breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/) </div> -<div class="deprecation removal-150 breaking-change"> - -### GitLab Serverless - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -GitLab Serverless is a feature set to support Knative-based serverless development with automatic deployments and monitoring. +The GitLab Composer repository can be used to push, search, fetch metadata about, and download PHP dependencies. All these actions require authentication, except for downloading dependencies. -We decided to remove the GitLab Serverless features as they never really resonated with our users. Besides, given the continuous development of Kubernetes and Knative, our current implementations do not even work with recent versions. +Downloading Composer dependencies without authentication is deprecated in GitLab 14.9, and will be removed in GitLab 15.0. Starting with GitLab 15.0, you must authenticate to download Composer dependencies. </div> - -<div class="deprecation removal-150 breaking-change"> - -### Legacy database configuration - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -The syntax of [GitLabs database](https://docs.gitlab.com/omnibus/settings/database.html) -configuration located in `database.yml` is changing and the legacy format is deprecated. The legacy format -supported using a single PostgreSQL adapter, whereas the new format is changing to support multiple databases. The `main:` database needs to be defined as a first configuration item. - -This deprecation mainly impacts users compiling GitLab from source because Omnibus will handle this configuration automatically. - </div> -<div class="deprecation removal-150 breaking-change"> +<div class="milestone-wrapper" data-milestone="14.9"> -### OmniAuth Kerberos gem +## GitLab 14.9 -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> +<div class="deprecation " data-milestone="14.9"> -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +### Configurable Gitaly `per_repository` election strategy -The `omniauth-kerberos` gem will be removed in our next major release, GitLab 15.0. +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.8</span> +</div> -This gem has not been maintained and has very little usage. We therefore plan to remove support for this authentication method and recommend using the Kerberos [SPNEGO](https://en.wikipedia.org/wiki/SPNEGO) integration instead. You can follow the [upgrade instructions](https://docs.gitlab.com/ee/integration/kerberos.html#upgrading-from-password-based-to-ticket-based-kerberos-sign-ins) to upgrade from the `omniauth-kerberos` integration to the supported one. +Configuring the `per_repository` Gitaly election strategy is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352612). +`per_repository` has been the only option since GitLab 14.0. -Note that we are not deprecating the Kerberos SPNEGO integration, only the old password-based Kerberos integration. +This change is part of regular maintenance to keep our codebase clean. </div> </div> -<div class="announcement-milestone"> +<div class="milestone-wrapper" data-milestone="14.8"> -## Announced in 14.2 +## GitLab 14.8 -<div class="deprecation removal-146"> +<div class="deprecation " data-milestone="14.8"> -### Release CLI distributed as a generic package - -Planned removal: GitLab <span class="removal-milestone">14.6</span> <span class="removal-date"></span> - -The [release-cli](https://gitlab.com/gitlab-org/release-cli) will be released as a [generic package](https://gitlab.com/gitlab-org/release-cli/-/packages) starting in GitLab 14.2. We will continue to deploy it as a binary to S3 until GitLab 14.5 and stop distributing it in S3 in GitLab 14.6. +### openSUSE Leap 15.2 packages +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.5</span> </div> -<div class="deprecation removal-145"> - -### Rename Task Runner pod to Toolbox - -Planned removal: GitLab <span class="removal-milestone">14.5</span> <span class="removal-date"></span> - -The Task Runner pod is used to execute periodic housekeeping tasks within the GitLab application and is often confused with the GitLab Runner. Thus, [Task Runner will be renamed to Toolbox](https://gitlab.com/groups/gitlab-org/charts/-/epics/25). +Distribution support and security updates for openSUSE Leap 15.2 are [ending December 2021](https://en.opensuse.org/Lifetime#openSUSE_Leap). -This will result in the rename of the sub-chart: `gitlab/task-runner` to `gitlab/toolbox`. Resulting pods will be named along the lines of `{{ .Release.Name }}-toolbox`, which will often be `gitlab-toolbox`. They will be locatable with the label `app=toolbox`. +Starting in 14.5 we are providing packages for openSUSE Leap 15.3, and will stop providing packages for openSUSE Leap 15.2 in the 14.8 milestone. </div> </div> -<div class="announcement-milestone"> +<div class="milestone-wrapper" data-milestone="14.6"> -## Announced in 14.0 +## GitLab 14.6 -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation " data-milestone="14.6"> -### Changing merge request approvals with the `/approvals` API endpoint - -Planned removal: GitLab <span class="removal-milestone">16.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -To change the approvals required for a merge request, you should no longer use the `/approvals` API endpoint, which was deprecated in GitLab 14.0. - -Instead, use the [`/approval_rules` endpoint](https://docs.gitlab.com/ee/api/merge_request_approvals.html#merge-request-level-mr-approvals) to [create](https://docs.gitlab.com/ee/api/merge_request_approvals.html#create-merge-request-level-rule) or [update](https://docs.gitlab.com/ee/api/merge_request_approvals.html#update-merge-request-level-rule) the approval rules for a merge request. +### Release CLI distributed as a generic package +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.2</span> </div> -<div class="deprecation removal-156"> - -### NFS for Git repository storage +The [release-cli](https://gitlab.com/gitlab-org/release-cli) will be released as a [generic package](https://gitlab.com/gitlab-org/release-cli/-/packages) starting in GitLab 14.2. We will continue to deploy it as a binary to S3 until GitLab 14.5 and stop distributing it in S3 in GitLab 14.6. -Planned removal: GitLab <span class="removal-milestone">15.6</span> <span class="removal-date"></span> +</div> +</div> -With the general availability of Gitaly Cluster ([introduced in GitLab 13.0](https://about.gitlab.com/releases/2020/05/22/gitlab-13-0-released/)), we have deprecated development (bugfixes, performance improvements, etc) for NFS for Git repository storage in GitLab 14.0. We will continue to provide technical support for NFS for Git repositories throughout 14.x, but we will remove all support for NFS on November 22, 2022. This was originally planned for May 22, 2022, but in an effort to allow continued maturity of Gitaly Cluster, we have chosen to extend our deprecation of support date. Please see our official [Statement of Support](https://about.gitlab.com/support/statement-of-support/#gitaly-and-nfs) for further information. +<div class="milestone-wrapper" data-milestone="14.5"> -Gitaly Cluster offers tremendous benefits for our customers such as: +## GitLab 14.5 -- [Variable replication factors](https://docs.gitlab.com/ee/administration/gitaly/index.html#replication-factor). -- [Strong consistency](https://docs.gitlab.com/ee/administration/gitaly/index.html#strong-consistency). -- [Distributed read capabilities](https://docs.gitlab.com/ee/administration/gitaly/index.html#distributed-reads). +<div class="deprecation " data-milestone="14.5"> -We encourage customers currently using NFS for Git repositories to plan their migration by reviewing our documentation on [migrating to Gitaly Cluster](https://docs.gitlab.com/ee/administration/gitaly/index.html#migrate-to-gitaly-cluster). +### Rename Task Runner pod to Toolbox +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">14.2</span> </div> -<div class="deprecation removal-150 breaking-change"> - -### OAuth implicit grant - -Planned removal: GitLab <span class="removal-milestone">15.0</span> <span class="removal-date"></span> - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. +The Task Runner pod is used to execute periodic housekeeping tasks within the GitLab application and is often confused with the GitLab Runner. Thus, [Task Runner will be renamed to Toolbox](https://gitlab.com/groups/gitlab-org/charts/-/epics/25). -The OAuth implicit grant authorization flow will be removed in our next major release, GitLab 15.0. Any applications that use OAuth implicit grant should switch to alternative [supported OAuth flows](https://docs.gitlab.com/ee/api/oauth2.html). +This will result in the rename of the sub-chart: `gitlab/task-runner` to `gitlab/toolbox`. Resulting pods will be named along the lines of `{{ .Release.Name }}-toolbox`, which will often be `gitlab-toolbox`. They will be locatable with the label `app=toolbox`. </div> </div> diff --git a/doc/update/index.md b/doc/update/index.md index 07bd153907a..c8fa091c657 100644 --- a/doc/update/index.md +++ b/doc/update/index.md @@ -264,6 +264,12 @@ NOTE: Specific information that follow related to Ruby and Git versions do not apply to [Omnibus installations](https://docs.gitlab.com/omnibus/) and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with appropriate Ruby and Git versions and are not using system binaries for Ruby and Git. There is no need to install Ruby or Git when utilizing these two approaches. +### 16.0.0 + +- Sidekiq jobs are only routed to `default` and `mailers` queues by default, and as a result, + every Sidekiq process also listens to those queues to ensure all jobs are processed across + all queues. This behavior does not apply if you have configured the [routing rules](../administration/sidekiq/processing_specific_job_classes.md#routing-rules). + ### 15.10.0 - Gitaly configuration changes significantly in Omnibus GitLab 16.0. You can begin migrating to the new structure in Omnibus GitLab 15.10 while backwards compatibility is @@ -289,6 +295,11 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap You can find repositories with invalid metadata records prior in GitLab 15.0 and later by searching for the log records outputted by the verifier. [Read more about repository verification, and to see an example log entry](../administration/gitaly/praefect.md#repository-verification). - Praefect configuration changes significantly in Omnibus GitLab 16.0. You can begin migrating to the new structure in Omnibus GitLab 15.9 while backwards compatibility is maintained in the lead up to Omnibus GitLab 16.0. [Read more about this change](#praefect-omnibus-gitlab-configuration-structure-change). +- For **self-compiled (source) installations**, with the addition of `gitlab-sshd` the Kerberos headers are needed to build GitLab Shell. + + ```shell + sudo apt install libkrb5-dev + ``` ### 15.8.2 @@ -385,7 +396,7 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap consistent in our documentation and product defaults. For example, previously: - + - Omnibus GitLab default (`sidekiq['max_concurrency']`): 50 - From source installation default: 50 - Helm chart default (`gitlab.sidekiq.concurrency`): 25 @@ -606,6 +617,12 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap migration might take multiple hours or days to complete on larger GitLab instances. Make sure the migration has completed successfully before upgrading to 15.7.0 or later. - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information. +- A redesigned sign-in page is enabled by default in GitLab 15.4 and later, with improvements shipping in later releases. For more information, see [epic 8557](https://gitlab.com/groups/gitlab-org/-/epics/8557). + It can be disabled with a feature flag. Start [a Rails console](../administration/operations/rails_console.md) and run: + + ```ruby + Feature.disable(:restyle_login_page) + ``` ### 15.3.4 @@ -733,6 +750,36 @@ A [license caching issue](https://gitlab.com/gitlab-org/gitlab/-/issues/376706) - The `FF_GITLAB_REGISTRY_HELPER_IMAGE` [feature flag](../administration/feature_flags.md#enable-or-disable-the-feature) is removed and helper images are always pulled from GitLab Registry. - The `AES256-GCM-SHA384` SSL cipher is no longer allowed by NGINX. See how you can [add the cipher back](https://docs.gitlab.com/omnibus/update/gitlab_15_changes.html#aes256-gcm-sha384-ssl-cipher-no-longer-allowed-by-default-by-nginx) to the allow list. +- Support for more than one database has been added to GitLab. For **self-compiled (source) installations**, + `config/database.yml` must include a database name in the database configuration. + The `main: database` must be first. If an invalid or deprecated syntax is used, an error is generated + during application start: + + ```plaintext + ERROR: This installation of GitLab uses unsupported 'config/database.yml'. + The main: database needs to be defined as a first configuration item instead of primary. (RuntimeError) + ``` + + Previously, the `config/database.yml` file looked like the following: + + ```yaml + production: + adapter: postgresql + encoding: unicode + database: gitlabhq_production + ... + ``` + + Starting with GitLab 15.0, it must define a `main` database first: + + ```yaml + production: + main: + adapter: postgresql + encoding: unicode + database: gitlabhq_production + ... + ``` ### 14.10.0 @@ -925,6 +972,19 @@ or [init scripts](upgrading_from_source.md#configure-sysv-init-script) by [follo [There is a workaround to complete the data change and the upgrade manually](package/index.md#mixlibshelloutcommandtimeout-rails_migrationgitlab-rails--command-timed-out-after-3600s) +- As part of [enabling real-time issue assignees](https://gitlab.com/gitlab-org/gitlab/-/issues/330117), Action Cable is now enabled by default. + For **self-compiled (source) installations**, `config/cable.yml` is required to be present. + + Configure this by running: + + ```shell + cd /home/git/gitlab + sudo -u git -H cp config/cable.yml.example config/cable.yml + + # Change the Redis socket path if you are not using the default Debian / Ubuntu configuration + sudo -u git -H editor config/cable.yml + ``` + ### 14.4.4 - For [zero-downtime upgrades](zero_downtime.md) on a GitLab cluster with separate Web and API nodes, you must enable the `paginated_tree_graphql_query` [feature flag](../administration/feature_flags.md#enable-or-disable-the-feature) _before_ upgrading GitLab Web nodes to 14.4. @@ -1390,12 +1450,23 @@ all servers must first be upgraded to 13.1.Z before upgrading to 13.2.0 or later #### Custom Rack Attack initializers -From GitLab 13.0.1, custom Rack Attack initializers (`config/initializers/rack_attack.rb`) are replaced with initializers -supplied with GitLab during upgrades. We recommend you use these GitLab-supplied initializers. +From GitLab 13.1, custom Rack Attack initializers (`config/initializers/rack_attack.rb`) are replaced with initializers +supplied with GitLab during upgrades. You should use these GitLab-supplied initializers. If you persist your own Rack Attack initializers between upgrades, you might [get `500` errors](https://gitlab.com/gitlab-org/gitlab/-/issues/334681) when [upgrading to GitLab 14.0 and later](#1400). +For **self-compiled (source) installations**, the Rack Attack initializer on GitLab +was renamed from [`config/initializers/rack_attack_new.rb` to `config/initializers/rack_attack.rb`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33072). +The rename was part of [deprecating Rack Attack throttles on Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4750). + +If `rack_attack.rb` has been created on your installation, consider creating a backup before updating: + +```shell +cd /home/git/gitlab +cp config/initializers/rack_attack.rb config/initializers/rack_attack_backup.rb +``` + ### 12.10.0 - The final patch release (12.10.14) @@ -1558,19 +1629,24 @@ gitaly['configuration'] = { }, ], }, - 'gitaly-ruby': { - # gitaly['ruby_max_rss'] - max_rss: ..., - # gitaly['ruby_graceful_restart_timeout'] - graceful_restart_timeout: ..., - # gitaly['ruby_restart_delay'] - restart_delay: ..., - # gitaly['ruby_num_workers'] - num_workers: ..., - }, - # gitaly['storage']. While the structure is the same, the string keys in the array elements - # should be replaced by symbols as elsewhere. {'key' => 'value'}, should become {key: 'value'}. - storage: ..., + # Storage could previously be configured through either gitaly['storage'] or 'git_data_dirs'. Migrate + # the relevant configuration according to the instructions below. + storage: [ + { + # gitaly['storage'][<index>]['name'] + # + # git_data_dirs[<name>]. The storage name was configured as a key in the map. + name: ..., + # gitaly['storage'][<index>]['path'] + # + # git_data_dirs[<name>]['path']. Use the value from git_data_dirs[<name>]['path'] and append '/repositories' to it. + # + # For example, if the path in 'git_data_dirs' was '/var/opt/gitlab/git-data', use + # '/var/opt/gitlab/git-data/repositories'. The '/repositories' extension was automatically + # appended to the path configured in `git_data_dirs`. + path: ..., + }, + ], hooks: { # gitaly['custom_hooks_dir'] custom_hooks_dir: ..., diff --git a/doc/update/package/index.md b/doc/update/package/index.md index d3bd89975d2..07a294a4dde 100644 --- a/doc/update/package/index.md +++ b/doc/update/package/index.md @@ -294,7 +294,7 @@ To fix this issue: ### Error `Failed to connect to the internal GitLab API` on a separate GitLab Pages server -See [GitLab Pages troubleshooting](../../administration/pages/index.md#failed-to-connect-to-the-internal-gitlab-api). +See [GitLab Pages administration troubleshooting](../../administration/pages/troubleshooting.md#failed-to-connect-to-the-internal-gitlab-api). ### Error `An error occurred during the signature verification` when running `apt-get update` diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index 8e62718125b..b6530418f97 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -2,7 +2,6 @@ stage: Systems group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Universal update guide for patch versions of source installations **(FREE SELF)** diff --git a/doc/update/plan_your_upgrade.md b/doc/update/plan_your_upgrade.md index 0d0084cf530..e1354ea3665 100644 --- a/doc/update/plan_your_upgrade.md +++ b/doc/update/plan_your_upgrade.md @@ -60,16 +60,6 @@ to ensure the major components of GitLab are working: 1. If using Elasticsearch, verify that searches are successful. -1. If you are using [Reply by Email](../administration/reply_by_email.md) or [Service Desk](../user/project/service_desk.md), - manually install the latest version of `gitlab-mail_room`: - - ```shell - gem install gitlab-mail_room - ``` - - NOTE: This step is necessary to avoid thread deadlocks and to support the latest MailRoom features. See - [this explanation](../development/emails.md#mailroom-gem-updates) for more details. - If in any case something goes wrong, see [how to troubleshoot](#troubleshooting). ## Rollback plan @@ -173,7 +163,7 @@ If you have Kubernetes clusters connected with GitLab, [upgrade your GitLab agen #### Elasticsearch Before updating GitLab, confirm advanced search migrations are complete by -[checking for pending advanced search migrations](background_migrations.md). +[checking for pending advanced search migrations](index.md#checking-for-pending-advanced-search-migrations). After updating GitLab, you may have to upgrade [Elasticsearch if the new version breaks compatibility](../integration/advanced_search/elasticsearch.md#version-requirements). diff --git a/doc/update/removals.md b/doc/update/removals.md index 248d03cbbcb..a8c21cf5f1e 100644 --- a/doc/update/removals.md +++ b/doc/update/removals.md @@ -34,6 +34,25 @@ For removal reviewers (Technical Writers only): https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-removals-doc --> +## Removed in 15.11 + +### Exporting and importing projects in JSON format not supported + +GitLab previously created project file exports in JSON format. In GitLab 12.10, NDJSON was added as the preferred format. + +To support transitions, importing JSON-formatted project file exports was still possible if you configured the +relevant feature flags. + +From GitLab 15.11, importing a JSON-formatted project file exports is not supported. + +### openSUSE Leap 15.3 packages + +Distribution support and security updates for openSUSE Leap 15.3 [ended December 2022](https://en.opensuse.org/Lifetime#Discontinued_distributions). + +Starting in GitLab 15.7 we started providing packages for openSUSE Leap 15.4, and in GitLab 15.11 we stopped providing packages for openSUSE Leap 15.3. + +To continue using GitLab, [upgrade](https://en.opensuse.org/SDB:System_upgrade) to openSUSE Leap 15.4. + ## Removed in 15.9 ### Live Preview no longer available in the Web IDE @@ -44,6 +63,22 @@ Review the details carefully before upgrading. The Live Preview feature of the Web IDE was intended to provide a client-side preview of static web applications. However, complex configuration steps and a narrow set of supported project types have limited its utility. With the introduction of the Web IDE Beta in GitLab 15.7, you can now connect to a full server-side runtime environment. With upcoming support for installing extensions in the Web IDE, we’ll also support more advanced workflows than those available with Live Preview. As of GitLab 15.9, Live Preview is no longer available in the Web IDE. +### `omniauth-authentiq` gem no longer available + +WARNING: +This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +Review the details carefully before upgrading. + +`omniauth-authentiq` is an OmniAuth strategy gem that was part of GitLab. The company providing authentication services, Authentiq, has shut down. Therefore the gem is being removed. + +### `omniauth-shibboleth` gem no longer available + +WARNING: +This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +Review the details carefully before upgrading. + +`omniauth-shibboleth` is an OmniAuth strategy gem that was part of GitLab. The gem has not received security updates and does not meet GitLab quality guidance criteria. This gem was originally scheduled for removal by 14.1, but it was not removed at that time. The gem is being removed now. + ## Removed in 15.8 ### CiliumNetworkPolicy within the auto deploy Helm chart is removed @@ -55,6 +90,15 @@ If you want to preserve this functionality, you can follow one of these two path 1. Fork the [GitLab Auto Deploy Helm chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app) into the `chart/` path within your project 1. Set `AUTO_DEPLOY_IMAGE_VERSION` and `DAST_AUTO_DEPLOY_IMAGE_VERSION` to the most recent version of the image that included the CiliumNetworkPolicy +### Exporting and importing groups in JSON format not supported + +GitLab previously created group file exports in JSON format. In GitLab 13.10, NDJSON was added as the preferred format. + +To support transitions, importing JSON-formatted group file exports was still possible if you configured the +relevant feature flags. + +From GitLab 15.8, importing a JSON-formatted group file exports is not supported. + ### `artifacts:public` CI/CD keyword refactored WARNING: @@ -211,7 +255,7 @@ If your object storage provider does not support `background_upload`, please [mi #### Encrypted S3 buckets -Additionally, this also breaks the use of [encrypted S3 buckets](https://docs.gitlab.com/ee/administration/object_storage.html#encrypted-s3-buckets) with [storage-specific configuration form](https://docs.gitlab.com/ee/administration/object_storage.html#storage-specific-configuration). +Additionally, this also breaks the use of [encrypted S3 buckets](https://docs.gitlab.com/ee/administration/object_storage.html#encrypted-s3-buckets) with [storage-specific configuration form](https://docs.gitlab.com/ee/administration/object_storage.html#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form). If your S3 buckets have [SSE-S3 or SSE-KMS encryption enabled](https://docs.aws.amazon.com/kms/latest/developerguide/services-s3.html), please [migrate your configuration to use consolidated object storage form](https://docs.gitlab.com/ee/administration/object_storage.html#transition-to-consolidated-form) before upgrading to GitLab 15.0. Otherwise, you may start getting `ETag mismatch` errors during objects upload. @@ -229,7 +273,7 @@ Some users found that they could specify a path prefix to the bucket. In direct If you have set a prefix, you can use a workaround to revert to background uploads: -1. Continue to use [storage-specific configuration](https://docs.gitlab.com/ee/administration/object_storage.html#storage-specific-configuration). +1. Continue to use [storage-specific configuration](https://docs.gitlab.com/ee/administration/object_storage.html#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form). 1. In Omnibus GitLab, set the `GITLAB_LEGACY_BACKGROUND_UPLOADS` to re-enable background uploads: ```ruby diff --git a/doc/update/upgrading_from_ce_to_ee.md b/doc/update/upgrading_from_ce_to_ee.md index b99bb3d7992..8a66da507ec 100644 --- a/doc/update/upgrading_from_ce_to_ee.md +++ b/doc/update/upgrading_from_ce_to_ee.md @@ -2,7 +2,6 @@ stage: Systems group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Upgrading from Community Edition to Enterprise Edition from source **(FREE SELF)** diff --git a/doc/update/upgrading_from_source.md b/doc/update/upgrading_from_source.md index b6efaa7ec25..16bab1eb690 100644 --- a/doc/update/upgrading_from_source.md +++ b/doc/update/upgrading_from_source.md @@ -2,7 +2,6 @@ stage: Systems group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -comments: false --- # Upgrading Community Edition and Enterprise Edition from source **(FREE SELF)** @@ -59,21 +58,10 @@ sudo service gitlab stop ### 3. Update Ruby -From GitLab 15.10, we only support Ruby 3.0 or higher and dropped support for Ruby 2.7. Be sure to upgrade if necessary. +From GitLab 15.10, we only support Ruby 3.0.x and dropped support for Ruby 2.7. Be sure to upgrade if necessary. You can check which version you are running with `ruby -v`. -Download Ruby and compile it: - -```shell -mkdir /tmp/ruby && cd /tmp/ruby -curl --remote-name --location --progress-bar "https://cache.ruby-lang.org/pub/ruby/3.0/ruby-3.0.5.tar.gz" -echo '9afc6380a027a4fe1ae1a3e2eccb6b497b9c5ac0631c12ca56f9b7beb4848776 ruby-3.0.5.tar.gz' | sha256sum -c - && tar xzf ruby-3.0.5.tar.gz -cd ruby-2.7.6 - -./configure --disable-install-rdoc --enable-shared -make -sudo make install -``` +[Install Ruby](https://www.ruby-lang.org/en/documentation/installation/). ### 4. Update Node.js @@ -335,8 +323,8 @@ sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workh ``` NOTE: -If you get any errors concerning Rack attack, see the [13.0](#1301) specific -upgrade instructions. +If you get any errors concerning Rack attack, see the [13.1](index.md#custom-rack-attack-initializers) +specific changes. ### 13. Update Gitaly @@ -405,85 +393,12 @@ If all items are green, then congratulations, the upgrade is complete! This is an optional step. If you [installed the product documentation](../install/installation.md#install-the-product-documentation), see how to [upgrade to a later version](../administration/docs_self_host.md#upgrade-the-product-documentation-to-a-later-version). -## Version specific upgrading instructions - -This section contains upgrading instructions for specific versions. When -present, first follow the upgrading guidelines for all versions. If the version -you are upgrading to is not listed here, then no additional steps are required. - -<!-- -Example: - -### 11.8.0 - -Additional instructions here. ---> - -### 15.9.0 - -With the addition of `gitlab-sshd` the Kerberos headers are needed to build GitLab Shell. +## Version specific changes -```shell -sudo apt install libkrb5-dev -``` - -### 15.0.0 - -Support for more than one database has been added to GitLab. [As part of this](https://gitlab.com/gitlab-org/gitlab/-/issues/338182), -`config/database.yml` must include a database name in the database configuration. -The `main: database` must be first. If an invalid or deprecated syntax is used, an error is generated -during application start: - -```plaintext -ERROR: This installation of GitLab uses unsupported 'config/database.yml'. -The main: database needs to be defined as a first configuration item instead of primary. (RuntimeError) -``` - -Previously, the `config/database.yml` file looked like the following: - -```yaml -production: - adapter: postgresql - encoding: unicode - database: gitlabhq_production - ... -``` - -Starting with GitLab 15.0, it must define a `main` database first: - -```yaml -production: - main: - adapter: postgresql - encoding: unicode - database: gitlabhq_production - ... -``` - -### 14.5.0 - -As part of [enabling real-time issue assignees](https://gitlab.com/gitlab-org/gitlab/-/issues/330117), Action Cable is now enabled by default, and requires `config/cable.yml` to be present. -You can configure this by running: - -```shell -cd /home/git/gitlab - -sudo -u git -H cp config/cable.yml.example config/cable.yml - -# Change the Redis socket path if you are not using the default Debian / Ubuntu configuration -sudo -u git -H editor config/cable.yml -``` - -### 13.0.1 - -As part of [deprecating Rack Attack throttles on Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4750), the Rack Attack initializer on GitLab -was renamed from [`config/initializers/rack_attack_new.rb` to `config/initializers/rack_attack.rb`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33072). -If this file exists on your installation, consider creating a backup before updating: - -```shell -cd /home/git/gitlab -cp config/initializers/rack_attack.rb ~/config/initializers/rack_attack_backup.rb -``` +Upgrading versions might need some manual intervention. For more information, +[check the version you are upgrading to](index.md#version-specific-upgrading-instructions) +for additional steps required for all GitLab installations, and for +steps that apply to self-compiled (source) installations. ## Troubleshooting diff --git a/doc/user/admin_area/analytics/dev_ops_reports.md b/doc/user/admin_area/analytics/dev_ops_reports.md index 2d19c0a0058..31cc9825452 100644 --- a/doc/user/admin_area/analytics/dev_ops_reports.md +++ b/doc/user/admin_area/analytics/dev_ops_reports.md @@ -39,7 +39,7 @@ feature is available. ## DevOps Adoption **(ULTIMATE SELF)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247112) in GitLab 13.7 as a [Beta feature](../../../policy/alpha-beta-support.md#beta-features). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247112) in GitLab 13.7 as a [Beta feature](../../../policy/alpha-beta-support.md#beta). > - The Overview tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330401) in GitLab 14.1. > - DAST and SAST metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328033) in GitLab 14.1. > - Fuzz Testing metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/330398) in GitLab 14.2. diff --git a/doc/user/admin_area/appearance.md b/doc/user/admin_area/appearance.md index b9850048e7d..a5311b083c3 100644 --- a/doc/user/admin_area/appearance.md +++ b/doc/user/admin_area/appearance.md @@ -2,7 +2,6 @@ stage: none group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/customization/branded_login_page.html' --- # GitLab Appearance **(FREE SELF)** diff --git a/doc/user/admin_area/custom_project_templates.md b/doc/user/admin_area/custom_project_templates.md index 78819def5de..ea42596059e 100644 --- a/doc/user/admin_area/custom_project_templates.md +++ b/doc/user/admin_area/custom_project_templates.md @@ -41,6 +41,24 @@ To select the group to use as the source for the project templates: Projects in subgroups of the template group are **not** included in the template list. +## What is copied from the templates + +The entire custom instance-level project templates repository is copied, including: + +- Branches +- Commits +- Tags + +If the user: + +- Has the Owner role on the custom instance-level project templates project or is a GitLab administrator, all project settings are copied over to the new + project. +- Doesn't have the Owner role or is not a GitLab administrator, project [deploy keys](../project/deploy_keys/index.md#view-deploy-keys) and project + [webhooks](../project/integrations/webhooks.md) aren't copied over because they contain sensitive data. + +To learn more about what is migrated, see +[Items that are exported](../project/settings/import_export.md#items-that-are-exported). + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/admin_area/external_users.md b/doc/user/admin_area/external_users.md index 8b968a3da01..5127630d65e 100644 --- a/doc/user/admin_area/external_users.md +++ b/doc/user/admin_area/external_users.md @@ -48,6 +48,7 @@ Additionally, users can be set as external users using: - [SAML groups](../../integration/saml.md#external-groups). - [LDAP groups](../../administration/auth/ldap/ldap_synchronization.md#external-groups). +- the [External providers list](../../integration/omniauth.md#create-an-external-providers-list). ## Set a new user to external diff --git a/doc/user/admin_area/labels.md b/doc/user/admin_area/labels.md index 8e1ca979707..16721d144e5 100644 --- a/doc/user/admin_area/labels.md +++ b/doc/user/admin_area/labels.md @@ -10,6 +10,7 @@ type: reference To manage labels for the GitLab instance, select **Labels** (**{labels}**) from the Admin Area sidebar. For more details on how to manage labels, see [Labels](../project/labels.md). Labels created in the Admin Area are automatically added to new projects. +They are not available to new groups. Updating or adding labels in the Admin Area does not modify labels in existing projects.  diff --git a/doc/user/admin_area/moderate_users.md b/doc/user/admin_area/moderate_users.md index a273798c8eb..b0e24559e47 100644 --- a/doc/user/admin_area/moderate_users.md +++ b/doc/user/admin_area/moderate_users.md @@ -5,7 +5,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Moderate users **(FREE SELF)** +# Moderate users (administration) **(FREE SELF)** + +This is the administration documentation. For information about moderating users at the group level, see the [group-level documentation](../group/moderate_users.md). GitLab administrators can moderate user access by approving, blocking, banning, or deactivating users. @@ -161,8 +163,8 @@ A user can be deactivated from the Admin Area. To do this: For the deactivation option to be visible to an administrator, the user: -- Must be currently active. -- Must not be [dormant](#automatically-deactivate-dormant-users). +- Must have a state of active. +- Must be [dormant](#automatically-deactivate-dormant-users). NOTE: Users can also be deactivated using the [GitLab API](../../api/users.md#deactivate-user). diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index 668d34af024..f3b09c61532 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -92,8 +92,6 @@ Example response: On failure, the endpoint returns a `503` HTTP status code. -This check does hit the database and Redis if authenticated via `token`. - This check is being exempt from Rack Attack. ## Liveness diff --git a/doc/user/admin_area/reporting/git_abuse_rate_limit.md b/doc/user/admin_area/reporting/git_abuse_rate_limit.md index 1dd9497fed9..83b28404714 100644 --- a/doc/user/admin_area/reporting/git_abuse_rate_limit.md +++ b/doc/user/admin_area/reporting/git_abuse_rate_limit.md @@ -4,27 +4,17 @@ group: Anti-Abuse info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Git abuse rate limit (administration) **(ULTIMATE)** +# Git abuse rate limit (administration) **(ULTIMATE SELF)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with a flag](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. On GitLab.com, this feature is available. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with a flag](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/394996) in GitLab 15.10. Feature flag `git_abuse_rate_limit_feature_flag` removed. This is the administration documentation. For information about Git abuse rate limiting at the group level, see the [group-level documentation](../../group/reporting/git_abuse_rate_limit.md). -Git abuse rate limiting is a feature to automatically [ban users](../moderate_users.md#ban-and-unban-users) who download or clone more than a specified number of repositories in any project in the instance in a given time frame. Banned users cannot sign in to the instance and cannot access any non-public group via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). +Git abuse rate limiting is a feature to automatically [ban users](../moderate_users.md#ban-and-unban-users) who download, clone, or fork more than a specified number of repositories in any project in the instance in a given time frame. Banned users cannot sign in to the instance and cannot access any non-public group via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). Git abuse rate limiting does not apply to instance administrators, [deploy tokens](../../../user/project/deploy_tokens/index.md), or [deploy keys](../../../user/project/deploy_keys/index.md). -## Automatic ban notifications - -If the `git_abuse_rate_limit_feature_flag` feature flag is enabled, selected users receive an email when a user is about to be banned. - -If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, notifications are still sent. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning. - -If automatic banning is enabled, an email notification is sent when a user is about to be banned, and the user is automatically banned from the GitLab instance. - ## Configure Git abuse rate limiting 1. On the top bar, select **Main menu > Admin**. @@ -38,6 +28,12 @@ If automatic banning is enabled, an email notification is sent when a user is ab 1. Optional. Turn on the **Automatically ban users from this namespace when they exceed the specified limits** toggle to enable automatic banning. 1. Select **Save changes**. +## Automatic ban notifications + +If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, notifications are still sent to the users listed under **Send notifications to**. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning. + +If automatic banning is enabled, an email notification is sent when a user is about to be banned, and the user is automatically banned from the GitLab instance. + ## Unban a user 1. On the top bar, select **Main menu > Admin**. diff --git a/doc/user/admin_area/review_abuse_reports.md b/doc/user/admin_area/review_abuse_reports.md index b8531fded18..314e0c77f36 100644 --- a/doc/user/admin_area/review_abuse_reports.md +++ b/doc/user/admin_area/review_abuse_reports.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Authentication and Authorization +stage: Anti-Abuse +group: Anti-Abuse info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference, howto --- diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 8f7081a909d..5d5cd15372a 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -148,7 +148,7 @@ are locked against deletion and kept regardless of the expiry time. When disabled, the latest artifacts for any **new** successful or fixed pipelines are allowed to expire. -This setting takes precedence over the [project level setting](../../../ci/pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). +This setting takes precedence over the [project level setting](../../../ci/jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). If disabled at the instance level, you cannot enable this per-project. To disable the setting: @@ -227,6 +227,7 @@ from the Admin Area: - **Maximum number of DAG dependencies that a job can have** - **Maximum number of runners registered per group** - **Maximum number of runners registered per project** + - **Maximum number of downstream pipelines in a pipeline's hierarchy tree** ## Enable or disable the pipeline suggestion banner @@ -345,8 +346,8 @@ To restrict all users in an instance from registering runners: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. -1. Clear the checkbox if you don't want to display runner registration - information in the UI for group or project members. +1. In the **Runner registration** section, clear the **Members of the project can register runners** and + **Members of the group can register runners** checkboxes to remove runner registration from the UI. 1. Select **Save changes**. NOTE: @@ -370,6 +371,20 @@ To restrict runner registration by members in a specific group: 1. Clear the **New group runners can be registered** checkbox if you want to disable runner registration by all members in the group. If the setting is read-only, you must enable runner registration for the [instance](#restrict-runner-registration-by-all-users-in-an-instance). 1. Select **Save changes**. +## Disable runner version management + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114041) in GitLab 15.10. + +By default, GitLab instances periodically fetch official runner version data from GitLab.com to [determine whether the runners need upgrades](../../../ci/runners/configure_runners.md#determine-which-runners-need-to-be-upgraded). + +To disable your instance fetching this data: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Runners**. +1. In the **Runner version management** section, clear the **Fetch GitLab Runner release version data from GitLab.com** checkbox. +1. Select **Save changes**. + ## Troubleshooting ### 413 Request Entity Too Large diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md index 484f51d8739..90852463e9d 100644 --- a/doc/user/admin_area/settings/email.md +++ b/doc/user/admin_area/settings/email.md @@ -86,12 +86,13 @@ To disable these notifications: ### Custom additional text in deactivation emails **(FREE SELF)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355964) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `deactivation_email_additional_text`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355964) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `deactivation_email_additional_text`. Disabled by default. +> - [Enabled on self-managed and GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111882) in GitLab 15.9. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an -administrator to [enable the feature flag](../../../administration/feature_flags.md) named -`deactivation_email_additional_text`. On GitLab.com, this feature is unavailable. +On self-managed GitLab, by default this feature is available. To hide the feature, ask an +administrator to [disable the feature flag](../../../administration/feature_flags.md) named +`deactivation_email_additional_text`. You can add additional text at the bottom of the email that GitLab sends to users when their account is deactivated. This email text is separate from the [custom additional text](#custom-additional-text) diff --git a/doc/user/admin_area/settings/help_page.md b/doc/user/admin_area/settings/help_page.md index 07d3ae83d74..5d9fc23aaff 100644 --- a/doc/user/admin_area/settings/help_page.md +++ b/doc/user/admin_area/settings/help_page.md @@ -71,14 +71,7 @@ You can specify a custom URL to which users are directed when they: > - [Feature flag `help_page_documentation_redirect`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71737) removed in GitLab 14.4. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71737) in GitLab 14.4. -The `/help` URL of a GitLab instance displays a basic version of the documentation sourced from the -[`doc` directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc) of GitLab. `/help` links -are often used for contextual help. - -You can redirect these `/help` links to either: - -- The more navigable and searchable version published at [`docs.gitlab.com`](https://docs.gitlab.com). -- A destination that meets [necessary requirements](#destination-requirements). +You can redirect all `/help` links to a destination that meets the [necessary requirements](#destination-requirements). 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. @@ -86,17 +79,19 @@ You can redirect these `/help` links to either: 1. In the **Documentation pages URL** field, enter the URL. 1. Select **Save changes**. +If the "Documentation pages URL" field is empty, the GitLab instance displays a basic version of the documentation sourced from the [`doc` directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc) of GitLab. + ### Destination requirements When redirecting `/help`, GitLab: - Redirects requests to the specified URL. -- Appends `ee` and the documentation path to the URL. +- Appends `ee` and the documentation path, which includes the version number, to the URL. - Appends `.html` to the URL, and removes `.md` if necessary. For example, if the URL is set to `https://docs.gitlab.com`, requests for `/help/user/admin_area/settings/help_page.md` redirect to: -`https://docs.gitlab.com/ee/user/admin_area/settings/help_page.html`. +`https://docs.gitlab.com/${VERSION}/ee/user/admin_area/settings/help_page.html`. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index 5c4d7ee4be0..11c14102efb 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -66,6 +66,10 @@ The **CI/CD** settings contain: [risks are involved](../../packages/container_registry/reduce_container_registry_storage.md#use-with-external-container-registries) in enabling some of these settings. +## Security and Compliance settings + +- [License compliance settings](security_and_compliance.md#choose-package-registry-metadata-to-sync): Enable or disable synchronization of package metadata by a registry type. + ### Geo **(PREMIUM SELF)** The **Geo** setting contains: @@ -136,6 +140,7 @@ The **Network** settings contain: number of inbound alerts that can be sent to a project. - [Notes creation limit](rate_limit_on_notes_creation.md) - Set a rate limit on the note creation requests. - [Get single user limit](rate_limit_on_users_api.md) - Set a rate limit on users API endpoint to get a user by ID. +- [Projects API rate limits for unauthenticated requests](rate_limit_on_projects_api.md) - Set a rate limit on Projects list API endpoint for unauthenticated requests. ### Preferences diff --git a/doc/user/admin_area/settings/security_and_compliance.md b/doc/user/admin_area/settings/security_and_compliance.md new file mode 100644 index 00000000000..13abf1027cd --- /dev/null +++ b/doc/user/admin_area/settings/security_and_compliance.md @@ -0,0 +1,20 @@ +--- +stage: Secure +group: Composition Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: howto +--- + +# Security and Compliance Admin Area settings **(ULTIMATE SELF)** + +The settings for package metadata synchronization are located in the [Admin Area](index.md). + +## Choose package registry metadata to sync + +To choose the packages you want to synchronize with the GitLab License Database for [License Compliance](../../compliance/license_scanning_of_cyclonedx_files/index.md): + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > Security and Compliance**. +1. Expand **License Compliance**. +1. Select or clear checkboxes for the package registries that you want to sync. +1. Select **Save changes**. diff --git a/doc/user/admin_area/settings/sign_in_restrictions.md b/doc/user/admin_area/settings/sign_in_restrictions.md index 82a54787101..a3558991e36 100644 --- a/doc/user/admin_area/settings/sign_in_restrictions.md +++ b/doc/user/admin_area/settings/sign_in_restrictions.md @@ -158,7 +158,7 @@ see [Email notification for unknown sign-ins](../../profile/notifications.md#not All users that are not logged in are redirected to the page represented by the configured **Home page URL** if value is not empty. -All users are redirected to the page represented by the configured **After sign-out path** +All users are redirected to the page represented by the configured **Sign-out page URL** after sign out if value is not empty. In the **Sign-in restrictions** section, scroll to the **Sign-in text** field. You can add a diff --git a/doc/user/admin_area/settings/terraform_limits.md b/doc/user/admin_area/settings/terraform_limits.md index 4e54c7a3459..05b1f2d8838 100644 --- a/doc/user/admin_area/settings/terraform_limits.md +++ b/doc/user/admin_area/settings/terraform_limits.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index 212769ed89b..1db62bce056 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -119,6 +119,11 @@ To enable or disable Service Ping and version check: 1. Select or clear the **Enable version check** and **Enable Service Ping** checkboxes. 1. Select **Save changes**. +NOTE: +Service Ping settings only control whether the data is being shared with GitLab, or used only internally. +Even if you disable Service Ping, the `gitlab_service_ping_worker` background job still periodically generates a Service Ping payload for your instance. +The payload is available in the [Service Usage data](#manually-upload-service-ping-payload) admin section. + ## Disable usage statistics with the configuration file NOTE: @@ -194,6 +199,11 @@ To upload the payload manually: The uploaded file is encrypted and sent using secure HTTPS protocol. HTTPS creates a secure communication channel between web browser and the server, and protects transmitted data against man-in-the-middle attacks. +If there are problems with the manual upload: + +1. Open a confidential issue in the [security fork of version app project](https://gitlab.com/gitlab-org/security/version.gitlab.com). +1. Attach the JSON payload if possible. +1. Tag `@gitlab-org/analytics-section/product-intelligence` who will triage the issue. <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index cd0b461c26b..07067945ea6 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -54,6 +54,7 @@ By default both administrators and anyone with the **Owner** role can delete a p > - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) from default delayed project deletion in GitLab 15.1. > - [Enabled for projects in personal namespaces](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89466) in GitLab 15.1. > - [Disabled for projects in personal namespaces](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95495) in GitLab 15.3. +> - [Removed option to delete immediately](https://gitlab.com/gitlab-org/gitlab/-/issues/389557) in GitLab 15.11 [with a flag](../../../administration/feature_flags.md) named `always_perform_delayed_deletion`. Disabled by default. Instance-level protection against accidental deletion of groups and projects. @@ -82,6 +83,7 @@ To configure delayed project deletion: 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Scroll to: + - (GitLab 15.11 and later with `always_perform_delayed_deletion` feature flag enabled) **Deletion protection** and set the retention period to a value between `1` and `90`. - (GitLab 15.1 and later) **Deletion protection** and select keep deleted groups and projects, and select a retention period. - (GitLab 15.0 and earlier) **Default delayed project protection** and select **Enable delayed project deletion by default for newly-created groups.** Then set a retention period in **Default deletion delay**. @@ -98,6 +100,10 @@ In GitLab 15.1, and later this setting is enforced on groups when disabled and i Groups remain restorable if the retention period is `1` or more days. In GitLab 15.1 and later, delayed group deletion can be enabled by setting **Deletion projection** to **Keep deleted**. +In GitLab 15.11 and later with the `always_perform_delayed_deletion` feature flag enabled: + +- The **Keep deleted** option is removed. +- Delayed group deletion is the default. ### Override defaults and delete immediately diff --git a/doc/user/analytics/ci_cd_analytics.md b/doc/user/analytics/ci_cd_analytics.md index 4130ff6e036..bde7fdc6c2d 100644 --- a/doc/user/analytics/ci_cd_analytics.md +++ b/doc/user/analytics/ci_cd_analytics.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -39,7 +39,7 @@ To view CI/CD analytics: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275991) in GitLab 13.8. -The deployment frequency charts show information about the deployment +The [deployment frequency](dora_metrics.md#deployment-frequency) charts show information about the deployment frequency to the `production` environment. The environment must be part of the [production deployment tier](../../ci/environments/index.md#deployment-tier-of-environments) for its deployment information to appear on the graphs. @@ -60,7 +60,7 @@ To view the deployment frequency chart: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250329) in GitLab 13.11. -The lead time for changes chart shows information about how long it takes for +The [lead time for changes](dora_metrics.md#lead-time-for-changes) chart shows information about how long it takes for merge requests to be deployed to a production environment. This chart is available for groups and projects. - Small lead times indicate fast, efficient deployment @@ -82,7 +82,7 @@ To view the lead time for changes chart: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356959) in GitLab 15.1 -The time to restore service chart shows information about the median time an incident was open in a production environment. This chart is available for groups and projects. +The [time to restore service](dora_metrics.md#time-to-restore-service) chart shows information about the median time an incident was open in a production environment. This chart is available for groups and projects. Time to restore service is one of the four DORA metrics that DevOps teams use for measuring excellence in software delivery. @@ -98,7 +98,7 @@ To view the time to restore service chart: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357072) in GitLab 15.2 -The change failure rate chart shows information about the percentage of deployments that cause an incident in a production environment. This chart is available for groups and projects. +The [change failure rate](dora_metrics.md#change-failure-rate) chart shows information about the percentage of deployments that cause an incident in a production environment. This chart is available for groups and projects. Change failure rate is one of the four DORA metrics that DevOps teams use for measuring excellence in software delivery. diff --git a/doc/user/analytics/code_review_analytics.md b/doc/user/analytics/code_review_analytics.md index fae9545003f..2e76252f7d3 100644 --- a/doc/user/analytics/code_review_analytics.md +++ b/doc/user/analytics/code_review_analytics.md @@ -5,7 +5,6 @@ group: Optimize info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- - # Code review analytics **(PREMIUM)** > Moved to GitLab Premium in 13.9. @@ -24,6 +23,9 @@ and improve your code review process. - Opportunities to accelerate your development cycle. - Few comments and approvers may indicate a lack of available team members. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a video explanation, see [Code review analytics: Faster code review](https://www.youtube.com/watch?v=OkLaWhYkASM). + ## View code review analytics Prerequisite: diff --git a/doc/user/analytics/dora_metrics.md b/doc/user/analytics/dora_metrics.md index b85ec60f00e..70ce005dfb9 100644 --- a/doc/user/analytics/dora_metrics.md +++ b/doc/user/analytics/dora_metrics.md @@ -15,19 +15,15 @@ Using these metrics helps improve DevOps efficiency and communicate performance DORA includes four key metrics, divided into two core areas of DevOps: -- [Deployment Frequency](#deployment-frequency) and [Lead Time for Change](#lead-time-for-changes) measure team velocity. -- [Change Failure Rate](#change-failure-rate) and [Time to Restore Service](#time-to-restore-service) measure stability. +- [Deployment frequency](#deployment-frequency) and [Lead time for changes](#lead-time-for-changes) measure team velocity. +- [Change failure rate](#change-failure-rate) and [Time to restore service](#time-to-restore-service) measure stability. For software leaders, tracking velocity alongside quality metrics ensures they're not sacrificing quality for speed. -<div class="video-fallback"> - For an overview, see <a href="https://www.youtube.com/watch?v=1BrcMV6rCDw">GitLab Speed Run: DORA metrics in GitLab One DevOps Platform</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube-nocookie.com/embed/1BrcMV6rCDw" frameborder="0" allowfullscreen> </iframe> -</figure> +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a video explanation, see [DORA metrics: User analytics](https://www.youtube.com/watch?v=lM_FbVYuN8s) and [GitLab speed run: DORA metrics](https://www.youtube.com/watch?v=1BrcMV6rCDw). -## DORA Metrics dashboard in Value Stream Analytics +## DORA metrics dashboard in Value Stream Analytics The four DORA metrics are available out-of-the-box in the [Value Stream Analytics (VSA) overview dashboard](../group/value_stream_analytics/index.md#view-value-stream-analytics). This helps you visualize the engineering work in the context of end-to-end value delivery. @@ -140,7 +136,7 @@ The DORA metrics are displayed on the following charts: The table below provides an overview of the DORA metrics' data aggregation in different charts. -| Metric name | Measured values | Data aggregation in the [Value Streams Dashboard](value_streams_dashboard.md) | Data aggregation in CI/CD analytics charts | Data aggregation in Value stream analytics | Data aggregation in Custom insights reporting | +| Metric name | Measured values | Data aggregation in the [Value Streams Dashboard](value_streams_dashboard.md) | Data aggregation in [CI/CD analytics charts](ci_cd_analytics.md) | Data aggregation in Value stream analytics | Data aggregation in [Custom insights reporting](../../user/project/insights/index.md#dora-query-parameters) | |---------------------------|-------------------|-----------------------------------------------------|------------------------|-----------------------|----------| | Deployment frequency | Number of successful deployments | daily average per month | daily average | daily average | `day` (default) or `month` | | Lead time for changes | Number of seconds to successfully deliver a commit into production | daily median per month | median time | median time | `day` (default) or `month` | @@ -158,6 +154,26 @@ These deployment records are not created for pull-based deployments, for example To track DORA metrics in these cases, you can [create a deployment record](../../api/deployments.md#create-a-deployment) using the Deployments API. See also the documentation page for [Track deployments of an external deployment tool](../../ci/environments/external_deployment_tools.md). +### Measure DORA metrics with Jira + +- Deployment frequency and Lead time for changes are calculated based on GitLab CI/CD and Merge Requests (MRs), and do not require Jira data. +- Time to restore service and Change failure rate require GitLab incidents for the calculation. For more information, see [Measure DORA Time to restore service and Change failure rate with external incidents](#measure-dora-time-to-restore-service-and-change-failure-rate-with-external-incidents). + +### Measure DORA Time to restore service and Change failure rate with external incidents + +[Time to restore service](#time-to-restore-service) and [Change failure rate](#change-failure-rate) +require [GitLab incidents](../../operations/incident_management/manage_incidents.md) to calculate the metrics. + +For PagerDuty, you can set up a [webhook to automatically create a GitLab incident for each PagerDuty incident](../../operations/incident_management/manage_incidents.md#using-the-pagerduty-webhook). +This configuration requires you to make changes in both PagerDuty and GitLab. + +For others incident management tools, you can set up the +[HTTP integration](../../operations/incident_management/integrations.md#http-endpoints), +and use it to automatically: + +1. [Create an incident when an alert is triggered](../../operations/incident_management/manage_incidents.md#automatically-when-an-alert-is-triggered). +1. [Close incidents via recovery alerts](../../operations/incident_management/manage_incidents.md#automatically-close-incidents-via-recovery-alerts). + ### Supported DORA metrics in GitLab | Metric | Level | API | UI chart | Comments | diff --git a/doc/user/analytics/index.md b/doc/user/analytics/index.md index 774063810f3..44af9dc9dea 100644 --- a/doc/user/analytics/index.md +++ b/doc/user/analytics/index.md @@ -59,7 +59,7 @@ You can use the following analytics features to analyze and visualize the perfor - [Value stream analytics for groups](../group/value_stream_analytics/index.md) - [Value streams dashboard](value_streams_dashboard.md) -## Definitions +## Glossary We use the following terms to describe GitLab analytics: diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index f22a6a483b8..292a6f430d9 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -1,246 +1,13 @@ --- -stage: Plan -group: Optimize -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-07-22' +redirect_to: '../group/value_stream_analytics/index.md' --- # Value stream analytics for projects **(FREE)** -> - Introduced as cycle analytics prior to GitLab 12.3 at the project level. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12077) in GitLab Premium 12.3 at the group level. -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/23427) from cycle analytics to value stream analytics in GitLab 12.8. +This document was moved to [another location](../group/value_stream_analytics/index.md). -Value stream analytics provides metrics about each stage of your software development process. - -A **value stream** is the entire work process that delivers value to customers. For example, -the [DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/) is a value stream that starts -with the Manage stage and ends with the Protect stage. - -Use value stream analytics to identify: - -- The amount of time it takes to go from an idea to production. -- The velocity of a given project. -- Bottlenecks in the development process. -- Factors that cause your software development lifecycle to slow down. - -Value stream analytics is also available for [groups](../group/value_stream_analytics). - -## View value stream analytics - -> - Filtering [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326701) in GitLab 14.3 -> - Sorting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335974) in GitLab 14.4. - -To view value stream analytics for your project: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. To view metrics for a particular stage, select a stage below the **Filter results** text box. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. -1. Optional. Sort results by ascending or descending: - - To sort by most recent or oldest workflow item, select the **Last event** header. - - To sort by most or least amount of time spent in each stage, select the **Duration** header. - -The table shows a list of related workflow items for the selected stage. Based on the stage you choose, this can be: - -- CI/CD jobs -- Issues -- Merge requests -- Pipelines - -A badge next to the workflow items table header shows the number of workflow items that completed the selected stage. - -## View time spent in each development stage - -Value stream analytics shows the median time spent by issues or merge requests in each development stage. - -To view the median time spent in each stage: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. -1. To view the median time for each stage, above the **Filter results** text box, point to a stage. - -## View the lead time and cycle time for issues - -Value stream analytics shows the lead time and cycle time for issues in your project: - -- Lead time: Median time from when the issue was created to when it was closed. -- Cycle time: Median time from first commit to issue closed. GitLab measures cycle time from the earliest commit of a [linked issue's merge request](../project/issues/crosslinking_issues.md) to when that issue is closed. The cycle time approach underestimates the lead time because merge request creation is always later than commit time. - -To view the lead time and cycle time for issues: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. - -The **Lead Time** and **Cycle Time** metrics display below the **Filter results** text box. - -## View lead time for changes for merge requests **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340150) in GitLab 14.5. - -Lead time for changes is the median duration between when a merge request is merged and when it's deployed to production. - -To view the lead time for changes for merge requests in your project: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. - -The **Lead Time for Changes** metrics display below the **Filter results** text box. - -## View number of successful deployments **(FREE)** - -Prerequisites: - -- To view deployment metrics, you must have a -[production environment configured](../../ci/environments/index.md#deployment-tier-of-environments). - -Value stream analytics shows the following deployment metrics for your project within the specified date range: - -- Deploys: The number of successful deployments in the date range. -- Deployment Frequency: The average number of successful deployments per day in the date range. - -If you have a GitLab Premium or Ultimate subscription: - -- The number of successful deployments is calculated with DORA data. -- The data is filtered based on environment and environment tier. - -To view deployment metrics for your project: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. - -The **Deploys** and **Deployment Frequency** metrics display below the **Filter results** text box. - -Deployment metrics are calculated based on data from the -[DORA API](../../api/dora/metrics.md#devops-research-and-assessment-dora-key-metrics-api). - -NOTE: -In GitLab 13.9 and later, metrics are calculated based on when the deployment was finished. -In GitLab 13.8 and earlier, metrics are calculated based on when the deployment was created. - -## Access permissions for value stream analytics - -Access permissions for value stream analytics depend on the project type. - -| Project type | Permissions | -|--------------|----------------------------------------| -| Public | Anyone can access. | -| Internal | Any authenticated user can access. | -| Private | Any member Guest and above can access. | - -## How value stream analytics measures each stage - -Value stream analytics uses start and end events to measure the time that an issue or merge request -spends in each stage. - -For example, a stage might start when a user adds a label to an issue, and ends when they add another label. -Items aren't included in the stage time calculation if they have not reached the end event. - -| Stage | Measurement method | -|---------|----------------------| -| Issue | The median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone. The label is tracked only if it already includes an [issue board list](../project/issue_board.md) that has been created for the label. | -| Plan | The median time between the action you took for the previous stage, and when you push the first commit to the branch. The first branch commit triggers the transition from **Plan** to **Code**, and at least one of the commits in the branch must include the related issue number (such as `#42`). If the issue number is not included in a commit, that data is not included in the measurement time of the stage. | -| Code | The median time between pushing a first commit (previous stage) and creating a merge request. The process is tracked with the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) in the description of the merge request. For example, if the issue is closed with `Closes #xxx`, `xxx` is the issue number for the merge request. If there is no closing pattern, the start time is set to the create time of the first commit. | -| Test | The time from start to finish for all pipelines. Measures the median time to run the entire pipeline for that project. Related to the time required by GitLab CI/CD to run every job for the commits pushed to that merge request, as defined in the previous stage. | -| Review | The median time taken to review merge requests with a closing issue pattern, from creation to merge. | -| Staging | The median time between merging the merge request (with a closing issue pattern) to the first deployment to a [production environment](../../ci/environments/index.md#deployment-tier-of-environments). Data is not collected without a production environment. | - -## Example workflow - -This example shows a workflow through all seven stages in one day. In this -example, milestones have been created and CI for testing and setting environments is configured. - -- 09:00: Create issue. **Issue** stage starts. -- 11:00: Add issue to a milestone, start work on the issue, and create a branch locally. -**Issue** stage stops and **Plan** stage starts. -- 12:00: Make the first commit. -- 12:30: Make the second commit to the branch that mentions the issue number. **Plan** stage stops and **Code** stage starts. -- 14:00: Push branch and create a merge request that contains the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically). **Code** stage stops and **Test** and **Review** stages start. -- The CI takes 5 minutes to run scripts defined in [`.gitlab-ci.yml`](../../ci/yaml/index.md). -**Test** stage stops. -- Review merge request. -- 19:00: Merge the merge request. **Review** stage stops and **Staging** stage starts. -- 19:30: Deployment to the `production` environment starts and finishes. **Staging** stops. - -Value stream analytics records the following times for each stage: - -- **Issue**: 09:00 to 11:00: 2 hrs -- **Plan**: 11:00 to 12:30: 1.5 hr -- **Code**: 12:30 to 14:00: 1.5 hrs -- **Test**: 5 minutes -- **Review**: 14:00 to 19:00: 5 hrs -- **Staging**: 19:00 to 19:30: 30 minutes - -Keep in mind the following observations related to this example: - -- Although this example specifies the issue number in a later commit, the process -still collects analytics data for the issue. -- The time required in the **Test** stage is included in the **Review** process, -as every merge request should be tested. -- This example illustrates only one cycle of multiple stages. The value -stream analytics dashboard shows the calculated median elapsed time for these issues. -- Value stream analytics identifies production environments based on the -[deployment tier of environments](../../ci/environments/index.md#deployment-tier-of-environments). - -## Troubleshooting - -### 100% CPU utilization by Sidekiq `cronjob:analytics_cycle_analytics` - -It is possible that Value stream analytics background jobs -strongly impact performance by monopolizing CPU resources. - -To recover from this situation: - -1. Disable the feature for all projects in [the Rails console](../../administration/operations/rails_console.md), - and remove existing jobs: - - ```ruby - Project.find_each do |p| - p.analytics_access_level='disabled'; - p.save! - end - - Analytics::CycleAnalytics::GroupStage.delete_all - Analytics::CycleAnalytics::Aggregation.delete_all - ``` - -1. Configure a [Sidekiq routing](../../administration/sidekiq/processing_specific_job_classes.md) - with for example a single `feature_category=value_stream_management` - and multiple `feature_category!=value_stream_management` entries. - Find other relevant queue metadata in the - [Enterprise Edition list](../../administration/sidekiq/processing_specific_job_classes.md#list-of-available-job-classes). -1. Enable value stream analytics for one project after another. - You might need to tweak the Sidekiq routing further according to your performance requirements. +<!-- This redirect file can be deleted after 2023-07-22. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/analytics/value_streams_dashboard.md b/doc/user/analytics/value_streams_dashboard.md index fd4cdd0d65f..a8c0ad0a7aa 100644 --- a/doc/user/analytics/value_streams_dashboard.md +++ b/doc/user/analytics/value_streams_dashboard.md @@ -4,13 +4,14 @@ group: Optimize info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Value Streams Dashboard **(ULTIMATE)** +# Value Streams Dashboard (Beta) **(ULTIMATE)** -> Introduced in GitLab 15.8 as a Closed [Beta](../../policy/alpha-beta-support.md#beta-features) feature [with a flag](../../administration/feature_flags.md) named `group_analytics_dashboards_page`. Disabled by default. +> - Introduced in GitLab 15.8 as a Closed [Beta](../../policy/alpha-beta-support.md#beta) feature [with a flag](../../administration/feature_flags.md) named `group_analytics_dashboards_page`. Disabled by default. +> - Released in GitLab 15.11 as an Open [Beta](../../policy/alpha-beta-support.md#beta) feature [with a flag](../../administration/feature_flags.md) named `group_analytics_dashboards_page`. Enabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `group_analytics_dashboards_page`. -On GitLab.com, this feature is not available. This feature is not ready for production use. +On self-managed GitLab, this feature is available by default. To disable it, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `group_analytics_dashboards_page`. +On GitLab.com, this feature is available. This feature is not ready for production use. You can leave feedback on dashboard bugs or functionality in [issue 381787](https://gitlab.com/gitlab-org/gitlab/-/issues/381787). @@ -18,9 +19,6 @@ The Value Streams Dashboard is a customizable dashboard that enables decision-ma This page is a work in progress, and we're updating the information as we add more features. For more information, see the [Value Stream Management category direction page](https://about.gitlab.com/direction/plan/value_stream_management/). -After the feature flag is enabled, to open the new page, append this path `/analytics/dashboards/value_streams_dashboard` to the group URL -(for example, `https://gitlab.com/groups/gitlab-org/-/analytics/dashboards/value_streams_dashboard`). - ## Initial use case Our initial use case is focused on providing the ability to compare software delivery metrics. @@ -51,6 +49,22 @@ that are the largest value contributors, overperforming, or underperforming. You can also drill down the metrics for further analysis. When you hover over a metric, a tooltip displays an explanation of the metric and a link to the related documentation page. +## View the value streams dashboard + +Prerequisite: + +- To view the value streams dashboard for a group, you must have at least the Reporter role for the group. + +To view the value streams dashboard: + +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Below the **Filter results** text box, in the **Key metrics** row, select **Value Streams Dashboard | DORA**. +1. Optional. To open the new page, append this path `/analytics/dashboards/value_streams_dashboard` to the group URL +(for example, `https://gitlab.com/groups/gitlab-org/-/analytics/dashboards/value_streams_dashboard`). + ## Customize the dashboard panels You can customize the Value Streams Dashboard and configure what subgroups and projects to include in the page. @@ -73,7 +87,7 @@ For example, the parameter `query=gitlab-org/gitlab-foss,gitlab-org/gitlab,gitla | Lead time for changes | The time to successfully deliver a commit into production. This metric reflects the efficiency of CI/CD pipelines. | [Lead time tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=lead-time) | [Lead time for changes](dora_metrics.md#lead-time-for-changes) | | Time to restore service | The time it takes an organization to recover from a failure in production. | [Time to restore service tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=time-to-restore-service) | [Time to restore service](dora_metrics.md#time-to-restore-service) | | Change failure rate | Percentage of deployments that cause an incident in production. | [Change failure rate tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=change-failure-rate) | [Change failure rate](dora_metrics.md#change-failure-rate) | -| VSA Lead time | Median time from issue created to issue closed. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](value_stream_analytics.md#view-the-lead-time-and-cycle-time-for-issues) | -| VSA Cycle time | Median time from the earliest commit of a linked issue's merge request to when that issue is closed. | [VSA overview](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](value_stream_analytics.md#view-the-lead-time-and-cycle-time-for-issues) | +| Lead time | Median time from issue created to issue closed. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#key-metrics) | +| Cycle time | Median time from the earliest commit of a linked issue's merge request to when that issue is closed. | [VSA overview](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#key-metrics) | | New issues | Number of new issues created. | [Issue Analytics](https://gitlab.com/groups/gitlab-org/-/issues_analytics) | Issue analytics [for projects](issue_analytics.md) and [for groups](../../user/group/issues_analytics/index.md) | | Number of deploys | Total number of deploys to production. | [Merge Request Analytics](https://gitlab.com/gitlab-org/gitlab/-/analytics/merge_request_analytics) | [Merge request analytics](merge_request_analytics.md) | diff --git a/doc/user/application_security/api_security/api_discovery/index.md b/doc/user/application_security/api_security/api_discovery/index.md index b77bed74c70..6a67dfc3f6a 100644 --- a/doc/user/application_security/api_security/api_discovery/index.md +++ b/doc/user/application_security/api_security/api_discovery/index.md @@ -82,7 +82,10 @@ When running in this method, you provide a container image that has the required image: openjdk:11-jre-slim ``` -1. Provide the Java classpath needed by your application. This includes your compatible build artifact from step 2, along with any additional dependencies. For this example, the build artifact is `build/libs/spring-boot-app-0.0.0.jar` and contains all needed dependencies. The variable `API_DISCOVERY_JAVA_CLASSPATH` is used to provide the classpath. +1. Provide the Java class path needed by your application. This includes your compatible build + artifact from step 2, along with any additional dependencies. For this example, the build artifact + is `build/libs/spring-boot-app-0.0.0.jar` and contains all needed dependencies. The variable + `API_DISCOVERY_JAVA_CLASSPATH` is used to provide the class path. ```yaml api_discovery: @@ -92,7 +95,10 @@ When running in this method, you provide a container image that has the required API_DISCOVERY_JAVA_CLASSPATH: build/libs/spring-boot-app-0.0.0.jar ``` -1. [Optional] If the image provided is missing a dependency needed by API Discovery, it can be added using a `before_script`. In this example, the `openjdk:11-jre-slim` container doesn't include `curl` which is required by API Discovery. The dependency can be installed using the Debian package manager `apt` as shown below. +1. Optional. If the image provided is missing a dependency needed by API Discovery, it can be added + using a `before_script`. In this example, the `openjdk:11-jre-slim` container doesn't include + `curl` which is required by API Discovery. The dependency can be installed using the Debian + package manager `apt`: ```yaml api_discovery: @@ -104,7 +110,8 @@ When running in this method, you provide a container image that has the required - apt-get update && apt-get install -y curl ``` -1. [Optional] If the image provided doesn't automatically set the `JAVA_HOME` environment variable, or include `java` in the path, the `API_DISCOVERY_JAVA_HOME` variable can be used. +1. Optional. If the image provided doesn't automatically set the `JAVA_HOME` environment variable, + or include `java` in the path, the `API_DISCOVERY_JAVA_HOME` variable can be used. ```yaml api_discovery: @@ -115,7 +122,12 @@ When running in this method, you provide a container image that has the required API_DISCOVERY_JAVA_HOME: /opt/java ``` -1. [Optional] If the package registry at `API_DISCOVERY_PACKAGES` is not public, provide a token that has read access to the GitLab API and registry using the `API_DISCOVERY_PACKAGE_TOKEN` variable. This is not required if you are using `gitlab.com` and have not customized the `API_DISCOVERY_PACKAGES` variable. This example uses a [custom CI/CD variable](../../../../ci/variables/index.md#define-a-cicd-variable-in-the-ui) named `GITLAB_READ_TOKEN` to store the token. +1. Optional. If the package registry at `API_DISCOVERY_PACKAGES` is not public, provide a token that + has read access to the GitLab API and registry using the `API_DISCOVERY_PACKAGE_TOKEN` variable. + This is not required if you are using `gitlab.com` and have not customized the `API_DISCOVERY_PACKAGES` + variable. The following example uses a + [custom CI/CD variable](../../../../ci/variables/index.md#define-a-cicd-variable-in-the-ui) named + `GITLAB_READ_TOKEN` to store the token. ```yaml api_discovery: @@ -133,7 +145,7 @@ After the API Discovery job has successfully run, the OpenAPI document is availa - Linux container image. - Java versions 11 or 17 are officially supported, but other versions are likely compatible as well. - The `curl` command. -- A shell at `/bin/sh` (like busybox sh or bash). +- A shell at `/bin/sh` (like `busybox`, `sh`, or `bash`). ### Available CI/CD variables diff --git a/doc/user/application_security/breach_and_attack_simulation/index.md b/doc/user/application_security/breach_and_attack_simulation/index.md new file mode 100644 index 00000000000..0d662f7d469 --- /dev/null +++ b/doc/user/application_security/breach_and_attack_simulation/index.md @@ -0,0 +1,47 @@ +--- +stage: Secure +group: Incubation +info: Breach and Attack Simulation is a GitLab Incubation Engineering program. No technical writer assigned to this group. +type: reference, howto +--- + +# Breach and Attack Simulation **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/402784) in GitLab 15.11. + +DISCLAIMER: +Breach and Attack Simulation is a set of experimental features being developed by the Incubation Engineering Department and is subject to significant changes over time. + +Breach and Attack Simulation (BAS) uses additional security testing techniques to assess the risk of detected vulnerabilities and prioritize the remediation of exploitable vulnerabilities. + +For feedback, bug reports, and feature requests, see the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/404809). + +WARNING: +Only run BAS scans against test servers. Testing attacker behavior can lead to modification or loss of data. + +## Extend Dynamic Application Security Testing (DAST) + +You can simulate attacks with [DAST](../dast/index.md) to detect vulnerabilities. +By default, DAST active checks match an expected response, or determine by response +time whether a vulnerability was exploited. + +Enable the BAS feature flag in DAST to: + +- Enable callback, match response, and timing attacks inside of active checks. +- Perform Out-of-Band Application Security Testing (OAST) through callback attacks in active checks. + +To enable BAS: + +1. Create a CI/CD job using the [DAST browser-based analyzer](../dast/browser_based.md#create-a-dast-cicd-job). +1. Set the `DAST_FF_ENABLE_BAS` [CI/CD variable](../dast/browser_based.md#available-cicd-variables) to `true`. + +```yaml +include: + - template: DAST.gitlab-ci.yml + +dast: + variables: + DAST_BROWSER_SCAN: "true" + DAST_FF_ENABLE_BAS: "true" + DAST_WEBSITE: "https://my.site.com" +``` diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 380624c91ce..d5a05477ddf 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -51,7 +51,7 @@ You can configure the following security controls: - [Static Application Security Testing](../sast/index.md) (SAST) - Select **Enable SAST** to configure SAST for the current project. - For more details, read [Configure SAST in the UI](../sast/index.md#configure-sast-in-the-ui). + For more details, read [Configure SAST in the UI](../sast/index.md#configure-sast-by-using-the-ui). - [Dynamic Application Security Testing](../dast/index.md) (DAST) - Select **Enable DAST** to configure DAST for the current project. - Select **Manage scans** to manage the saved DAST scans, site profiles, and scanner profiles. diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index a38f7bcb77c..793c9a89ec0 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -22,6 +22,9 @@ vulnerabilities. By including an extra Container Scanning job in your pipeline t vulnerabilities and displays them in a merge request, you can use GitLab to audit your Docker-based apps. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Container Scanning](https://www.youtube.com/watch?v=C0jn2eN5MAs). + Container Scanning is often considered part of Software Composition Analysis (SCA). SCA can contain aspects of inspecting the items your code uses. These items typically include application and system dependencies that are almost always imported from external sources, rather than sourced from items @@ -696,6 +699,14 @@ These reports must follow a format defined in the For more information, see [Security scanner integration](../../../development/integrations/secure.md). +### CycloneDX Software Bill of Materials + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/396381) in GitLab 15.11. + +In addition to the [JSON report file](#reports-json-format), the [Container Scanning](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning) tool outputs a [CycloneDX](https://cyclonedx.org/) Software Bill of Materials (SBOM) for the scanned image. This CycloneDX SBOM is named `gl-sbom-report.cdx.json` and is saved in the same directory as the `JSON report file`. This feature is only supported when the `Trivy` analyzer is used. + +You can download CycloneDX SBOMs [the same way as other job artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts). + ## Security Dashboard The [Security Dashboard](../security_dashboard/index.md) shows you an overview of all @@ -774,7 +785,7 @@ To prevent the error, ensure the Docker version that the runner is using is ### Getting warning message `gl-container-scanning-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ## Changes diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index 4731e09474c..09370a9a7f5 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -15,6 +15,9 @@ We recommend that you use fuzz testing in addition to the other security scanner and your own test processes. If you're using [GitLab CI/CD](../../../ci/index.md), you can run your coverage-guided fuzz testing as part your CI/CD workflow. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Coverage Fuzzing](https://www.youtube.com/watch?v=bbIenVVcjW0). + ## Coverage-guided fuzz testing process The fuzz testing process: @@ -145,7 +148,7 @@ Each fuzzing step outputs these artifacts: previous jobs. You can download the JSON report file from the CI/CD pipelines page. For more information, see -[Downloading artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +[Downloading artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts). ## Corpus registry @@ -222,7 +225,7 @@ Prerequisites: ## Coverage-guided fuzz testing report -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220062) in GitLab 13.3 as an [Alpha feature](../../../policy/alpha-beta-support.md#alpha-features). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220062) in GitLab 13.3 as an [Experiment](../../../policy/alpha-beta-support.md#experiment). For detailed information about the `gl-coverage-fuzzing-report.json` file's format, read the [schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/coverage-fuzzing-report-format.json). diff --git a/doc/user/application_security/dast/authentication.md b/doc/user/application_security/dast/authentication.md index 03a273ffff9..27f56e5224c 100644 --- a/doc/user/application_security/dast/authentication.md +++ b/doc/user/application_security/dast/authentication.md @@ -47,7 +47,7 @@ To run a DAST authenticated scan: - You have the username and password of the user you would like to authenticate as during the scan. - You have checked the [known limitations](#known-limitations) to ensure DAST can authenticate to your application. -- You have satisfied the prerequisites depending on whether you're using [form authentication](#form-authentication) or [HTTP authentication]((#http-authentication). +- You have satisfied the prerequisites depending on whether you're using [form authentication](#form-authentication) or [HTTP authentication](#http-authentication). - You have thought about how you can [verify](#verifying-authentication-is-successful) whether or not authentication was successful. #### Form authentication @@ -80,6 +80,7 @@ To run a DAST authenticated scan: | `DAST_SUBMIT_FIELD` | string | The `id` or `name` of the element that when selected submits the login form or the password form of a multi-page login process. For example, `css:button[type='submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9894) in GitLab 12.4. | | `DAST_USERNAME` <sup>1</sup> | string | The username to authenticate to in the website. Example: `admin` | | `DAST_USERNAME_FIELD` <sup>1</sup> | string | The selector of username field at the sign-in HTML form. Example: `name:username` | +| `DAST_AUTH_DISABLE_CLEAR_FIELDS` | boolean | Disables clearing of username and password fields before attempting manual login. Set to `false` by default. | 1. Available to an on-demand proxy-based DAST scan. 1. Not available to proxy-based scans. diff --git a/doc/user/application_security/dast/browser_based.md b/doc/user/application_security/dast/browser_based.md index 88be88ad00e..7b263e5817d 100644 --- a/doc/user/application_security/dast/browser_based.md +++ b/doc/user/application_security/dast/browser_based.md @@ -173,7 +173,9 @@ For authentication CI/CD variables, see [Authentication](authentication.md). | `DAST_BROWSER_ALLOWED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are considered in scope when crawled. By default the `DAST_WEBSITE` hostname is included in the allowed hosts list. Headers set using `DAST_REQUEST_HEADERS` are added to every request made to these hostnames. | | `DAST_BROWSER_COOKIES` | dictionary | `abtesting_group:3,region:locked` | A cookie name and value to be added to every request. | | `DAST_BROWSER_CRAWL_GRAPH` | boolean | `true` | Set to `true` to generate an SVG graph of navigation paths visited during crawl phase of the scan. You must also define `gl-dast-crawl-graph.svg` as a CI job artifact to be able to access the generated graph. | -| `DAST_BROWSER_DEVTOOLS_LOG` | string | `Default:messageAndBody,truncate:2000` | Set to log protocol messages between DAST and the Chromium browser. | | +| `DAST_BROWSER_CRAWL_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `5m` | The maximum amount of time to wait for the crawl phase of the scan to complete. Defaults to `24h`. | +| `DAST_BROWSER_DEVTOOLS_LOG` | string | `Default:messageAndBody,truncate:2000` | Set to log protocol messages between DAST and the Chromium browser. | +| `DAST_BROWSER_DOM_READY_AFTER_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `200ms` | Define how long to wait for updates to the DOM before checking a page is stable. Defaults to `500ms`. | | `DAST_BROWSER_ELEMENT_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `600ms` | The maximum amount of time to wait for an element before determining it is ready for analysis. | | `DAST_BROWSER_EXCLUDED_ELEMENTS` | selector | `a[href='2.html'],css:.no-follow` | Comma-separated list of selectors that are ignored when scanning. | | `DAST_BROWSER_EXCLUDED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are considered excluded and connections are forcibly dropped. | @@ -192,11 +194,13 @@ For authentication CI/CD variables, see [Authentication](authentication.md). | `DAST_BROWSER_NUMBER_OF_BROWSERS` | number | `3` | The maximum number of concurrent browser instances to use. For shared runners on GitLab.com, we recommended a maximum of three. Private runners with more resources may benefit from a higher number, but are likely to produce little benefit after five to seven instances. | | `DAST_BROWSER_PAGE_LOADING_SELECTOR` | selector | `css:#page-is-loading` | Selector that when is no longer visible on the page, indicates to the analyzer that the page has finished loading and the scan can continue. Cannot be used with `DAST_BROWSER_PAGE_READY_SELECTOR`. | | `DAST_BROWSER_PAGE_READY_SELECTOR` | selector | `css:#page-is-ready` | Selector that when detected as visible on the page, indicates to the analyzer that the page has finished loading and the scan can continue. Cannot be used with `DAST_BROWSER_PAGE_LOADING_SELECTOR`. | +| `DAST_BROWSER_PASSIVE_CHECK_WORKERS` | int | `5` | Number of workers that passive scan in parallel. Recommend setting to the number of available CPUs. | | `DAST_BROWSER_SCAN` | boolean | `true` | Required to be `true` to run a browser-based scan. | | `DAST_BROWSER_SEARCH_ELEMENT_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `3s` | The maximum amount of time to allow the browser to search for new elements or user actions. | | `DAST_BROWSER_STABILITY_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `7s` | The maximum amount of time to wait for a browser to consider a page loaded and ready for analysis. | | `DAST_EXCLUDE_RULES` | string | `10020,10026` | Set to a comma-separated list of ZAP Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). | | `DAST_EXCLUDE_URLS` | URLs | `https://example.com/.*/sign-out` | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. | +| `DAST_FF_ENABLE_BAS` | boolean | `true` | Set to `true` to [enable Breach and Attack Simulation](../breach_and_attack_simulation/index.md#extend-dynamic-application-security-testing-dast) during this DAST scan. | | `DAST_FULL_SCAN_ENABLED` | boolean | `true` | Set to `true` to run both passive and active checks. Default: `false` | | `DAST_PATHS` | string | `/page1.html,/category1/page3.html` | Set to a comma-separated list of URL paths relative to `DAST_WEBSITE` for DAST to scan. | | `DAST_PATHS_FILE` | string | `/builds/project/urls.txt` | Set to a file path containing a list of URL paths relative to `DAST_WEBSITE` for DAST to scan. The file must be plain text with one path per line. | @@ -232,9 +236,14 @@ This can come at a cost of increased scan time. You can manage the trade-off between coverage and scan time with the following measures: +- Vertically scale the runner and use a higher number of browsers with the [variable](#available-cicd-variables) `DAST_BROWSER_NUMBER_OF_BROWSERS`. The default is `3`. - Limit the number of actions executed by the browser with the [variable](#available-cicd-variables) `DAST_BROWSER_MAX_ACTIONS`. The default is `10,000`. - Limit the page depth that the browser-based crawler checks coverage on with the [variable](#available-cicd-variables) `DAST_BROWSER_MAX_DEPTH`. The crawler uses a breadth-first search strategy, so pages with smaller depth are crawled first. The default is `10`. -- Vertically scale the runner and use a higher number of browsers with [variable](#available-cicd-variables) `DAST_BROWSER_NUMBER_OF_BROWSERS`. The default is `3`. +- Limit the time taken to crawl the target application with the [variable](#available-cicd-variables) `DAST_BROWSER_CRAWL_TIMEOUT`. The default is `24h`. Scans continue with passive and active checks when the crawler times out. +- Build the crawl graph with the [variable](#available-cicd-variables) `DAST_BROWSER_CRAWL_GRAPH` to see what pages are being crawled. +- Prevent pages from being crawled using the [variable](#available-cicd-variables) `DAST_EXCLUDE_URLS`. +- Prevent elements being selected using the [variable](#available-cicd-variables) `DAST_BROWSER_EXCLUDED_ELEMENTS`. Use with caution, as defining this variable causes an extra lookup for each page crawled. +- If the target application has minimal or fast rendering, consider reducing the [variable](#available-cicd-variables) `DAST_BROWSER_DOM_READY_AFTER_TIMEOUT` to a smaller value. The default is `500ms`. ## Timeouts diff --git a/doc/user/application_security/dast/dast_troubleshooting.md b/doc/user/application_security/dast/dast_troubleshooting.md index da382920604..c2e7f153e02 100644 --- a/doc/user/application_security/dast/dast_troubleshooting.md +++ b/doc/user/application_security/dast/dast_troubleshooting.md @@ -70,7 +70,7 @@ tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/20 ## Getting warning message `gl-dast-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ## Getting error `dast job: chosen stage does not exist` when including DAST CI template diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 3cb8967afd2..1f4506a22e5 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -15,6 +15,9 @@ visible from the source code. Dynamic Application Security Testing (DAST) examines applications for vulnerabilities like these in deployed environments. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Dynamic Application Security Testing (DAST)](https://www.youtube.com/watch?v=nbeDUoLZJTo). + NOTE: To learn how four of the top six attacks were application-based and how to protect your organization, download our diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md index 49d5859be45..1ebb5c8af0d 100644 --- a/doc/user/application_security/dast_api/index.md +++ b/doc/user/application_security/dast_api/index.md @@ -998,6 +998,14 @@ repository's root as `.gitlab/gitlab-dast-api-config.yml`. The following profiles are pre-defined in the default configuration file. Profiles can be added, removed, and modified by creating a custom configuration. +#### Passive + +- Application Information Check +- Cleartext Authentication Check +- JSON Hijacking Check +- Sensitive Information Check +- Session Cookie Check + #### Quick - Application Information Check diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md index f46c5e5e801..e02cc5929a6 100644 --- a/doc/user/application_security/dependency_list/index.md +++ b/doc/user/application_security/dependency_list/index.md @@ -13,6 +13,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w Use the dependency list to review your project's dependencies and key details about those dependencies, including their known vulnerabilities. It is a collection of dependencies in your project, including existing and new findings. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Project Dependency](https://www.youtube.com/watch?v=ckqkn9Tnbw4). + To see the dependency list, go to your project and select **Security and Compliance > Dependency list**. This information is sometimes referred to as a Software Bill of Materials, SBOM, or BOM. diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 7641c39d6d6..86e6264fab4 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -35,6 +35,9 @@ vulnerability.  +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Dependency Scanning](https://www.youtube.com/watch?v=TBnfbGk4c4o). + ## Dependency Scanning compared to Container Scanning GitLab offers both Dependency Scanning and Container Scanning @@ -212,8 +215,8 @@ table.supported-languages ul { <td>N</td> </tr> <tr> - <td rowspan="2">JavaScript and TypeScript</td> - <td>All versions</td> + <td rowspan="3">JavaScript and TypeScript</td> + <td rowspan="3">All versions</td> <td><a href="https://www.npmjs.com/">npm</a></td> <td> <ul> @@ -224,12 +227,16 @@ table.supported-languages ul { <td>Y</td> </tr> <tr> - <td>All versions</td> <td><a href="https://classic.yarnpkg.com/en/">yarn</a></td> <td><code>yarn.lock</code></td> <td>Y</td> </tr> <tr> + <td><a href="https://pnpm.io/">pnpm</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-4">4</a></b></sup></td> + <td><code>pnpm-lock.yaml</code></td> + <td>Y</td> + </tr> + <tr> <td>PHP</td> <td>All versions</td> <td><a href="https://getcomposer.org/">Composer</a></td> @@ -258,14 +265,14 @@ table.supported-languages ul { <td><a href="https://pipenv.pypa.io/en/latest/">Pipenv</a></td> <td> <ul> - <li><a href="https://pipenv.pypa.io/en/latest/basics/#example-pipfile-pipfile-lock"><code>Pipfile</code></a></li> - <li><a href="https://pipenv.pypa.io/en/latest/basics/#example-pipfile-pipfile-lock"><code>Pipfile.lock</code></a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-4">4</a></b></sup></li> + <li><a href="https://pipenv.pypa.io/en/latest/pipfile/#example-pipfile"><code>Pipfile</code></a></li> + <li><a href="https://pipenv.pypa.io/en/latest/pipfile/#example-pipfile-lock"><code>Pipfile.lock</code></a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-5">5</a></b></sup></li> </ul> </td> <td>N</td> </tr> <tr> - <td><a href="https://python-poetry.org/">Poetry</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-5">5</a></b></sup></td> + <td><a href="https://python-poetry.org/">Poetry</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-6">6</a></b></sup></td> <td><code>poetry.lock</code></td> <td>N</td> </tr> @@ -284,7 +291,7 @@ table.supported-languages ul { <tr> <td>Scala</td> <td>All versions</td> - <td><a href="https://www.scala-sbt.org/">sbt</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-6">6</a></b></sup></td> + <td><a href="https://www.scala-sbt.org/">sbt</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-7">7</a></b></sup></td> <td><code>build.sbt</code></td> <td>N</td> </tr> @@ -313,6 +320,12 @@ table.supported-languages ul { <li> <a id="notes-regarding-supported-languages-and-package-managers-4"></a> <p> + Support for <code>pnpm</code> lockfiles was <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/336809">introduced in GitLab 15.11</a>. <code>pnpm</code> lockfiles do not store bundled dependencies, so the reported dependencies may differ from <code>npm</code> or <code>yarn</code>. + </p> + </li> + <li> + <a id="notes-regarding-supported-languages-and-package-managers-5"></a> + <p> The presence of a <code>Pipfile.lock</code> file alone will <i>not</i> trigger the analyzer; the presence of a <code>Pipfile</code> is still required in order for the analyzer to be executed. However, if a <code>Pipfile.lock</code> file is found, it is used by <code>Gemnasium</code> to scan the exact package versions listed in this file. @@ -324,7 +337,7 @@ table.supported-languages ul { </p> </li> <li> - <a id="notes-regarding-supported-languages-and-package-managers-5"></a> + <a id="notes-regarding-supported-languages-and-package-managers-6"></a> <p> Support for <a href="https://python-poetry.org/">Poetry</a> projects with a <code>poetry.lock</code> file was <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/7006">added in GitLab 15.0</a>. Support for projects without a <code>poetry.lock</code> file is tracked in issue: @@ -332,7 +345,7 @@ table.supported-languages ul { </p> </li> <li> - <a id="notes-regarding-supported-languages-and-package-managers-6"></a> + <a id="notes-regarding-supported-languages-and-package-managers-7"></a> <p> Support for <a href="https://www.scala-sbt.org/">sbt</a> 1.3 and above was added in GitLab 13.9. </p> @@ -359,7 +372,8 @@ The following package managers use lockfiles that GitLab analyzers are capable o | Go | Not applicable | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/go-modules/gosum/default/go.sum) <sup><strong><a href="#notes-regarding-parsing-lockfiles-1">1</a></strong></sup> | | NuGet | v1 | [4.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/csharp-nuget-dotnetcore/default/src/web.api/packages.lock.json#L2) | | npm | v1, v2, v3<sup><b><a href="#notes-regarding-parsing-lockfiles-2">2</a></b></sup> | [6.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/default/package-lock.json#L4), [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/lockfileVersion2/package-lock.json#L4), [9.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/scanner/parser/npm/fixtures/lockfile-v3/simple/package-lock.json#L4) | -| yarn | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/default/yarn.lock#L2) | +| pnpm | v5.3, v5.4, v6 | [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-pnpm/default/pnpm-lock.yaml#L1), [8.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/scanner/parser/pnpm/fixtures/v6/simple/pnpm-lock.yaml#L1) | +| yarn | v1, v2<sup><b><a href="#notes-regarding-parsing-lockfiles-3">3</a></b></sup>, v3<sup><b><a href="#notes-regarding-parsing-lockfiles-3">3</a></b></sup> | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/classic/default/yarn.lock#L2), [2.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/berry/v2/default/yarn.lock), [3.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/berry/v3/default/yarn.lock) | | Poetry | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/python-poetry/default/poetry.lock) | <!-- markdownlint-disable MD044 --> @@ -377,6 +391,26 @@ The following package managers use lockfiles that GitLab analyzers are capable o Support for <code>lockfileVersion = 3</code> was <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/365176">introduced</a> in GitLab 15.7. </p> </li> + <li> + <a id="notes-regarding-parsing-lockfiles-3"></a> + <p> + Support for Yarn <code>v2</code> and <code>v3</code> was <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/263358">introduced in GitLab 15.11</a>. However, this feature is also available to versions of GitLab 15.0 and later. + </p> + <p> + The following features are not supported for Yarn <code>v2</code> or <code>v3</code>: + </p> + <ul> + <li> + <a href="https://yarnpkg.com/features/workspaces">workspaces</a> + </li> + <li> + <a href="https://yarnpkg.com/cli/patch">yarn patch</a> + </li> + </ul> + <p> + Yarn files that contain a patch, a workspace, or both, are still processed, but these features are ignored. + </p> + </li> </ol> <!-- markdownlint-enable MD044 --> @@ -402,7 +436,7 @@ To support the following package managers, the GitLab analyzers proceed in two s <li> <a id="exported-dependency-information-notes-1"></a> <p> - This test uses the default version of <code>maven</code> specified by the <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v3.10.6/build/gemnasium-maven/debian/config/.tool-versions#L3">.tool-versions</a> file. + This test uses the default version of <code>maven</code> specified by the <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v3.10.6/build/gemnasium-maven/debian/config/.tool-versions#L3">`.tool-versions`</a> file. </p> </li> <li> @@ -872,7 +906,7 @@ Here's an example dependency scanning report: ### CycloneDX Software Bill of Materials -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350509) in GitLab 14.8 in [Beta](../../../policy/alpha-beta-support.md#beta-features). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350509) in GitLab 14.8 in [Beta](../../../policy/alpha-beta-support.md#beta). > - Generally available in GitLab 15.7. In addition to the [JSON report file](#reports-json-format), the [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) @@ -913,7 +947,7 @@ Then the Gemnasium scanner generates the following CycloneDX SBOMs: └── gl-sbom-go-go.cdx.json ``` -The CycloneDX SBOMs can be downloaded [the same way as other job artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +You can download CycloneDX SBOMs [the same way as other job artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts). ### Merging multiple CycloneDX SBOMs @@ -1213,7 +1247,7 @@ affected. Read more in ### Getting warning message `gl-dependency-scanning-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ### Limitation when using rules:exists diff --git a/doc/user/application_security/iac_scanning/index.md b/doc/user/application_security/iac_scanning/index.md index 937aaf76280..f74a1a1f0da 100644 --- a/doc/user/application_security/iac_scanning/index.md +++ b/doc/user/application_security/iac_scanning/index.md @@ -261,7 +261,7 @@ The IaC tool emits a JSON report file in the existing SAST report format. For mo [schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json). The JSON report file can be downloaded from the CI pipelines page, or the -pipelines tab on merge requests by [setting `artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. For more information see [Downloading artifacts](../../../ci/pipelines/job_artifacts.md). +pipelines tab on merge requests by [setting `artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. For more information see [Downloading artifacts](../../../ci/jobs/job_artifacts.md). ## Troubleshooting diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 492b200d22b..32a4b0f4318 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -101,7 +101,7 @@ The following vulnerability scanners and their databases are regularly updated: | Secure scanning tool | Vulnerabilities database updates | |:----------------------------------------------------------------|:---------------------------------| -| [Container Scanning](container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. For more details, see [Vulnerabilities database update](container_scanning/index.md#vulnerabilities-database). | +| [Container Scanning](container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. GitLab monitors this job through an internal alert that tells the engineering team when the database becomes more than 48 hours old. For more information, see the [Vulnerabilities database update](container_scanning/index.md#vulnerabilities-database). | | [Dependency Scanning](dependency_scanning/index.md) | Relies on the [GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db). It is updated on a daily basis using [data from NVD, the `ruby-advisory-db` and the GitHub Advisory Database as data sources](https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/SOURCES.md). See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | | [Dynamic Application Security Testing (DAST)](dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/main/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | | [Static Application Security Testing (SAST)](sast/index.md) | The source of scan rules depends on which [analyzer](sast/analyzers.md) is used for each [supported programming language](sast/index.md#supported-languages-and-frameworks). GitLab maintains a ruleset for the Semgrep-based analyzer and updates it regularly based on internal research and user feedback. For other analyzers, the ruleset is sourced from the upstream open-source scanner. Each analyzer is updated at least once per month if a relevant update is available. | @@ -577,7 +577,7 @@ Debug logging can be a serious security risk. The output may contain the content environment variables and other secrets available to the job. The output is uploaded to the GitLab server and visible in job logs. -This message is often followed by the [error `No files to upload`](../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload), +This message is often followed by the [error `No files to upload`](../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload), and preceded by other errors or warnings that indicate why the JSON report wasn't generated. Check the entire job log for such messages. If you don't find these messages, retry the failed job after setting `SECURE_LOG_LEVEL: "debug"` as a [custom CI/CD variable](../../ci/variables/index.md#for-a-project). diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 5a075bf731b..1ee4d9b2a19 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -121,7 +121,7 @@ include: ``` The pipeline downloads the Docker images needed for the Security Scanners and saves them as -[job artifacts](../../../ci/pipelines/job_artifacts.md) or pushes them to the [Container Registry](../../packages/container_registry/index.md) +[job artifacts](../../../ci/jobs/job_artifacts.md) or pushes them to the [Container Registry](../../packages/container_registry/index.md) of the project where the pipeline is executed. These archives can be transferred to another location and [loaded](https://docs.docker.com/engine/reference/commandline/load/) in a Docker daemon. This method requires a runner with access to both `gitlab.com` (including diff --git a/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_11.png b/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_11.png Binary files differnew file mode 100644 index 00000000000..76b9a971172 --- /dev/null +++ b/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_11.png diff --git a/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_9.png b/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_9.png Binary files differdeleted file mode 100644 index 57e729158da..00000000000 --- a/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_9.png +++ /dev/null diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 1f53b671ed1..05996a70d3d 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -24,8 +24,12 @@ GitLab supports the following security policies: All security policies are stored as YAML in a separate security policy project that gets linked to the development project, group, or sub-group. This association can be a one-to-many relationship, allowing one security -policy project to apply to multiple development projects, groups, or sub-groups. Linked projects are not required to be in -the same group as the development projects to which they are linked. +policy project to apply to multiple development projects, groups, or sub-groups: + +- For self-managed GitLab instances, linked projects are not required to be in the same group + or the same subgroup as the development projects to which they are linked. +- For GitLab SaaS, the security policy project is required to be in the same top-level group + as the development project, although, it is not necessary for the project to be in the same subgroup.  @@ -34,10 +38,6 @@ project and the security policy project, this is not recommended. Keeping the se project separate from the development project allows for complete separation of duties between security/compliance teams and development teams. -You should not link a security policy project to a development project and to the group -or sub-group the development project belongs to at the same time. Linking this way will result in -approval rules from the Scan Result Policy not being applied to merge requests in the development project. - All security policies are stored in the `.gitlab/security-policies/policy.yml` YAML file inside the linked security policy project. The format for this YAML is specific to the type of policy that is stored there. Examples and schema information are available for the following policy types: diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index 96048bb2308..55a4994e168 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -44,7 +44,7 @@ Most policy changes take effect as soon as the merge request is merged. Any chan do not go through a merge request and are committed directly to the default branch may require up to 10 minutes before the policy changes take effect. - + ## Scan execution policies schema diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md index 104462529c1..f54b77bd45d 100644 --- a/doc/user/application_security/policies/scan-result-policies.md +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -87,11 +87,8 @@ This rule enforces the defined actions based on security scan findings. ## `license_finding` rule type -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8092) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `license_scanning_policies`. Enabled by default in GitLab 15.10. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `license_scanning_policies`. -On GitLab.com, this feature is not available. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8092) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `license_scanning_policies`. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/397644) in GitLab 15.11. Feature flag `license_scanning_policies` removed. This rule enforces the defined actions based on license findings. @@ -116,6 +113,7 @@ the defined policy. | `user_approvers_ids` | `array` of `integer` | ID of one of more users | The IDs of users to consider as approvers. Users must have access to the project to be eligible to approve. | | `group_approvers` | `array` of `string` | Path of one of more groups | The groups to consider as approvers. Users with [direct membership in the group](../../project/merge_requests/approvals/rules.md#group-approvers) are eligible to approve. | | `group_approvers_ids` | `array` of `integer` | ID of one of more groups | The IDs of groups to consider as approvers. Users with [direct membership in the group](../../project/merge_requests/approvals/rules.md#group-approvers) are eligible to approve. | +| `role_approvers` | `array` of `string` | One or more [roles](../../../user/permissions.md#roles) (for example: `owner`, `maintainer`) | The roles to consider as approvers. | Requirements and limitations: @@ -171,6 +169,8 @@ scan_result_policy: approvals_required: 1 user_approvers: - sam.white + role_approvers: + - owner ``` In this example: diff --git a/doc/user/application_security/sast/customize_rulesets.md b/doc/user/application_security/sast/customize_rulesets.md index d070883df9a..b45b25db69f 100644 --- a/doc/user/application_security/sast/customize_rulesets.md +++ b/doc/user/application_security/sast/customize_rulesets.md @@ -323,7 +323,7 @@ With the following custom ruleset configuration, vulnerabilities found with [semgrep] [[semgrep.ruleset]] [semgrep.ruleset.identifier] - type = "CWE" + type = "cwe" value = "322" [semgrep.ruleset.override] severity = "Critical" diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index a0c5adc89f5..13f05a63181 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -79,20 +79,20 @@ For more information about our plans for language support in SAST, see the [cate |------------------------------|--------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------| | .NET Core | [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | 11.0 | | .NET Framework<sup>1</sup> | [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | 13.0 | -| .NET (all versions, C# only) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 15.4 | +| .NET (all versions, C# only) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 15.4 | | Apex (Salesforce) | [PMD](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) | 12.1 | -| C | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 14.2 | +| C | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 14.2 | | C/C++ | [Flawfinder](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) | 10.7 | | Elixir (Phoenix) | [Sobelow](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) | 11.1 | | Go<sup>3</sup> | [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) | 10.7 | -| Go | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 14.4 | +| Go | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 14.4 | | Groovy<sup>2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 11.3 (Gradle) & 11.9 (Maven, SBT) | | Helm Charts | [Kubesec](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) | 13.1 | -| Java (any build system) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 14.10 | +| Java (any build system) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 14.10 | | Java<sup>2, 3</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (SBT) | | Java (Android) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 | | JavaScript<sup>3</sup> | [ESLint security plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 11.8 | -| JavaScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.10 | +| JavaScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 13.10 | | Kotlin (Android) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 | | Kotlin (General)<sup>2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 13.11 | | Kubernetes manifests | [Kubesec](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) | 12.6 | @@ -100,15 +100,15 @@ For more information about our plans for language support in SAST, see the [cate | Objective-C (iOS) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 | | PHP | [phpcs-security-audit](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) | 10.8 | | Python<sup>3</sup> | [bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) | 10.3 | -| Python | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.9 | +| Python | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 13.9 | | React<sup>3</sup> | [ESLint react plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 12.5 | -| React | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.10 | +| React | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 13.10 | | Ruby | [brakeman](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman) | 13.9 | | Ruby on Rails | [brakeman](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman) | 10.3 | | Scala<sup>2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 11.0 (SBT) & 11.9 (Gradle, Maven) | | Swift (iOS) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 | | TypeScript<sup>3</sup> | [ESLint security plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 11.9, [merged](https://gitlab.com/gitlab-org/gitlab/-/issues/36059) with ESLint in 13.2 | -| TypeScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.10 | +| TypeScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 13.10 | 1. .NET 4 support is limited. The analyzer runs in a Linux container and does not have access to Windows-specific libraries or features. Use the Semgrep-based scanner if you need .NET 4 support. 1. The SpotBugs-based analyzer supports [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/), and [SBT](https://www.scala-sbt.org/). It can also be used with variants like the @@ -238,7 +238,7 @@ as shown in the following table: | See new findings in merge request widget | **{dotted-circle}** | **{check-circle}** | | [Manage vulnerabilities](../vulnerabilities/index.md) | **{dotted-circle}** | **{check-circle}** | | [Access the Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | -| [Configure SAST in the UI](#configure-sast-in-the-ui) | **{dotted-circle}** | **{check-circle}** | +| [Configure SAST by using the UI](#configure-sast-by-using-the-ui) | **{dotted-circle}** | **{check-circle}** | | [Customize SAST rulesets](customize_rulesets.md) | **{dotted-circle}** | **{check-circle}** | | [Detect False Positives](#false-positive-detection) | **{dotted-circle}** | **{check-circle}** | | [Track moved vulnerabilities](#advanced-vulnerability-tracking) | **{dotted-circle}** | **{check-circle}** | @@ -257,7 +257,7 @@ To configure SAST for a project you can: - Use [Auto SAST](../../../topics/autodevops/stages.md#auto-sast), provided by [Auto DevOps](../../../topics/autodevops/index.md). - [Configure SAST in your CI/CD YAML](#configure-sast-in-your-cicd-yaml). -- [Configure SAST using the UI](#configure-sast-in-the-ui) (introduced in GitLab 13.3). +- [Configure SAST by using the UI](#configure-sast-by-using-the-ui). You can enable SAST across many projects by [enforcing scan execution](../index.md#enforce-scan-execution). @@ -282,15 +282,12 @@ The results are saved as a that you can later download and analyze. When downloading, you always receive the most recent SAST artifact available. -### Configure SAST in the UI +### Configure SAST by using the UI -You can enable and configure SAST in the UI, either with default settings, or with customizations. +You can enable and configure SAST by using the UI, either with the default settings or with customizations. The method you can use depends on your GitLab license tier. -- [Configure SAST in the UI with customizations](#configure-sast-in-the-ui-with-customizations). **(ULTIMATE)** -- [Configure SAST in the UI with default settings only](#configure-sast-in-the-ui-with-default-settings-only). - -### Configure SAST in the UI with customizations **(ULTIMATE)** +#### Configure SAST with customizations **(ULTIMATE)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3659) in GitLab 13.3. > - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/232862) in GitLab 13.4. @@ -319,7 +316,7 @@ To enable and configure SAST with customizations: Pipelines now include a SAST job. -### Configure SAST in the UI with default settings only +#### Configure SAST with default settings only > [Introduced](https://about.gitlab.com/releases/2021/02/22/gitlab-13-9-released/#security-configuration-page-for-all-users) in GitLab 13.9 @@ -665,7 +662,7 @@ To download the report file, you can either: - Download the file from the CI/CD pipelines page. - In the pipelines tab on merge requests, set [`artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. -For information, see [Download job artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +For information, see [Download job artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts). For details of the report file's schema, see [SAST report file schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json). @@ -820,7 +817,7 @@ affected. Read more in ### Getting warning message `gl-sast-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ### Error: `sast is used for configuration only, and its script should not be executed` diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 6dd20ea094e..49bab0b3b29 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -59,6 +59,7 @@ Different features are available in different [GitLab tiers](https://about.gitla | [Configure Secret Detection scanner](#enable-secret-detection) | **{check-circle}** Yes | **{check-circle}** Yes | | [Customize Secret Detection settings](#configure-scan-settings) | **{check-circle}** Yes | **{check-circle}** Yes | | Download [JSON Report](../sast/index.md#reports-json-format) | **{check-circle}** Yes | **{check-circle}** Yes | +| [Check text for potential secrets](#warnings-for-potential-leaks-in-text-content) before it's posted | **{check-circle}** Yes | **{check-circle}** Yes | | See new findings in the merge request widget | **{dotted-circle}** No | **{check-circle}** Yes | | View identified secrets in the pipelines' **Security** tab | **{dotted-circle}** No | **{check-circle}** Yes | | [Manage vulnerabilities](../vulnerability_report/index.md) | **{dotted-circle}** No | **{check-circle}** Yes | @@ -535,6 +536,26 @@ variable, or as a CI/CD variable. - If using a variable, set the value of `ADDITIONAL_CA_CERT_BUNDLE` to the text representation of the certificate. +## Warnings for potential leaks in text content + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368434) in GitLab 15.11. + +When you create an issue, propose a merge request, or write a comment, you might accidentally post a sensitive value. +For example, you might paste in the details of an API request or an environment variable that contains an authentication token. + +GitLab checks if the text of your issue description, merge request description, comment, or reply contains a sensitive token. +If a token is found, a warning message is displayed. You can then edit your message before posting it. +This check happens in your browser before the message is sent to the server. +The check is always on; you don't have to set it up. + +Your text is checked for the following secret types: + +- GitLab [personal access tokens](../../../security/token_overview.md#personal-access-tokens) +- GitLab [feed tokens](../../../security/token_overview.md#feed-token) + +This feature is separate from Secret Detection scanning, which checks your Git repository for leaked secrets. +[Issue 405147](https://gitlab.com/gitlab-org/gitlab/-/issues/405147) tracks efforts to align these two types of protection. + ## Troubleshooting ### Set the logging level @@ -555,7 +576,7 @@ visible in job logs. ### Warning: `gl-secret-detection-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ### Error: `Couldn't run the gitleaks command: exit status 2` diff --git a/doc/user/application_security/secret_detection/post_processing.md b/doc/user/application_security/secret_detection/post_processing.md index 22d7a8ba5af..b05b272b0b4 100644 --- a/doc/user/application_security/secret_detection/post_processing.md +++ b/doc/user/application_security/secret_detection/post_processing.md @@ -4,7 +4,7 @@ group: Static Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Secret Detection post-processing and revocation **(ULTIMATE SAAS)** +# Secret Detection post-processing and revocation **(ULTIMATE)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4639) in GitLab 13.6. > - [Disabled by default for GitLab personal access tokens](https://gitlab.com/gitlab-org/gitlab/-/issues/371658) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `gitlab_pat_auto_revocation`. Available to GitLab.com only. @@ -32,53 +32,25 @@ GitLab supports post-processing for the following vendors and secrets: ## Feature availability +> [Enabled for non-default branches](https://gitlab.com/gitlab-org/gitlab/-/issues/299212) in GitLab 15.11. + Credentials are only post-processed when Secret Detection finds them: - In public projects, because publicly exposed credentials pose an increased threat. Expansion to private projects is considered in [issue 391379](https://gitlab.com/gitlab-org/gitlab/-/issues/391379). -- On the project [default branch](../../project/repository/branches/default.md), for technical reasons. Expansion to all branches is tracked in [issue 299212](https://gitlab.com/gitlab-org/gitlab/-/issues/299212). - In projects with GitLab Ultimate, for technical reasons. Expansion to all tiers is tracked in [issue 391763](https://gitlab.com/gitlab-org/gitlab/-/issues/391763). -## High-level architecture - -This diagram describes how a post-processing hook revokes a secret within the GitLab application: - -```mermaid -sequenceDiagram - autonumber - GitLab Rails->>+Sidekiq: gl-secret-detection-report.json - Sidekiq-->+Sidekiq: StoreSecurityReportsWorker - Sidekiq-->+Token Revocation API: GET revocable keys types - Token Revocation API-->>-Sidekiq: OK - Sidekiq->>+Token Revocation API: POST revoke revocable keys - Token Revocation API-->>-Sidekiq: ACCEPTED - Token Revocation API-->>+Receiver Service: revoke revocable keys - Receiver Service-->>+Token Revocation API: ACCEPTED -``` - -1. A pipeline with a Secret Detection job completes on the project's default branch, producing a scan - report (**1**). -1. The report is processed (**2**) by an asynchronous worker, which communicates with an externally - deployed HTTP service (**3** and **4**) to determine which kinds of secrets can be automatically - revoked. -1. The worker sends (**5** and **6**) the list of detected secrets which the Token Revocation API is able to - revoke. -1. The Token Revocation API sends (**7** and **8**) each revocable token to their respective vendor's [receiver service](#integrate-your-cloud-provider-service-with-gitlabcom). - -See the [Token Revocation API](../../../development/sec/token_revocation_api.md) documentation for more -information. - -## Integrate your cloud provider service with GitLab.com +## Partner program for leaked-credential notifications -Third-party cloud and SaaS vendors interested in automated token revocation can -[express integration interest by filling out this form](https://forms.gle/wWpvrtLRK21Q2WJL9). -Vendors must [implement a revocation receiver service](#implement-a-revocation-receiver-service) -which will be called by the Token Revocation API. +GitLab notifies partners when credentials they issue are leaked in public repositories on GitLab.com. +If you operate a cloud or SaaS product and you're interested in receiving these notifications, learn more in [epic 4944](https://gitlab.com/groups/gitlab-org/-/epics/4944). +Partners must [implement a revocation receiver service](#implement-a-revocation-receiver-service), +which is called by the Token Revocation API. ### Implement a revocation receiver service A revocation receiver service integrates with a GitLab instance's Token Revocation API to receive and respond to leaked token revocation requests. The service should be a publicly accessible HTTP API that is -idempotent and rate-limited. Requests to your service from the Token Revocation API will follow the example +idempotent and rate-limited. Requests to your service from the Token Revocation API look similar to the example below: ```plaintext @@ -95,3 +67,32 @@ X-Gitlab-Token: MYSECRETTOKEN In this example, Secret Detection has determined that an instance of `my_api_token` has been leaked. The value of the token is provided to you, in addition to a publicly accessible URL to the raw content of the file containing the leaked token. + +## High-level architecture + +This diagram describes how a post-processing hook revokes a secret in the GitLab application: + +```mermaid +sequenceDiagram + autonumber + GitLab Rails-->+GitLab Rails: gl-secret-detection-report.json + GitLab Rails->>+Sidekiq: StoreScansService + Sidekiq-->+Sidekiq: ScanSecurityReportSecretsWorker + Sidekiq-->+Token Revocation API: GET revocable keys types + Token Revocation API-->>-Sidekiq: OK + Sidekiq->>+Token Revocation API: POST revoke revocable keys + Token Revocation API-->>-Sidekiq: ACCEPTED + Token Revocation API-->>+Receiver Service: revoke revocable keys + Receiver Service-->>+Token Revocation API: ACCEPTED +``` + +1. A pipeline with a Secret Detection job completes, producing a scan report (**1**). +1. The report is processed (**2**) by a service class, which schedules an asynchronous worker if token revocation is possible. +1. The asynchronous worker (**3**) communicates with an externally deployed HTTP service + (**4** and **5**) to determine which kinds of secrets can be automatically revoked. +1. The worker sends (**6** and **7**) the list of detected secrets which the Token Revocation API is able to + revoke. +1. The Token Revocation API sends (**8** and **9**) each revocable token to their respective vendor's [receiver service](#implement-a-revocation-receiver-service). + +See the [Token Revocation API](../../../development/sec/token_revocation_api.md) documentation for more +information. diff --git a/doc/user/application_security/terminology/index.md b/doc/user/application_security/terminology/index.md index 772a7d17e1e..df103976901 100644 --- a/doc/user/application_security/terminology/index.md +++ b/doc/user/application_security/terminology/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Secure and Govern terminology **(FREE)** +# Secure and Govern glossary **(FREE)** The glossary of terms aims to achieve the following: @@ -17,9 +17,7 @@ The glossary of terms aims to achieve the following: The definitions of the terms outlined in this document are in the context of the GitLab products. Therefore, a term may have a different meaning to users outside of GitLab. -## Terms - -### Analyzer +## Analyzer Software that performs a scan. The scan analyzes an attack surface for vulnerabilities and produces a report containing findings. Reports adhere to the [Secure report format](#secure-report-format). @@ -31,35 +29,35 @@ manage found vulnerabilities. For more information, see [Security Scanner Integr Many GitLab analyzers follow a standard approach using Docker to run a wrapped scanner. For example, the image `semgrep` is an analyzer that wraps the scanner `Semgrep`. -### Attack surface +## Attack surface The different places in an application that are vulnerable to attack. Secure products discover and search the attack surface during scans. Each product defines the attack surface differently. For example, SAST uses files and line numbers, and DAST uses URLs. -### Corpus +## Corpus The set of meaningful test cases that are generated while the fuzzer is running. Each meaningful test case produces new coverage in the tested program. It's advised to re-use the corpus and pass it to subsequent runs. -### CNA +## CNA [CVE](#cve) Numbering Authorities (CNAs) are organizations from around the world that are authorized by the [Mitre Corporation](https://cve.mitre.org/) to assign [CVE](#cve)s to vulnerabilities in products or services within their respective scope. [GitLab is a CNA](https://about.gitlab.com/security/cve/). -### CVE +## CVE Common Vulnerabilities and Exposures (CVE®) is a list of common identifiers for publicly known cybersecurity vulnerabilities. The list is managed by the [Mitre Corporation](https://cve.mitre.org/). -### CVSS +## CVSS The Common Vulnerability Scoring System (CVSS) is a free and open industry standard for assessing the severity of computer system security vulnerabilities. -### CWE +## CWE Common Weakness Enumeration (CWE™) is a community-developed list of common software and hardware weakness types that have security ramifications. Weaknesses are flaws, faults, bugs, @@ -68,22 +66,22 @@ architecture. If left unaddressed, weaknesses could result in systems, networks, vulnerable to attack. The CWE List and associated classification taxonomy serve as a language that you can use to identify and describe these weaknesses in terms of CWEs. -### Deduplication +## Deduplication When a category's process deems findings to be the same, or if they are similar enough that a noise reduction is required, only one finding is kept and the others are eliminated. Read more about the [deduplication process](../vulnerability_report/pipeline.md#deduplication-process). -### Duplicate finding +## Duplicate finding A legitimate finding that is reported multiple times. This can occur when different scanners discover the same finding, or when a single scan inadvertently reports the same finding more than once. -### False positive +## False positive A finding that doesn't exist but is incorrectly reported as existing. -### Finding +## Finding An asset that has the potential to be vulnerable, identified in a project by an analyzer. Assets include but are not restricted to source code, binary packages, containers, dependencies, networks, @@ -96,32 +94,32 @@ You can interact with vulnerability findings in two ways. 1. You can open an issue or merge request for the vulnerability finding. 1. You can dismiss the vulnerability finding. Dismissing the finding hides it from the default views. -### Grouping +## Grouping A flexible and non-destructive way to visually organize vulnerabilities in groups when there are multiple findings that are likely related but do not qualify for deduplication. For example, you can include findings that should be evaluated together, would be fixed by the same action, or come from the same source. Grouping behavior for vulnerabilities is under development and tracked in issue [267588](https://gitlab.com/gitlab-org/gitlab/-/issues/267588). -### Insignificant finding +## Insignificant finding A legitimate finding that a particular customer doesn't care about. -### Location fingerprint +## Location fingerprint A finding's location fingerprint is a text value that's unique for each location on the attack surface. Each security product defines this according to its type of attack surface. For example, SAST incorporates file path and line number. -### Package managers and package types +## Package managers and package types -#### Package managers +### Package managers A package manager is a system that manages your project dependencies. The package manager provides a method to install new dependencies (also referred to as "packages"), manage where packages are stored on your file system, and offer capabilities for you to publish your own packages. -#### Package types +### Package types Each package manager, platform, type, or ecosystem has its own conventions and protocols to identify, locate, and provision software packages. @@ -215,11 +213,11 @@ table.package-managers-and-types ul { </tbody> </table> -### Pipeline Security tab +## Pipeline Security tab A page that displays findings discovered in the associated CI pipeline. -### Post-filter +## Post-filter Post-filters help reduce noise in the scanner results and automate manual tasks. You can specify criteria that updates or modifies vulnerability data based on scanner results. For example, you can flag findings as likely False Positives @@ -228,7 +226,7 @@ and automatically resolve vulnerabilities that are no longer detected. These are Support for automatically resolving findings is tracked in epic [7478](https://gitlab.com/groups/gitlab-org/-/epics/7478) and support for cheap scan is proposed in issue [349926](https://gitlab.com/gitlab-org/gitlab/-/issues/349926). -### Pre-filter +## Pre-filter An irreversible action that is done to filter out targets before analysis occurs. This is usually provided to allow the user to reduce scope and noise as well as speed up the analysis. This should not be done if a record is needed as @@ -236,7 +234,7 @@ we currently do not store anything related to the skipped/excluded code or asset Examples: `DS_EXCLUDED_PATHS` should `Exclude files and directories from the scan based on the paths provided.` -### Primary identifier +## Primary identifier A finding's primary identifier is a value that is unique to each finding. The external type and external ID of the finding's [first identifier](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/v2.4.0-rc1/dist/sast-report-format.json#L228) @@ -246,13 +244,13 @@ Examples of primary identifiers include `PluginID` for OWASP Zed Attack Proxy (Z Trivy. The identifier must be stable. Subsequent scans must return the same value for the same finding, even if the location has slightly changed. -### Report finding +## Report finding A [finding](#finding) that only exists in a report produced by an analyzer, and is yet to be persisted to the database. The report finding becomes a [vulnerability finding](#vulnerability-finding) once it's imported into the database. -### Scan type (report type) +## Scan type (report type) Describes the type of scan. This must be one of the following: @@ -266,12 +264,12 @@ Describes the type of scan. This must be one of the following: This list is subject to change as scanners are added. -### Scanner +## Scanner Software that can scan for vulnerabilities. The resulting scan report is typically not in the [Secure report format](#secure-report-format). Examples include ESLint, Trivy, and ZAP. -### Secure product +## Secure product A group of features related to a specific area of application security with first-class support by GitLab. @@ -281,23 +279,23 @@ Testing (DAST), Secret Detection, Static Application Security Testing (SAST), an Each of these products typically include one or more analyzers. -### Secure report format +## Secure report format A standard report format that Secure products comply with when creating JSON reports. The format is described by a [JSON schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas). -### Security Dashboard +## Security Dashboard Provides an overview of all the vulnerabilities for a project, group, or GitLab instance. Vulnerabilities are only created from findings discovered on the project's default branch. -### Seed corpus +## Seed corpus The set of test cases given as initial input to the fuzz target. This usually speeds up the fuzz target substantially. This can be either manually created test cases or auto-generated with the fuzz target itself from previous runs. -### Vendor +## Vendor The party maintaining an analyzer. As such, a vendor is responsible for integrating a scanner into GitLab and keeping it compatible as they evolve. A vendor isn't necessarily the author or maintainer @@ -305,7 +303,7 @@ of the scanner, as in the case of using an open core or OSS project as a base so offering. For scanners included as part of a GitLab distribution or GitLab subscription, the vendor is listed as GitLab. -### Vulnerability +## Vulnerability A flaw that has a negative impact on the security of its environment. Vulnerabilities describe the error or weakness, and don't describe where the error is located (see [finding](#finding)). @@ -314,12 +312,12 @@ Each vulnerability maps to a unique finding. Vulnerabilities exist in the default branch. Findings (see [finding](#finding)) are all potential vulnerability items scanners identify in MRs/feature branches. Only after merging to default does a finding become a vulnerability. -### Vulnerability finding +## Vulnerability finding When a [report finding](#report-finding) is stored to the database, it becomes a vulnerability [finding](#finding). -### Vulnerability tracking +## Vulnerability tracking Deals with the responsibility of matching findings across scans so that a finding's life cycle can be understood. Engineers and security teams use this information to decide whether to merge code @@ -327,6 +325,6 @@ changes, and to see unresolved findings and when they were introduced. Vulnerabilities are tracked by comparing the location fingerprint, primary identifier, and report type. -### Vulnerability occurrence +## Vulnerability occurrence Deprecated, see [finding](#finding). diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 8107b9d8484..0cfeafe1fed 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -8,10 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13561) in GitLab 13.0. -Each vulnerability in a project has a Vulnerability Page, containing details of the -vulnerability. The details included vary according to the type of vulnerability. - -Details of each vulnerability include: +Each vulnerability in a project has a vulnerability page containing details of the vulnerability, +including: - Description - When it was detected @@ -20,17 +18,13 @@ Details of each vulnerability include: - Linked issues - Actions log -In GitLab 14.3 and later, if the scanner determined the vulnerability to be a false positive, an -alert message is included at the top of the vulnerability's page. - -On the vulnerability's page, you can: +If the scanner determined the vulnerability to be a false positive, an alert message is included at +the top of the vulnerability's page. -- [Change the vulnerability's status](#change-status-of-a-vulnerability). -- [Create an issue](#creating-an-issue-for-a-vulnerability). -- [Link issues to the vulnerability](#linking-a-vulnerability-to-issues). -- [Resolve the vulnerability](#resolve-a-vulnerability) if a solution is - available. -- [View security training specific to the detected vulnerability](#view-security-training-for-a-vulnerability). +When a vulnerability is no longer detected in a project's default branch, you should +change its status to **Resolved**. This ensures that if it is accidentally reintroduced in a future +merge, it is reported again as a new record. To change the status of multiple vulnerabilities, use +the Vulnerability Report's [Activity filter](../vulnerability_report/index.md#activity-filter). ## Vulnerability status values @@ -38,16 +32,24 @@ A vulnerability's status can be: - **Detected**: The default state for a newly discovered vulnerability. Appears as "Needs triage" in the UI. - **Confirmed**: A user has seen this vulnerability and confirmed it to be accurate. -- **Dismissed**: A user has seen this vulnerability and dismissed it because it is not accurate or otherwise not to be resolved. -- **Resolved**: The vulnerability has been fixed or is no longer present. +- **Dismissed**: A user has seen this vulnerability and dismissed it because it is not accurate or + otherwise not to be resolved. Dismissed vulnerabilities are ignored if detected in subsequent + scans. +- **Resolved**: The vulnerability has been fixed or is no longer present. Resolved vulnerabilities + that are reintroduced and detected by subsequent scans have a _new_ vulnerability record created. + +## Vulnerability dismissal reasons + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4942) in GitLab 15.11 with a feature flag named `dismissal_reason`. +> - Enabled on GitLab.com in GitLab 15.11. For self-managed customers, [contact Support](https://about.gitlab.com/support/) if you would like to use this feature in GitLab 15.11. -Dismissed vulnerabilities are ignored if detected in subsequent scans. Resolved vulnerabilities that -are reintroduced and detected by subsequent scans have a _new_ vulnerability record created. When an -existing vulnerability is no longer detected in a project's `default` branch, you should change its -status to **Resolved**. This ensures that if it is accidentally reintroduced in a future merge, it -is reported again as a new record. You can use the Vulnerability Report's -[Activity filter](../vulnerability_report/index.md#activity-filter) to select all vulnerabilities that are -no longer detected, and change their status. +When dismissing a vulnerability, one of the following reasons must be chosen to clarify why it is being dismissed: + +- **Acceptable risk**: The vulnerability is known, and has not been remediated or mitigated, but is considered to be an acceptable business risk. +- **False positive**: An error in reporting in which a test result incorrectly indicates the presence of a vulnerability in a system when the vulnerability is not present. +- **Mitigating control**: A management, operational, or technical control (that is, safeguard or countermeasure) employed by an organization that provides equivalent or comparable protection for an information system. +- **Used in tests**: The finding is not a vulnerability because it is part of a test or is test data. +- **Not applicable**: The vulnerability is known, and has not been remediated or mitigated, but is considered to be in a part of the application that will not be updated. ## Change status of a vulnerability @@ -57,9 +59,13 @@ To change a vulnerability's status from its Vulnerability Page: 1. On the left sidebar, select **Security and Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. From the **Status** dropdown list select a status, then select **Change status**. + + In GitLab 15.11 and later, you must select a [dismissal reason](#vulnerability-dismissal-reasons) when you change a vulnerability's status to **Dismissed**. + 1. Optionally, at the bottom of the page, add a comment to the log entry. -The Actions log records each status change along with which user changed the status and the time of the change. +Details of the status change, including who made the change and when, are recorded in the +vulnerability's action log. ## Creating an issue for a vulnerability diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md index e75d0a45f7d..ab90ac18b8e 100644 --- a/doc/user/application_security/vulnerabilities/severities.md +++ b/doc/user/application_security/vulnerabilities/severities.md @@ -56,7 +56,7 @@ the following tables: |------------------------------------------------------------------------------------------|------------------------------|----------------------------|-------------------------------------| | [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) | **{check-circle}** Yes | CVSS v2.0 Rating and CVSS v3.1 Qualitative Severity Rating <sup>1</sup> | `(AV:N/AC:L/Au:S/C:P/I:P/A:N)`, `CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H` | -1. The CVSS v3.1 rating is used to calculate the severity level. If it's not available, the CVSS v2.0 rating is used instead. +The CVSS v3.1 rating is used to calculate the severity level. If it's not available, the CVSS v2.0 rating is used instead. ## Container Scanning @@ -64,6 +64,8 @@ the following tables: |------------------------------------------------------------------------|--------------------------|----------------------------|--------------------------------------------------------------| | [`container-scanning`](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning)| **{check-circle}** Yes | String | `Unknown`, `Low`, `Medium`, `High`, `Critical` | +When available, the vendor severity level takes precedence and is used by the analyzer. If that is not available then it falls back on the CVSS v3.1 rating. If that is also not available, then the CVSS v2.0 rating is used instead. Details on this implementation are available on the respective issues for [trivy](https://github.com/aquasecurity/trivy/issues/310) and [grype](https://github.com/anchore/grype/issues/287). + ## Fuzz Testing All fuzz testing results are reported as Unknown. They should be reviewed and triaged manually to find exploitable faults to prioritize for fixing. diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index 3d7565f97bc..a4c4737f767 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -19,6 +19,9 @@ At all levels, the Vulnerability Report contains: - Filters for common vulnerability attributes. - Details of each vulnerability, presented in tabular layout. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Vulnerability Management](https://www.youtube.com/watch?v=8SJHz6BCgXM). + The **Activity** column contains icons to indicate the activity, if any, taken on the vulnerability in that row: diff --git a/doc/user/clusters/agent/ci_cd_workflow.md b/doc/user/clusters/agent/ci_cd_workflow.md index 982411d35f9..1ca558adbc2 100644 --- a/doc/user/clusters/agent/ci_cd_workflow.md +++ b/doc/user/clusters/agent/ci_cd_workflow.md @@ -1,13 +1,13 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Using GitLab CI/CD with a Kubernetes cluster **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327409) in GitLab 14.1. -> - The pre-configured `KUBECONFIG` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324275) in GitLab 14.2. +> - The pre-configured variable `$KUBECONFIG` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324275) in GitLab 14.2. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) the `ci_access` attribute in GitLab 14.3. > - The ability to authorize groups was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) to GitLab Free in 14.5. @@ -78,7 +78,8 @@ To authorize the agent to access the GitLab project where you keep Kubernetes ma - You can install additional agents into the same cluster to accommodate additional hierarchies. - You can authorize up to 100 projects. -All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +All CI/CD jobs now include a `kubeconfig` file with contexts for every shared agent connection. +The `kubeconfig` path is available in the environment variable `$KUBECONFIG`. Choose the context to run `kubectl` commands from your CI/CD scripts. ### Authorize the agent to access projects in your groups @@ -104,7 +105,8 @@ To authorize the agent to access all of the GitLab projects in a group or subgro - You can authorize up to 100 groups. All the projects that belong to the group and its subgroups are now authorized to access the agent. -All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +All CI/CD jobs now include a `kubeconfig` file with contexts for every shared agent connection. +The `kubeconfig` path is available in an environment variable `$KUBECONFIG`. Choose the context to run `kubectl` commands from your CI/CD scripts. ## Update your `.gitlab-ci.yml` file to run `kubectl` commands @@ -159,10 +161,10 @@ When you deploy to an environment that has both a - The certificate-based cluster's context is called `gitlab-deploy`. This context is always selected by default. -- In GitLab 14.9 and later, agent contexts are included in the - `KUBECONFIG`. You can select them by using `kubectl config use-context path/to/agent/repository:agent-name`. +- In GitLab 14.9 and later, agent contexts are included in `$KUBECONFIG`. + You can select them by using `kubectl config use-context path/to/agent/repository:agent-name`. - In GitLab 14.8 and earlier, you can still use agent connections, but for environments that - already have a certificate-based cluster, the agent connections are not included in the `KUBECONFIG`. + already have a certificate-based cluster, the agent connections are not included in `$KUBECONFIG`. To use an agent connection when certificate-based connections are present, you can manually configure a new `kubectl` configuration context. For example: @@ -183,6 +185,17 @@ deploy: # ... rest of your job configuration ``` +### Environments with KAS that use self-signed certificates + +If you use an environment with KAS and a self-signed certificate, you must configure your Kubernetes client to trust the certificate authority (CA) that signed your certificate. + +To configure your client, do one of the following: + +- Set a CI/CD variable `SSL_CERT_FILE` with the KAS certificate in PEM format. +- Configure the Kubernetes client with `--certificate-authority=$KAS_CERTIFICATE`, where `KAS_CERTIFICATE` is a CI/CD variable with the CA certificate of KAS. +- Place the certificates in an appropriate location in the job container by updating the container image or mounting via the runner. +- Not recommended. Configure the Kubernetes client with `--insecure-skip-tls-verify=true`. + ## Restrict project and group access by using impersonation **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345014) in GitLab 14.5. @@ -340,3 +353,16 @@ If you attempt to use `kubectl` without TLS, you might get an error like: $ kubectl get pods error: You must be logged in to the server (the server has asked for the client to provide credentials) ``` + +### Unable to connect to the server: certificate signed by unknown authority + +If you use an environment with KAS and a self-signed certificate, your `kubectl` call might return this error: + +```plaintext +kubectl get pods +Unable to connect to the server: x509: certificate signed by unknown authority +``` + +The error occurs because the job does not trust the certificate authority (CA) that signed the KAS certificate. + +To resolve the issue, [configure `kubectl` to trust the CA](#environments-with-kas-that-use-self-signed-certificates). diff --git a/doc/user/clusters/agent/gitops.md b/doc/user/clusters/agent/gitops.md index 1d71f6ac511..106f751555a 100644 --- a/doc/user/clusters/agent/gitops.md +++ b/doc/user/clusters/agent/gitops.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/agent/gitops/flux.md b/doc/user/clusters/agent/gitops/flux.md index 9a3537aacbf..98840080716 100644 --- a/doc/user/clusters/agent/gitops/flux.md +++ b/doc/user/clusters/agent/gitops/flux.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -15,7 +15,7 @@ You can use Flux to: To get started, see the [Flux installation documentation](https://fluxcd.io/flux/installation). -Support for Flux is in [Open Beta](../../../../policy/alpha-beta-support.md#beta-features). +Support for Flux is in [Beta](../../../../policy/alpha-beta-support.md#beta). ## Bootstrap installation diff --git a/doc/user/clusters/agent/gitops/flux_tutorial.md b/doc/user/clusters/agent/gitops/flux_tutorial.md index 5f902002f83..88cc824b6b2 100644 --- a/doc/user/clusters/agent/gitops/flux_tutorial.md +++ b/doc/user/clusters/agent/gitops/flux_tutorial.md @@ -1,19 +1,21 @@ --- -stage: Configure -group: Configure -info: A tutorial using Flux with Project Access Tokens +stage: Deploy +group: Environments +info: A tutorial using Flux --- # Tutorial: Set up Flux for GitOps **(FREE)** This tutorial teaches you how to set up Flux for GitOps. You'll set up a sample project, complete a bootstrap Flux installation, and authenticate your installation with a -[project access token](../../../project/settings/project_access_tokens.md). +[project deploy token](../../../project/deploy_tokens/index.md). -You can find the fully configured tutorial project [in this GitLab repository](https://gitlab.com/gitlab-org/configure/examples/flux/flux-config). It works in conjunction with [this repository](https://gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests/-/tree/main), which contains the example Kubernetes manifests. +You can find the fully configured tutorial project [in this GitLab repository](https://gitlab.com/gitlab-org/configure/examples/flux/flux-config). +It works in conjunction with [this repository](https://gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests/-/tree/main), which contains the example Kubernetes manifests. -To set up Flux with a project access token: +To set up Flux for GitOps: +1. [Create a personal access token](#create-a-personal-access-token) 1. [Create the Flux repository](#create-the-flux-repository) 1. [Create the Kubernetes manifest repository](#create-the-kubernetes-manifest-repository) 1. [Configure Flux to sync your manifests](#configure-flux-to-sync-your-manifests) @@ -21,27 +23,31 @@ To set up Flux with a project access token: Prerequisites: -- On GitLab SaaS, you must have the Premium or Ultimate tier to use a project access token. - On self-managed instances, you can have any tier. -- Not recommended. You can authenticate with a [personal or group access token](flux.md#bootstrap-installation) in all tiers. - You must have a Kubernetes cluster running. -## Create the Flux repository +## Create a personal access token -To start, create a Git repository, install Flux, and authenticate Flux with your repo: +To authenticate with the Flux CLI, you must create a personal access token +with the `api` scope: -1. Make sure your `kubectl` is configured to access your cluster. -1. [Install the Flux CLI](https://fluxcd.io/flux/installation/#install-the-flux-cli). -1. In GitLab, create a new empty project called `flux-config`. -1. In the `flux-config` project, create a [project access token](../../../project/settings/project_access_tokens.md#create-a-project-access-token) with the following settings: +1. In the upper-right corner, select your avatar. +1. Select **Edit profile**. +1. On the left sidebar, select **Access Tokens**. +1. Enter a name and optional expiry date for the token. +1. Select the `api` scope. +1. Select **Create personal access token**. - - From the **Select a role** dropdown list, select **Maintainer**. - - Under **Select scopes**, select the **API** and **write_repository** checkboxes. +You can also use a [project](../../../project/settings/project_access_tokens.md) or [group access token](../../../group/settings/group_access_tokens.md) with the `api` scope. - Name the project token `flux-project-access-token`. +## Create the Flux repository -1. From your shell, export a `GITLAB_TOKEN` environment variable with the value of your project access token. - For example, `export GITLAB_TOKEN=<flux-project-access-token>`. +Create a Git repository, install Flux, and authenticate Flux with your repo: + +1. Make sure your `kubectl` is configured to access your cluster. +1. [Install the Flux CLI](https://fluxcd.io/flux/installation/#install-the-flux-cli). You must install Flux v2 or higher. +1. In GitLab, create a new empty project called `flux-config`. +1. From your shell, export a `GITLAB_TOKEN` environment variable with the value of your personal access token. + For example, `export GITLAB_TOKEN=<personal-access-token>`. 1. Run the `bootstrap` command. The exact command depends on whether you are creating the Flux repository under a GitLab user, group, or subgroup. For more information, see the [Flux bootstrap documentation](https://fluxcd.io/flux/installation/#gitlab-and-gitlab-enterprise). @@ -54,11 +60,13 @@ To start, create a Git repository, install Flux, and authenticate Flux with your --repository=flux-config \ --branch=main \ --path=clusters/my-cluster + --deploy-token-auth ``` This command installs Flux on the Kubernetes cluster and configures it to manage itself from the repository `flux-config`. + The command also automatically creates the project deploy token required to access the `flux-config` repository. -Great work! You now have a repository, a project access token, and a Flux bootstrap installation authenticated to your repo. Any updates to your repo are automatically synced to the cluster. +Great work! You now have a repository bootstrapped with a Flux configuration. Any updates to your repository are automatically synced to the cluster. ## Create the Kubernetes manifest repository @@ -93,7 +101,7 @@ Next, create a repository for your Kubernetes manifests: - containerPort: 80 ``` -1. In the new repository, [create a deploy token](../../../project/deploy_tokens/index.md#create-a-deploy-token) with only the **read_repository** scope. +1. In the new repository, [create a deploy token](../../../project/deploy_tokens/index.md#create-a-deploy-token) with only the `read_repository` scope. 1. Store your deploy token username and password somewhere safe. 1. In Flux CLI, create a secret with your deploy token and point the secret to the new repository. For example: diff --git a/doc/user/clusters/agent/gitops/helm.md b/doc/user/clusters/agent/gitops/helm.md index b9a59d37f5d..182fd87e6f9 100644 --- a/doc/user/clusters/agent/gitops/helm.md +++ b/doc/user/clusters/agent/gitops/helm.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Using Helm charts to update a Kubernetes cluster (Alpha) **(FREE)** +# Using Helm charts to update a Kubernetes cluster (Experiment) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371019) in GitLab 15.4. > - Specifying a branch, tag, or commit reference to fetch the Kubernetes manifest files [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4516) in GitLab 15.7. @@ -13,11 +13,11 @@ You can deploy Helm charts to your Kubernetes cluster and keep the resources in with your charts and values. To do this, you use the pull-based GitOps features of the agent for Kubernetes. -This feature is in Alpha and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/7938) +This feature is an Experiment and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/7938) to track future work. Tell us about your use cases by leaving comments in the epic. NOTE: -This feature is Alpha. In future releases, to accommodate new features, the configuration format might change without notice. +This feature is an Experiment. In future releases, to accommodate new features, the configuration format might change without notice. ## GitOps workflow steps diff --git a/doc/user/clusters/agent/gitops/secrets_management.md b/doc/user/clusters/agent/gitops/secrets_management.md index 8a47dcfaf72..6e1b7da9c6c 100644 --- a/doc/user/clusters/agent/gitops/secrets_management.md +++ b/doc/user/clusters/agent/gitops/secrets_management.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index 0d84a617808..07149ccd8fc 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -57,9 +57,9 @@ This workflow has a weaker security model and is not recommended for production GitLab supports the following Kubernetes versions. You can upgrade your Kubernetes version to a supported version at any time: +- 1.26 (support ends on March 22, 2024 or when 1.29 becomes supported) - 1.25 (support ends on October 22, 2023 or when 1.28 becomes supported) - 1.24 (support ends on July 22, 2023 or when 1.27 becomes supported) -- 1.23 (support ends on February 22, 2023 or when 1.26 becomes supported) GitLab aims to support a new minor Kubernetes version three months after its initial release. GitLab supports at least three production-ready Kubernetes minor versions at any given time. diff --git a/doc/user/clusters/agent/install/index.md b/doc/user/clusters/agent/install/index.md index 297210ab8ef..1bcbb42fc8e 100644 --- a/doc/user/clusters/agent/install/index.md +++ b/doc/user/clusters/agent/install/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -138,6 +138,23 @@ By default, the Helm installation command generated by GitLab: To see the full list of customizations available, see the Helm chart's [default values file](https://gitlab.com/gitlab-org/charts/gitlab-agent/-/blob/main/values.yaml). +##### Use the agent when KAS is behind a self-signed certificate + +When [KAS](../../../../administration/clusters/kas.md) is behind a self-signed certificate, +you can set the value of `config.caCert` to the certificate. For example: + +```shell +helm update --install gitlab-agent gitlab/gitlab-agent \ + --set-file config.caCert=my-custom-ca.pem +``` + +In this example, `my-custom-ca.pem` is the path to a local file that contains +the CA certificate used by KAS. The certificate is automatically stored in a +config map and mounted in the `agentk` pod. + +If KAS is installed with the GitLab chart, and the chart is configured to provide +an [auto-generated self-signed wildcard certificate](https://docs.gitlab.com/charts/installation/tls.html#option-4-use-auto-generated-self-signed-wildcard-certificate), you can extract the CA certificate from the `RELEASE-wildcard-tls-ca` secret. + ##### Use the agent behind an HTTP proxy > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/351867) in GitLab 15.0, the GitLab agent Helm chart supports setting environment variables. @@ -155,6 +172,10 @@ helm upgrade --install gitlab-agent gitlab/gitlab-agent \ ... ``` +NOTE: +DNS rebind protection is disabled when either the `HTTP_PROXY` or the `HTTPS_PROXY` environment variable is set, +and the domain DNS can't be resolved. + #### Advanced installation method GitLab also provides a [KPT package for the agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/build/deployment/gitlab-agent). This method provides greater flexibility, but is only recommended for advanced users. diff --git a/doc/user/clusters/agent/troubleshooting.md b/doc/user/clusters/agent/troubleshooting.md index 84cdbbcc096..d058a583b19 100644 --- a/doc/user/clusters/agent/troubleshooting.md +++ b/doc/user/clusters/agent/troubleshooting.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/agent/work_with_agent.md b/doc/user/clusters/agent/work_with_agent.md index 0b0e1365604..2d54f67724e 100644 --- a/doc/user/clusters/agent/work_with_agent.md +++ b/doc/user/clusters/agent/work_with_agent.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/cost_management.md b/doc/user/clusters/cost_management.md index 910092eb4e0..6f00466bc7e 100644 --- a/doc/user/clusters/cost_management.md +++ b/doc/user/clusters/cost_management.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/create/index.md b/doc/user/clusters/create/index.md index 0721cea965a..57526394301 100644 --- a/doc/user/clusters/create/index.md +++ b/doc/user/clusters/create/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/environments.md b/doc/user/clusters/environments.md index dc6898bd3d3..c6a61f58974 100644 --- a/doc/user/clusters/environments.md +++ b/doc/user/clusters/environments.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/integrations.md b/doc/user/clusters/integrations.md index b5600096856..2c67afefd98 100644 --- a/doc/user/clusters/integrations.md +++ b/doc/user/clusters/integrations.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index f2eb0cfbc39..0ad3fdbef2e 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/management_project_template.md b/doc/user/clusters/management_project_template.md index 9edb012ba92..4b6c8dcac4c 100644 --- a/doc/user/clusters/management_project_template.md +++ b/doc/user/clusters/management_project_template.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/migrating_from_gma_to_project_template.md b/doc/user/clusters/migrating_from_gma_to_project_template.md index 42510a93495..fde8e455421 100644 --- a/doc/user/clusters/migrating_from_gma_to_project_template.md +++ b/doc/user/clusters/migrating_from_gma_to_project_template.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/compliance/compliance_report/index.md b/doc/user/compliance/compliance_report/index.md index d5af7edf515..1962dfefdf5 100644 --- a/doc/user/compliance/compliance_report/index.md +++ b/doc/user/compliance/compliance_report/index.md @@ -85,10 +85,10 @@ The following are unavailable compliance violations that are tracked in [epic 52 |:-------------------------------------|:---------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------| | Pipeline failed | Medium | [Pipeline results](../../../ci/pipelines/index.md) | Merge requests pipeline failed and was merged. | | Pipeline passed with warnings | Info | [Pipeline results](../../../ci/pipelines/index.md) | Merge request pipeline passed with warnings and was merged. | -| Code coverage down more than 10% | High | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | Code coverage report for the merge request indicates a reduction in coverage of more than 10%. | -| Code coverage down between 5% to 10% | Medium | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | Code coverage report for the merge request indicates a reduction in coverage of between 5% to 10%. | -| Code coverage down between 1% to 5% | Low | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | Code coverage report for the merge request indicates a reduction in coverage of between 1% to 5%. | -| Code coverage down less than 1% | Info | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | Code coverage report for the merge request indicates a reduction in coverage of less than 1%. | +| Code coverage down more than 10% | High | [Code coverage](../../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) | Code coverage report for the merge request indicates a reduction in coverage of more than 10%. | +| Code coverage down between 5% to 10% | Medium | [Code coverage](../../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) | Code coverage report for the merge request indicates a reduction in coverage of between 5% to 10%. | +| Code coverage down between 1% to 5% | Low | [Code coverage](../../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) | Code coverage report for the merge request indicates a reduction in coverage of between 1% to 5%. | +| Code coverage down less than 1% | Info | [Code coverage](../../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) | Code coverage report for the merge request indicates a reduction in coverage of less than 1%. | <!-- vale gitlab.SubstitutionWarning = YES --> @@ -170,7 +170,8 @@ passing in an optional value to the `commit_sha` query parameter. ## Compliance frameworks report -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387910) in GitLab 15.10. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387910) in GitLab 15.10. +> - Support for applying/removing compliance framework [added](https://gitlab.com/gitlab-org/gitlab/-/issues/383209) in GitLab 15.11 With compliance frameworks report, you can see the compliance frameworks that are applied to projects in a group. Each row of the report shows: @@ -191,3 +192,51 @@ To view the compliance frameworks report: 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Compliance report**. 1. On the page, select the **Frameworks** tab. + +### Apply a compliance framework to projects in a group + +Prerequisites: + +- You must have the Owner role for the group. + +To apply a compliance framework to projects in a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security and Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. +1. Select one or more projects. +1. From the **Choose one bulk action** dropdown list, select **Apply framework to selected projects**. +1. Select framework to apply. +1. Select **Apply**. + +### Remove a compliance framework from projects in a group + +Prerequisites: + +- You must have the Owner role for the group. + +To remove a compliance framework from projects in a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security and Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. +1. Select one or more projects. +1. From the **Choose one bulk action** dropdown list, select **Remove framework from selected projects**. +1. Select **Remove**. + +#### Filter the compliance frameworks report + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387911) in GitLab 15.11. + +To filter the list of compliance frameworks: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security & Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. +1. In the search field: + 1. Select the attribute you want to filter by. + 1. Select an operator. + 1. Select from the list of options or enter text for the search. +1. Select **Search** (**{search}**). + +Repeat this process to filter by multiple attributes. diff --git a/doc/user/compliance/license_approval_policies.md b/doc/user/compliance/license_approval_policies.md index 2769732c49a..4e10d01b18e 100644 --- a/doc/user/compliance/license_approval_policies.md +++ b/doc/user/compliance/license_approval_policies.md @@ -7,7 +7,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # License Approval Policies **(ULTIMATE)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8092) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `license_scanning_policies`. Disabled by default. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8092) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `license_scanning_policies`. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/397644) in GitLab 15.11. Feature flag `license_scanning_policies` removed. License Approval Policies allow you to specify multiple types of criteria that define when approval is required before a merge request can be merged in. diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index 55aa5b2b653..8c6d5ee893f 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -5,13 +5,13 @@ group: Composition Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# License compliance (deprecated) **(ULTIMATE)** +# License Compliance (deprecated) **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5483) in GitLab 11.0. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387561) in GitLab 15.9. WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387561) in GitLab 15.9. Users should migrate over to use the [new method of license scanning](../license_scanning_of_cyclonedx_files/index.md) prior to GitLab 16.0. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387561) in GitLab 15.9. You should instead migrate to use [License approval policies](../license_approval_policies.md) and the [new method of license scanning](../license_scanning_of_cyclonedx_files/index.md) prior to GitLab 16.0. If you're using [GitLab CI/CD](../../../ci/index.md), you can use License Compliance to search your project's dependencies for their licenses. You can then decide whether to allow or deny the use of @@ -49,6 +49,38 @@ that you can later download and analyze. WARNING: License Compliance Scanning does not support run-time installation of compilers and interpreters. +## Enable License Compliance + +To enable License Compliance in your project's pipeline, either: + +- Enable [Auto License Compliance](../../../topics/autodevops/stages.md#auto-license-compliance) + (provided by [Auto DevOps](../../../topics/autodevops/index.md)). +- Include the [`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml) in your `.gitlab-ci.yml` file. + +License Compliance is not supported when GitLab is run with FIPS mode enabled. + +### Include the License Scanning template + +Prerequisites: + +- [GitLab Runner](../../../ci/runners/index.md) available, with the + [`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). If you're using the + shared runners on GitLab.com, this is enabled by default. +- License Scanning runs in the `test` stage, which is available by default. If you redefine the stages in the + `.gitlab-ci.yml` file, the `test` stage is required. +- [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) must be disabled. + +To [include](../../../ci/yaml/index.md#includetemplate) the +[`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml), add it to your `.gitlab-ci.yml` file: + +```yaml +include: + - template: Security/License-Scanning.gitlab-ci.yml +``` + +The included template creates a `license_scanning` job in your CI/CD pipeline and scans your +dependencies to find their licenses. + ## License expressions GitLab has limited support for [composite licenses](https://spdx.github.io/spdx-spec/v2-draft/SPDX-license-expressions/). @@ -89,39 +121,7 @@ The reported licenses might be incomplete or inaccurate. | Rust | [Cargo](https://crates.io/) | | PHP | [Composer](https://getcomposer.org/) | -## Enable License Compliance - -To enable License Compliance in your project's pipeline, either: - -- Enable [Auto License Compliance](../../../topics/autodevops/stages.md#auto-license-compliance) - (provided by [Auto DevOps](../../../topics/autodevops/index.md)). -- Include the [`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml) in your `.gitlab-ci.yml` file. - -License Compliance is not supported when GitLab is run with FIPS mode enabled. - -### Include the License Scanning template - -Prerequisites: - -- [GitLab Runner](../../../ci/runners/index.md) available, with the - [`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). If you're using the - shared runners on GitLab.com, this is enabled by default. -- License Scanning runs in the `test` stage, which is available by default. If you redefine the stages in the - `.gitlab-ci.yml` file, the `test` stage is required. -- [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) must be disabled. - -To [include](../../../ci/yaml/index.md#includetemplate) the -[`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml), add it to your `.gitlab-ci.yml` file: - -```yaml -include: - - template: Security/License-Scanning.gitlab-ci.yml -``` - -The included template creates a `license_scanning` job in your CI/CD pipeline and scans your -dependencies to find their licenses. - -### Available CI/CD variables +## Available CI/CD variables License Compliance can be configured using CI/CD variables. @@ -141,7 +141,7 @@ License Compliance can be configured using CI/CD variables. | `SECURE_ANALYZERS_PREFIX` | no | Set the Docker registry base address to download the analyzer from. | | `SETUP_CMD` | no | Custom setup for the dependency installation (experimental). | -### Installing custom dependencies +## Installing custom dependencies > Introduced in GitLab 11.4. @@ -169,7 +169,7 @@ variables: In this example, `my-custom-install-script.sh` is a shell script at the root directory of your project. -### Working with Monorepos +## Working with Monorepos Depending on your language, you may need to specify the path to the individual projects of a monorepo using the `LICENSE_FINDER_CLI_OPTS` variable. Passing in @@ -184,7 +184,7 @@ variables: LICENSE_FINDER_CLI_OPTS: "--aggregate_paths=relative-path/to/sub-project/one relative-path/to/sub-project/two" ``` -### Overriding the template +## Overriding the template WARNING: Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except) @@ -203,7 +203,7 @@ license_scanning: CI_DEBUG_TRACE: "true" ``` -### Configuring Maven projects +## Configuring Maven projects The License Compliance tool provides a `MAVEN_CLI_OPTS` CI/CD variable which can hold the command line arguments to pass to the `mvn install` command which is executed under the hood. @@ -227,7 +227,7 @@ to explicitly add `-DskipTests` to your options. If you still need to run tests during `mvn install`, add `-DskipTests=false` to `MAVEN_CLI_OPTS`. -#### Using private Maven repositories +### Using private Maven repositories If you have a private Maven repository which requires login credentials, you can use the `MAVEN_CLI_OPTS` CI/CD variable. @@ -250,13 +250,13 @@ Alternatively, you can use a Java key store to verify the TLS connection. For in generate a key store file, see the [Maven Guide to Remote repository access through authenticated HTTPS](https://maven.apache.org/guides/mini/guide-repository-ssl.html). -### Selecting the version of Java +## Selecting the version of Java License Compliance uses Java 8 by default. You can specify a different Java version using `LM_JAVA_VERSION`. `LM_JAVA_VERSION` only accepts versions: 8, 11, 14, 15. -### Selecting the version of Python +## Selecting the version of Python > - [Introduced](https://gitlab.com/gitlab-org/security-products/license-management/-/merge_requests/36) in GitLab 12.0. > - In [GitLab 12.2](https://gitlab.com/gitlab-org/gitlab/-/issues/12032), Python 3.5 became the default. @@ -277,22 +277,22 @@ license_scanning: `LM_PYTHON_VERSION` or `ASDF_PYTHON_VERSION` can be used to specify the desired version of Python. When both variables are specified `LM_PYTHON_VERSION` takes precedence. -### Custom root certificates for Python +## Custom root certificates for Python You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables). -#### Using private Python repositories +### Using private Python repositories If you have a private Python repository you can use the `PIP_INDEX_URL` [CI/CD variable](#available-cicd-variables) to specify its location. -### Configuring npm projects +## Configuring npm projects You can configure npm projects by using an [`.npmrc`](https://docs.npmjs.com/configuring-npm/npmrc.html/) file. -#### Using private npm registries +### Using private npm registries If you have a private npm registry you can use the [`registry`](https://docs.npmjs.com/using-npm/config/#registry) @@ -304,7 +304,7 @@ For example: registry = https://npm.example.com ``` -#### Custom root certificates for npm +### Custom root certificates for npm You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables). @@ -318,12 +318,12 @@ For example: strict-ssl = false ``` -### Configuring Yarn projects +## Configuring Yarn projects You can configure Yarn projects by using a [`.yarnrc.yml`](https://yarnpkg.com/configuration/yarnrc/) file. -#### Using private Yarn registries +### Using private Yarn registries If you have a private Yarn registry you can use the [`npmRegistryServer`](https://yarnpkg.com/configuration/yarnrc/#npmRegistryServer) @@ -335,17 +335,17 @@ For example: npmRegistryServer: "https://npm.example.com" ``` -#### Custom root certificates for Yarn +### Custom root certificates for Yarn You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables). -### Configuring Bower projects +## Configuring Bower projects You can configure Bower projects by using a [`.bowerrc`](https://bower.io/docs/config/#bowerrc-specification) file. -#### Using private Bower registries +### Using private Bower registries If you have a private Bower registry you can use the [`registry`](https://bower.io/docs/config/#bowerrc-specification) @@ -359,16 +359,16 @@ For example: } ``` -#### Custom root certificates for Bower +### Custom root certificates for Bower You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables), or by specifying a `ca` setting in a [`.bowerrc`](https://bower.io/docs/config/#bowerrc-specification) file. -### Configuring Bundler projects +## Configuring Bundler projects -#### Using private Bundler registries +### Using private Bundler registries If you have a private Bundler registry you can use the [`source`](https://bundler.io/man/gemfile.5.html#GLOBAL-SOURCES) @@ -380,7 +380,7 @@ For example: source "https://gems.example.com" ``` -#### Custom root certificates for Bundler +### Custom root certificates for Bundler You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables), or by @@ -388,9 +388,9 @@ specifying a [`BUNDLE_SSL_CA_CERT`](https://bundler.io/v2.0/man/bundle-config.1. [variable](../../../ci/variables/index.md#define-a-cicd-variable-in-the-gitlab-ciyml-file) in the job definition. -### Configuring Cargo projects +## Configuring Cargo projects -#### Using private Cargo registries +### Using private Cargo registries If you have a private Cargo registry you can use the [`registries`](https://doc.rust-lang.org/cargo/reference/registries.html) @@ -403,7 +403,7 @@ For example: my-registry = { index = "https://my-intranet:8080/git/index" } ``` -#### Custom root certificates for Cargo +### Custom root certificates for Cargo To supply a custom root certificate to complete TLS verification, do one of the following: @@ -412,9 +412,9 @@ To supply a custom root certificate to complete TLS verification, do one of the [variable](../../../ci/variables/index.md#define-a-cicd-variable-in-the-gitlab-ciyml-file) in the job definition. -### Configuring Composer projects +## Configuring Composer projects -#### Using private Composer registries +### Using private Composer registries If you have a private Composer registry you can use the [`repositories`](https://getcomposer.org/doc/05-repositories.md) @@ -437,7 +437,7 @@ For example: } ``` -#### Custom root certificates for Composer +### Custom root certificates for Composer You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables), or by @@ -445,7 +445,7 @@ specifying a [`COMPOSER_CAFILE`](https://getcomposer.org/doc/03-cli.md#composer- [variable](../../../ci/variables/index.md#define-a-cicd-variable-in-the-gitlab-ciyml-file) in the job definition. -### Configuring Conan projects +## Configuring Conan projects You can configure [Conan](https://conan.io/) projects by adding a `.conan` directory to your project root. The project root serves as the [`CONAN_USER_HOME`](https://docs.conan.io/en/latest/reference/env_vars.html#conan-user-home). @@ -474,7 +474,7 @@ NOTE: `license_scanning` image ships with [Mono](https://www.mono-project.com/) and [MSBuild](https://github.com/mono/msbuild#microsoftbuild-msbuild). Additional setup may be required to build packages for this project configuration. -#### Using private Conan registries +### Using private Conan registries By default, [Conan](https://conan.io/) uses the `conan-center` remote. For example: @@ -508,7 +508,7 @@ example: If credentials are required to authenticate then you can configure a [protected CI/CD variable](../../../ci/variables/index.md#protect-a-cicd-variable) following the naming convention described in the [`CONAN_LOGIN_USERNAME` documentation](https://docs.conan.io/en/latest/reference/env_vars.html#conan-login-username-conan-login-username-remote-name). -#### Custom root certificates for Conan +### Custom root certificates for Conan You can provide custom certificates by adding a `.conan/cacert.pem` file to the project root and setting [`CA_CERT_PATH`](https://docs.conan.io/en/latest/reference/env_vars.html#conan-cacert-path) @@ -518,7 +518,7 @@ If you specify the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd- variable's X.509 certificates are installed in the Docker image's default trust store and Conan is configured to use this as the default `CA_CERT_PATH`. -### Configuring Go projects +## Configuring Go projects To configure [Go modules](https://github.com/golang/go/wiki/Modules) based projects, specify [CI/CD variables](https://pkg.go.dev/cmd/go#hdr-Environment_variables) @@ -528,14 +528,14 @@ If a project has [vendored](https://pkg.go.dev/cmd/go#hdr-Vendor_Directories) it then the combination of the `vendor` directory and `mod.sum` file are used to detect the software licenses associated with the Go module dependencies. -#### Using private Go registries +### Using private Go registries You can use the [`GOPRIVATE`](https://pkg.go.dev/cmd/go#hdr-Environment_variables) and [`GOPROXY`](https://pkg.go.dev/cmd/go#hdr-Environment_variables) environment variables to control where modules are sourced from. Alternatively, you can use [`go mod vendor`](https://go.dev/ref/mod#tmp_28) to vendor a project's modules. -#### Custom root certificates for Go +### Custom root certificates for Go You can specify the [`-insecure`](https://pkg.go.dev/cmd/go/internal/get) flag by exporting the [`GOFLAGS`](https://pkg.go.dev/cmd/go#hdr-Environment_variables) @@ -550,7 +550,7 @@ license_scanning: GOFLAGS: '-insecure' ``` -#### Using private NuGet registries +### Using private NuGet registries If you have a private NuGet registry you can add it as a source by adding it to the [`packageSources`](https://learn.microsoft.com/en-us/nuget/reference/nuget-config-file#package-source-sections) @@ -568,7 +568,7 @@ For example: </configuration> ``` -#### Custom root certificates for NuGet +### Custom root certificates for NuGet You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables). diff --git a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md index 0ad73f9939d..ce0c6451688 100644 --- a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md +++ b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md @@ -7,10 +7,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w # License scanning of CycloneDX files **(ULTIMATE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/384932) in GitLab 15.9 [with two flags](../../../administration/feature_flags.md) named `license_scanning_sbom_scanner` and `package_metadata_synchronization`. Both flags are disabled by default and both flags must be enabled for this feature to work. - -FLAG: -On self-managed GitLab, this feature is not available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/384932) in GitLab 15.9 for GitLab SaaS [with two flags](../../../administration/feature_flags.md) named `license_scanning_sbom_scanner` and `package_metadata_synchronization`. Both flags are disabled by default and both flags must be enabled for this feature to work. +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/385173) in GitLab 15.10 for GitLab SaaS. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385173) in GitLab 15.10 for self-managed GitLab [with two flags](../../../administration/feature_flags.md) named `license_scanning_sbom_scanner` and `package_metadata_synchronization`, both of which must be enabled for this feature to work. The flags are disabled by default due to the initial performance load when the license database is first imported. Work to improve performance is being tracked in [issue 397670](https://gitlab.com/gitlab-org/gitlab/-/issues/397670). +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/385173) in GitLab 15.11 for self-managed GitLab. To detect the licenses in use, License Compliance relies on running the [Dependency Scanning CI Jobs](../../application_security/dependency_scanning/index.md), @@ -19,13 +19,15 @@ Other 3rd party scanners may also be used as long as they produce a CycloneDX fi This method of scanning is also capable of parsing and identifying over 500 different types of licenses, as defined in [the SPDX list](https://spdx.org/licenses/). Licenses not in the SPDX list are reported as "Unknown". License information can also be extracted from packages that are dual-licensed, or have multiple different licenses that apply. -To enable license detection using Dependency Scanning in a project, -include the `Jobs/Dependency-Scanning.gitlab-ci.yml` template in its CI configuration, -but do not include the `Jobs/License-Scanning.gitlab-ci.yml` template. +## Enable license scanning + +Prerequisites: -## Requirements +- Enable [Synchronization with the GitLab License Database](../../admin_area/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in Admin Area for the GitLab instance. +- Enable [Dependency Scanning](../../application_security/dependency_scanning/index.md#configuration). -The license scanning requirements are the same as those for [Dependency Scanning](../../application_security/dependency_scanning/index.md#requirements). +From the `.gitlab-ci.yml` file, remove the deprecated line `Jobs/License-Scanning.gitlab-ci.yml`, if +it's present. ## Supported languages and package managers @@ -66,10 +68,13 @@ License scanning is supported for the following languages and package managers: <td><a href="https://maven.apache.org/">Maven</a></td> </tr> <tr> - <td rowspan="2">JavaScript and TypeScript</td> + <td rowspan="3">JavaScript and TypeScript</td> <td><a href="https://www.npmjs.com/">npm</a></td> </tr> <tr> + <td><a href="https://pnpm.io/">pnpm</a></td> + </tr> + <tr> <td><a href="https://classic.yarnpkg.com/en/">yarn</a></td> </tr> <tr> @@ -104,11 +109,6 @@ License scanning is supported for the following languages and package managers: The supported files and versions are the ones supported by [Dependency Scanning](../../application_security/dependency_scanning/index.md#supported-languages-and-package-managers). -## Configuration - -To enable license scanning of CycloneDX files, -you must configure [Dependency Scanning](../../application_security/dependency_scanning/index.md#configuration). - ## License expressions GitLab has limited support for [composite licenses](https://spdx.github.io/spdx-spec/v2-draft/SPDX-license-expressions/). diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index c71ea6576ef..096b4dc2cc5 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -316,7 +316,7 @@ At the top of the page, the number of unresolved threads is updated: If you have multiple unresolved threads in a merge request, you can create an issue to resolve them separately. In the merge request, at the top of the page, -select the ellipsis icon button (**{ellipsis_v}**) in the threads control and then select **Create issue to resolve all threads**: +select the ellipsis icon button (**{ellipsis_v}**) in the threads control and then select **Resolve all with new issue**:  diff --git a/doc/user/enterprise_user/index.md b/doc/user/enterprise_user/index.md index b6a823c656f..901731ad6f9 100644 --- a/doc/user/enterprise_user/index.md +++ b/doc/user/enterprise_user/index.md @@ -22,7 +22,7 @@ A user account is considered an enterprise account when: - [SCIM](../group/saml_sso/scim_setup.md) creates the user account on behalf of the group. -A user can also [manually connect an identity provider (IdP) to a GitLab account whose email address matches the subscribing organization's domain](../group/saml_sso/index.md#linking-saml-to-your-existing-gitlabcom-account). +A user can also [manually connect an identity provider (IdP) to a GitLab account whose email address matches the subscribing organization's domain](../group/saml_sso/index.md#link-saml-to-your-existing-gitlabcom-account). By selecting **Authorize** when connecting these two accounts, the user account with the matching email address is classified as an enterprise user. However, this user account does not have an **Enterprise** badge in GitLab. diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 4fb16d6ea27..68d1b51ec08 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -65,7 +65,7 @@ The IP addresses for `mg.gitlab.com` are subject to change at any time. On GitLab.com, there's a mailbox configured for Service Desk with the email address: `contact-project+%{key}@incoming.gitlab.com`. To use this mailbox, configure the -[custom suffix](../project/service_desk.md#configuring-a-custom-email-address-suffix) in project +[custom suffix](../project/service_desk.md#configure-a-custom-email-address-suffix) in project settings. ## Backups @@ -128,13 +128,14 @@ Host gitlab.com Some settings for [GitLab Pages](../project/pages/index.md) differ from the [defaults for self-managed instances](../../administration/pages/index.md): -| Setting | GitLab.com | -|------------------------------|------------------------| -| Domain name | `gitlab.io` | -| IP address | `35.185.44.232` | -| Support for custom domains | **{check-circle}** Yes | -| Support for TLS certificates | **{check-circle}** Yes | -| Maximum site size | 1 GB | +| Setting | GitLab.com | +|---------------------------------------------------|------------------------| +| Domain name | `gitlab.io` | +| IP address | `35.185.44.232` | +| Support for custom domains | **{check-circle}** Yes | +| Support for TLS certificates | **{check-circle}** Yes | +| Maximum site size | 1 GB | +| Number of custom domains per GitLab Pages website | 150 | The maximum size of your Pages site depends on the maximum artifact size, which is part of [GitLab CI/CD](#gitlab-cicd). @@ -152,7 +153,7 @@ the related documentation. | Artifacts maximum size (compressed) | 1 GB | See [Maximum artifacts size](../../user/admin_area/settings/continuous_integration.md#maximum-artifacts-size). | | Artifacts [expiry time](../../ci/yaml/index.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | See [Default artifacts expiration](../admin_area/settings/continuous_integration.md#default-artifacts-expiration). | | Scheduled Pipeline Cron | `*/5 * * * *` | See [Pipeline schedules advanced configuration](../../administration/cicd.md#change-maximum-scheduled-pipeline-frequency). | -| Maximum jobs in active pipelines | `500` for Free tier, `1000` for all trial tiers, and unlimited otherwise. | See [Number of jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines). | +| Maximum jobs in active pipelines | `500` for Free tier, `1000` for all trial tiers, `20000` for Premium, and `100000` for Ultimate. | See [Number of jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines). | | Maximum CI/CD subscriptions to a project | `2` | See [Number of CI/CD subscriptions to a project](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project). | | Maximum number of pipeline triggers in a project | `25000` for Free tier, Unlimited for all paid tiers | See [Limit the number of pipeline triggers](../../administration/instance_limits.md#limit-the-number-of-pipeline-triggers). | | Maximum pipeline schedules in projects | `10` for Free tier, `50` for all paid tiers | See [Number of pipeline schedules](../../administration/instance_limits.md#number-of-pipeline-schedules). | @@ -189,7 +190,7 @@ the default value [is the same as for self-managed instances](../admin_area/sett | Setting | GitLab.com default | |-------------------------------|--------------------| | [Repository size including LFS](../admin_area/settings/account_and_limit_settings.md#repository-size-limit) | 10 GB | -| [Maximum import size](../project/settings/import_export.md#maximum-import-file-size) | 5 GB | +| [Maximum import size](../project/settings/import_export.md#import-a-project-and-its-data) | 5 GB | | Maximum attachment size | 100 MB | If you are near or over the repository size limit, you can either diff --git a/doc/user/group/access_and_permissions.md b/doc/user/group/access_and_permissions.md index 0299f5c1a74..ad27255f42e 100644 --- a/doc/user/group/access_and_permissions.md +++ b/doc/user/group/access_and_permissions.md @@ -102,8 +102,9 @@ Keep in mind that restricting group access by IP address has the following impli restricted IP address, the IP restriction prevents code from being cloned. - Users might still see some events from the IP-restricted groups and projects on their dashboard. Activity might include push, merge, issue, or comment events. -- IP access restrictions for Git operations via SSH are supported only on GitLab SaaS. - IP access restrictions applied to self-managed instances block SSH completely. +- IP access restrictions for Git operations via SSH are supported on GitLab SaaS. + IP access restrictions applied to self-managed instances are possible with [`gitlab-sshd`](../../administration/operations/gitlab_sshd.md) + with [PROXY protocol](../../administration/operations/gitlab_sshd.md#proxy-protocol-support) enabled. ## Restrict group access by domain **(PREMIUM)** diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index a6dba3afacd..4c448d8e5c2 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -1,7 +1,7 @@ --- type: reference -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/group/compliance_frameworks.md b/doc/user/group/compliance_frameworks.md index 4ea474069cd..69642833d8a 100644 --- a/doc/user/group/compliance_frameworks.md +++ b/doc/user/group/compliance_frameworks.md @@ -34,10 +34,6 @@ default framework cannot be deleted. A compliance framework that is set to default has a **default** label. -NOTE: -Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/394630), only group owners can apply the default compliance framework when creating -new projects or importing projects. - ### Set and remove as default > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375038) in GitLab 15.7. @@ -108,8 +104,11 @@ However, the compliance pipeline configuration can reference the `.gitlab-ci.yml - Jobs and variables defined in the compliance pipeline can't be changed by variables in the labeled project's `.gitlab-ci.yml` file. -See [example configuration](#example-configuration) for help configuring a compliance pipeline that runs jobs from -labeled project pipeline configuration. +For more information, see: + +- [Example configuration](#example-configuration) for help configuring a compliance pipeline that runs jobs from + labeled project pipeline configuration. +- The [Create a compliance pipeline](../../tutorials/create_compliance_pipeline.md) tutorial. ### Effect on labeled projects @@ -348,3 +347,53 @@ mutation { } } ``` + +### Compliance jobs are overwritten by target repository + +If you use the `extends` statement in a compliance pipeline configuration, compliance jobs are overwritten by the target repository job. For example, +you could have the following `.compliance-gitlab-ci.yml` configuration: + +```yaml +"compliance job": + extends: + - .compliance_template + stage: build + +.compliance_template: + script: + - echo "take compliance action" +``` + +You could also have the following `.gitlab-ci.yml` configuration: + +```yaml +"compliance job": + stage: test + script: + - echo "overwriting compliance action" +``` + +This configuration results in the target repository pipeline overwriting the compliance pipeline, and you get the following message: +`overwriting compliance action`. + +To avoid overwriting a compliance job, don't use the `extends` keyword in compliance pipeline configuration. For example, +you could have the following `.compliance-gitlab-ci.yml` configuration: + +```yaml +"compliance job": + stage: build + script: + - echo "take compliance action" +``` + +You could also have the following `.gitlab-ci.yml` configuration: + +```yaml +"compliance job": + stage: test + script: + - echo "overwriting compliance action" +``` + +This configuration doesn't overwrite the compliance pipeline and you get the following message: +`take compliance action`. diff --git a/doc/user/group/devops_adoption/index.md b/doc/user/group/devops_adoption/index.md index a81b61c50ce..7105318e5df 100644 --- a/doc/user/group/devops_adoption/index.md +++ b/doc/user/group/devops_adoption/index.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Group DevOps Adoption **(ULTIMATE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321083) in GitLab 13.11 as a [Beta feature](../../../policy/alpha-beta-support.md#beta-features). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321083) in GitLab 13.11 as a [Beta feature](../../../policy/alpha-beta-support.md#beta). > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/333556) in GitLab 14.1. > - The Overview tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330401) in GitLab 14.1. > - DAST and SAST metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328033) in GitLab 14.1. diff --git a/doc/user/group/epics/epic_boards.md b/doc/user/group/epics/epic_boards.md index 6ec56d0d510..897504942f7 100644 --- a/doc/user/group/epics/epic_boards.md +++ b/doc/user/group/epics/epic_boards.md @@ -8,11 +8,21 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5067) in GitLab 13.10. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/290039) in GitLab 14.1. +> - Displaying total weight on the top of lists [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/364503) in GitLab 15.11. Epic boards build on the existing [epic tracking functionality](index.md) and [labels](../../project/labels.md). Your epics appear as cards in vertical lists, organized by their assigned labels. +On the top of each list, you can see the number of epics in the list (**{epic}**) and the total weight of all its epics (**{weight}**). + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=I1bFIAQBHB8">Epics and Issue Boards - Project Management</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/I1bFIAQBHB8" frameborder="0" allowfullscreen> </iframe> +</figure> + To view an epic board: 1. On the top bar, select **Main menu > Groups** and find your group. @@ -100,7 +110,7 @@ To remove a list from an epic board: 1. Select **Remove list**. A confirmation dialog appears. 1. Select **OK**. -## Create an epic from an epic board +### Create an epic from an epic board > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233568) in GitLab 14.0. @@ -216,15 +226,3 @@ To edit the scope of an epic board: - Show or hide the Open and Closed columns. - Select other labels as the board's scope. 1. Select **Save changes**. - -#### Display total weight on board lists - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/364503) in GitLab 15.1 [with a flag](../../../administration/feature_flags.md) named `fe_epic_board_total_weight`. Disabled by default. -> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/364503) in GitLab 15.9. -> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/364503) in GitLab 15.10. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `fe_epic_board_total_weight`. -On GitLab.com, this feature is available. - -When enabled, this feature displays total weight of all epics at the top of each list. diff --git a/doc/user/group/epics/img/epic_board_v15_10.png b/doc/user/group/epics/img/epic_board_v15_10.png Binary files differindex d836abdbb59..03bc96d9623 100644 --- a/doc/user/group/epics/img/epic_board_v15_10.png +++ b/doc/user/group/epics/img/epic_board_v15_10.png diff --git a/doc/user/group/epics/img/issue_list_v13_1.png b/doc/user/group/epics/img/issue_list_v13_1.png Binary files differdeleted file mode 100644 index ed0b4842bfe..00000000000 --- a/doc/user/group/epics/img/issue_list_v13_1.png +++ /dev/null diff --git a/doc/user/group/epics/img/issue_list_v15_11.png b/doc/user/group/epics/img/issue_list_v15_11.png Binary files differnew file mode 100644 index 00000000000..601406a8a33 --- /dev/null +++ b/doc/user/group/epics/img/issue_list_v15_11.png diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index 3a0fc86e803..0dc87b7e4e4 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -421,7 +421,7 @@ To remove an issue from an epic: The **Remove issue** warning appears. 1. Select **Remove**. - + ### Reorder issues assigned to an epic diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index b0297076a43..c5a12d3fc8e 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -40,7 +40,7 @@ Migrating groups by direct transfer copies the groups from one place to another. - The subgroup of any existing top-level group. - Another GitLab instance, including GitLab.com. - In the [API](../../../api/bulk_imports.md), copy top-level groups and subgroups to these locations. -- Copy groups with projects (in [beta](../../../policy/alpha-beta-support.md#beta-features) and not ready for production +- Copy groups with projects (in [Beta](../../../policy/alpha-beta-support.md#beta) and not ready for production use) or without projects. Copying projects with groups is available: - On GitLab.com by default. @@ -50,7 +50,7 @@ Not all group and project resources are copied. See list of copied resources bel - [Migrated project items](#migrated-project-items-beta). WARNING: -Importing groups with projects is in [beta](../../../policy/alpha-beta-support.md#beta-features). This feature is not +Importing groups with projects is in [Beta](../../../policy/alpha-beta-support.md#beta). This feature is not ready for production use. We invite you to leave your feedback about migrating by direct transfer in @@ -122,7 +122,7 @@ To ensure GitLab maps users and their contributions correctly: 1. Ensure that users have a public email on the source GitLab instance that matches any confirmed email address on the destination GitLab instance. Most users receive an email asking them to confirm their email address. 1. If users already exist on the destination instance and you use [SAML SSO for GitLab.com groups](../../group/saml_sso/index.md), all users must - [link their SAML identity to their GitLab.com account](../../group/saml_sso/index.md#linking-saml-to-your-existing-gitlabcom-account). + [link their SAML identity to their GitLab.com account](../../group/saml_sso/index.md#link-saml-to-your-existing-gitlabcom-account). ### Connect the source GitLab instance @@ -137,7 +137,7 @@ Create the group you want to import to and connect the source GitLab instance: 1. Enter the [personal access token](../../../user/profile/personal_access_tokens.md) for your source GitLab instance. 1. Select **Connect instance**. -### Select the groups to import +### Select the groups and projects to import > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385689) in GitLab 15.8, option to import groups with or without projects. @@ -153,7 +153,7 @@ role. 1. After a group has been imported, select its GitLab path to open its GitLab URL. WARNING: -Importing groups with projects is in [beta](../../../policy/alpha-beta-support.md#beta-features). This feature is not +Importing groups with projects is in [Beta](../../../policy/alpha-beta-support.md#beta). This feature is not ready for production use. ### Group import history @@ -220,11 +220,15 @@ Group items that are migrated to the destination GitLab instance include: - Already exists in the destination GitLab instance. - Has a public email in the source GitLab instance that matches a confirmed email in the destination GitLab instance. -### Migrated project items (beta) +### Migrated project items (Beta) > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267945) in GitLab 14.4 [with a flag](../../feature_flags.md) named `bulk_import_projects`. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/339941) in GitLab 15.6. > - `bulk_import_projects` feature flag [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/339941) in GitLab 15.10. +> - Project-only migrations using API [added](https://gitlab.com/gitlab-org/gitlab/-/issues/390515) in GitLab 15.11. + +If you choose to migrate projects when you [select groups to migrate](#select-the-groups-and-projects-to-import), +project items are migrated with the projects. The project items that are migrated depends on the version of GitLab you use on the destination. To determine if a specific project item is migrated: @@ -243,8 +247,11 @@ specific project item is migrated: Any other project items are **not** migrated. +If you choose not to migrate projects along with groups or if you want to retry a project migration, you can +initiate project-only migrations using the [API](../../../api/bulk_imports.md). + WARNING: -Migrating projects when migrating groups by direct transfer is in [beta](../../../policy/alpha-beta-support.md#beta-features) +Migrating projects when migrating groups by direct transfer is in [Beta](../../../policy/alpha-beta-support.md#beta) and is not ready for production use. Project items that are migrated to the destination GitLab instance include: @@ -256,6 +263,7 @@ Project items that are migrated to the destination GitLab instance include: | Badges | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75029) | | Branches (including protected branches) | [GitLab 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/339414) | | CI Pipelines | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/339407) | +| Commit comments | [GitLab 15.10](https://gitlab.com/gitlab-org/gitlab/-/issues/391601) | | Designs | [GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/339421) | | Issues | [GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/issues/267946) | | Issue boards | [GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71661) | @@ -344,7 +352,7 @@ entities.where(status: [-1]).pluck(:destination_name, :destination_namespace, :s ``` You can also see all migrated entities with any failures related to them using an -[API endpoint](../../../api/bulk_imports.md#list-all-group-migrations-entities). +[API endpoint](../../../api/bulk_imports.md#list-all-group-or-project-migrations-entities). #### Stale imports @@ -429,13 +437,11 @@ Note the following: ### Compatibility -Group file exports are in NDJSON format. GitLab previously produced group file exports in JSON format, however: +> Support for JSON-formatted project file exports [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/383682) in GitLab 15.8. -- From GitLab 15.8, GitLab no longer supports importing a JSON-formatted group file export. -- Between GitLab 14.0 and GitLab 14.7, GitLab no longer produces group file exports in JSON format but, to support - transitions, can still import JSON-formatted group file exports. +Group file exports are in NDJSON format. -From GitLab 13.0, GitLab can import group file exports that were exported from a version of GitLab up to two +You can import group file exports that were exported from a version of GitLab up to two [minor](../../../policy/maintenance.md#versioning) versions behind, which is similar to our process for [security releases](../../../policy/maintenance.md#security-releases). diff --git a/doc/user/group/index.md b/doc/user/group/index.md index d35a0920cf2..7e093ccb01b 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -20,6 +20,16 @@ For larger organizations, you can also create [subgroups](subgroups/index.md). For more information about creating and managing your groups, see [Manage groups](manage.md). +NOTE: +Self-managed customers should create a top-level group so you can see an overview of +your organization. For more information about efforts to create an +organization view of all groups, [see epic 9266](https://gitlab.com/groups/gitlab-org/-/epics/9266). +A top-level group can also have one complete +[Security Dashboard and Center](../application_security/security_dashboard/index.md), +[Vulnerability](../application_security/vulnerability_report/index.md#vulnerability-report) and +[Compliance Report](../compliance/compliance_report/index.md), and +[Value stream analytics](../group/value_stream_analytics/index.md). + ## Group visibility Like projects, a group can be configured to limit the visibility of it to: diff --git a/doc/user/group/manage.md b/doc/user/group/manage.md index 622112d9735..8d20c764bef 100644 --- a/doc/user/group/manage.md +++ b/doc/user/group/manage.md @@ -8,6 +8,16 @@ info: To determine the technical writer assigned to the Stage/Group associated w Use groups to manage one or more related projects at the same time. +NOTE: +Self-managed customers should create a top-level group so you can see an overview of +your organization. For more information about efforts to create an +organization view of all groups, [see epic 9266](https://gitlab.com/groups/gitlab-org/-/epics/9266). +A top-level group can also have one complete +[Security Dashboard and Center](../application_security/security_dashboard/index.md), +[Vulnerability](../application_security/vulnerability_report/index.md#vulnerability-report) and +[Compliance Report](../compliance/compliance_report/index.md), and +[Value stream analytics](../group/value_stream_analytics/index.md). + ## View groups To view groups, on the top bar, select **Main menu > Groups > View all groups**. @@ -237,6 +247,23 @@ To change this setting for a specific group: To change this setting globally, see [Default project creation protection](../admin_area/settings/visibility_and_access_controls.md#define-which-roles-can-create-projects). +## Add Group README + +As a group owner or member, you can use a README to provide more information about your team, and invite users to contribute to your projects. +The README is displayed on the group overview page, and can be changed in the group settings. All group members can edit the README. + +Prerequisite: + +- To create the README from the group settings, you must have the Owner role for the group. + +To add a group README: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. In the **Group README** section, select **Add README**. This action creates a new project `gitlab-profile` that contains the `README.md` file. +1. On the prompt for creating a README, select **Create and add README**. You're redirected to the Web IDE, where a README file is created. +1. In the Web IDE, edit and commit the `README.md` file. + ## Change the owner of a group You can change the owner of a group. Each group must always have at least one @@ -371,6 +398,7 @@ To transfer a group: > - [Instance setting to enable by default added](https://gitlab.com/gitlab-org/gitlab/-/issues/255449) in GitLab 14.2. > - [Instance setting is inherited and enforced when disabled](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. > - [User interface changed](https://gitlab.com/gitlab-org/gitlab/-/issues/352961) in GitLab 15.1. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/389557) in GitLab 15.11 [with a flag](../../administration/feature_flags.md) named `always_perform_delayed_deletion`. Disabled by default. [Delayed project deletion](../project/settings/index.md#delayed-project-deletion) is locked and disabled unless the instance-level settings for [deletion protection](../admin_area/settings/visibility_and_access_controls.md#deletion-protection) are enabled for either groups only or groups and projects. @@ -398,8 +426,10 @@ To enable delayed deletion of projects in a group: - (GitLab 15.0 and earlier), **Enforce for all subgroups**. 1. Select **Save changes**. -NOTE: -In GitLab 13.11 and above the group setting for delayed project deletion is inherited by subgroups. As discussed in [Cascading settings](../../development/cascading_settings.md) inheritance can be overridden, unless enforced by an ancestor. +In GitLab 13.11 and later, the group setting for delayed project deletion is inherited by subgroups. As discussed in [Cascading settings](../../development/cascading_settings.md), inheritance can be overridden unless enforced by an ancestor. + +In GitLab 15.11 with the `always_perform_delayed_deletion` feature flag enabled, this setting is removed +and all projects are deleted after the [retention period defined by the admin](../admin_area/settings/visibility_and_access_controls.md#retention-period). ## Disable email notifications @@ -447,6 +477,10 @@ You can export a list of members in a group or subgroup as a CSV. 1. Select **Export as CSV**. 1. After the CSV file has been generated, it is emailed as an attachment to the user that requested it. +The output lists direct members and members inherited from the ancestor groups. +For members with `Minimal Access` in the selected group, their `Max Role` and `Source` are derived from their membership in subgroups. +[Issue 390358](https://gitlab.com/gitlab-org/gitlab/-/issues/390358) tracks the discussion about the group members CSV export list not matching the UI members list. + ## User cap for groups > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330027) in GitLab 14.7. @@ -465,7 +499,7 @@ disabled for the group and its subgroups. Prerequisite: -- You must be assigned the Owner role) for the group. +- You must be assigned the Owner role for the group. To specify a user cap: @@ -640,9 +674,28 @@ To view the merge request approval settings for a group: Support for group-level settings for merge request approval rules is tracked in this [epic](https://gitlab.com/groups/gitlab-org/-/epics/4367). +## Group Code Suggestions **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/405126) in GitLab 15.11. + +WARNING: +This feature is in [Beta](/ee/policy/alpha-beta-support.md#beta). Code Suggestions use generative AI to suggest code while you're developing. Due to high demand, this feature will have unscheduled downtime and code suggestions in VS Code may be delayed. Code Suggestions may produce [low-quality or incomplete suggestions](../project/repository/code_suggestions.md#model-accuracy-and-quality). Beta users should read about the [known limitations](../project/repository/code_suggestions.md#known-limitations). We look forward to hearing your feedback. + +This setting enables users in the group to access [Code Suggestions](../project/repository/code_suggestions.md). +This setting [cascades to all projects](../project/merge_requests/approvals/settings.md#settings-cascading) +that belong to the group. + +To enable Code Suggestions for a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Expand the **Permissions and group features** section. +1. Find the **Code Suggestions** settings. +1. Select **Save changes**. + ## Group activity analytics **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207164) in GitLab 12.10 as a [Beta feature](../../policy/alpha-beta-support.md#beta-features). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207164) in GitLab 12.10 as a [Beta feature](../../policy/alpha-beta-support.md#beta). For a group, you can view how many merge requests, issues, and members were created in the last 90 days. diff --git a/doc/user/group/moderate_users.md b/doc/user/group/moderate_users.md new file mode 100644 index 00000000000..4ec86893524 --- /dev/null +++ b/doc/user/group/moderate_users.md @@ -0,0 +1,48 @@ +--- +stage: Anti-Abuse +group: Anti-Abuse +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Moderate users **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/modelops/anti-abuse/team-tasks/-/issues/155) in GitLab 15.8. + +This is the group-level documentation. For self-managed instances, see the [administration documentation](../admin_area/moderate_users.md). + +A group Owner can moderate user access by banning and unbanning users. + +## Unban a user + +To unban a user with the GraphQL API, see [`Mutation.namespaceBanDestroy`](../../api/graphql/reference/index.md#mutationnamespacebandestroy). + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo on unbanning a user at the group level, see [Namespace level ban - Unbanning a user](https://www.youtube.com/watch?v=mTQVbP3MQrs). + +Prerequisites: + +- In the top-level group, you must have the Owner role. + +To unban a user: + +1. Go to the top-level group. +1. On the left sidebar, select **Group information > Members**. +1. Select the **Banned** tab. +1. For the account you want to unban, select **Unban**. + +## Ban a user + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo on banning a user at the group level, see [Namespace level ban - Banning a user](https://youtu.be/1rbi1uEJmOI). + +Prerequisites: + +- In the top-level group, you must have the Owner role. +- In the top-level group, if the user you want to ban has the Owner role, you must [demote the user](manage.md#change-the-owner-of-a-group). + +To manually ban a user: + +1. Go to the top-level group. +1. On the left sidebar, select **Group information > Members**. +1. Next to the member you want to ban, select the vertical ellipsis (**{ellipsis_v}**). +1. From the dropdown list, select **Ban member**. diff --git a/doc/user/group/reporting/git_abuse_rate_limit.md b/doc/user/group/reporting/git_abuse_rate_limit.md index d96f950edcd..d5c44f4e245 100644 --- a/doc/user/group/reporting/git_abuse_rate_limit.md +++ b/doc/user/group/reporting/git_abuse_rate_limit.md @@ -13,7 +13,7 @@ On self-managed GitLab, by default this feature is not available. To make it ava This is the group-level documentation. For self-managed instances, see the [administration documentation](../../admin_area/reporting/git_abuse_rate_limit.md). -Git abuse rate limiting is a feature to automatically [ban users](#unban-a-user) who download or clone more than a specified number of repositories of a group in a given time frame. Banned users cannot access the top-level group or any of its non-public subgroups via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). Access to unrelated groups is unaffected. +Git abuse rate limiting is a feature to automatically ban users who download, clone, or fork more than a specified number of repositories of a group in a given time frame. Banned users cannot access the top-level group or any of its non-public subgroups via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). Access to unrelated groups is unaffected. Git abuse rate limiting does not apply to top-level group owners, [deploy tokens](../../../user/project/deploy_tokens/index.md), or [deploy keys](../../../user/project/deploy_keys/index.md). @@ -36,26 +36,6 @@ If automatic banning is enabled, an email notification is sent when a user is ab 1. Optional. Turn on the **Automatically ban users from this namespace when they exceed the specified limits** toggle to enable automatic banning. 1. Select **Save changes**. -## Unban a user +## Related topics -Prerequisites: - -- You must have the Owner role. - -1. On the left sidebar, select **Group information > Members**. -1. Select the **Banned** tab. -1. For the account you want to unban, select **Unban**. - -## Ban a user - -> [Introduced](https://gitlab.com/gitlab-org/modelops/anti-abuse/team-tasks/-/issues/155) in GitLab 15.8. - -Prerequisites: - -- You must have the Owner role. - -To manually ban a user: - -1. On the left sidebar, select **Group information > Members**. -1. Next to the member you want to ban, select the vertical ellipsis (**{ellipsis_v}**). -1. From the dropdown list, select **Ban member**. +- [Ban and unban users](../moderate_users.md). diff --git a/doc/user/group/repositories_analytics/index.md b/doc/user/group/repositories_analytics/index.md index edb3e1b5079..c2a4d8175b3 100644 --- a/doc/user/group/repositories_analytics/index.md +++ b/doc/user/group/repositories_analytics/index.md @@ -42,7 +42,7 @@ To see the latest code coverage for each project in your group: 1. In the **Latest test coverage results** section, from the **Select projects** dropdown list, choose the projects you want to check. You can download code coverage data for specific projects using -[code coverage history](../../../ci/pipelines/settings.md#view-code-coverage-history). +[code coverage history](../../../ci/testing/code_coverage.md#view-history-of-project-code-coverage). ## Download historic test coverage data diff --git a/doc/user/group/saml_sso/example_saml_config.md b/doc/user/group/saml_sso/example_saml_config.md index 7778648e985..0121df4fdb3 100644 --- a/doc/user/group/saml_sso/example_saml_config.md +++ b/doc/user/group/saml_sso/example_saml_config.md @@ -58,6 +58,8 @@ Attribute mapping: NOTE: Using the **Group ID** source attribute requires users to enter the group ID or object ID when configuring SAML group links. If available, use the **sAMAccountName** source attribute for the friendly group name instead. +[Azure AD limits the number of groups that can be sent in a SAML response to 150](https://support.esri.com/en/technical-article/000022190). If a user is a member of more than 150 groups, Azure does not include that user's group claim in the SAML response. + ## Google Workspace Basic SAML app configuration: diff --git a/doc/user/group/saml_sso/group_sync.md b/doc/user/group/saml_sso/group_sync.md index 7c9b6effc43..ee59eeb98db 100644 --- a/doc/user/group/saml_sso/group_sync.md +++ b/doc/user/group/saml_sso/group_sync.md @@ -134,7 +134,7 @@ To enable global group memberships lock: After a group sync, users who are not members of a mapped SAML group are removed from the group. On GitLab.com, users in the top-level group are assigned the -[default membership role](index.md#role) instead of being removed. +default membership role instead of being removed. For example, in the following diagram: diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 04dfdbc6892..a54b3fea53e 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -18,41 +18,25 @@ Users can sign in to GitLab through their SAML identity provider. You can configure SAML SSO for the top-level group only. -## Configure your identity provider - -1. Find the information in GitLab required for configuration: - 1. On the top bar, select **Main menu > Groups** and find your group. - 1. On the left sidebar, select **Settings > SAML SSO**. - 1. Note the **Assertion consumer service URL**, **Identifier**, and **GitLab single sign-on URL**. -1. Configure your SAML identity provider app using the noted details. - Alternatively, GitLab provides a [metadata XML configuration](#set-up-identity-provider-using-metadata). - See [specific identity provider documentation](#set-up-identity-provider) for more details. -1. Configure the SAML response to include a [NameID](#nameid) that uniquely identifies each user. -1. Configure the required [user attributes](#user-attributes), ensuring you include the user's email address. -1. While the default is enabled for most SAML providers, ensure the app is set to have service provider - initiated calls to link existing GitLab accounts. -1. Once the identity provider is set up, move on to [configuring GitLab](#configure-gitlab). - - - -If your account is the only owner in the group after SAML is set up, you can't unlink the account. To [unlink the account](#unlinking-accounts), -set up another user as a group owner. - -## Set up identity provider +## Set up your identity provider The SAML standard means that you can use a wide range of identity providers with GitLab. Your identity provider might have relevant documentation. It can be generic SAML documentation or specifically targeted for GitLab. -When [configuring your identity provider](#configure-your-identity-provider), consider the notes below for specific providers to help avoid common issues and as a guide for terminology used. +When setting up your identity provider, use the following provider-specific documentation +to help avoid common issues and as a guide for terminology used. -For providers not listed below, you can refer to the [instance SAML notes on configuring an identity provider](../../../integration/saml.md#configure-saml-on-your-idp) -for additional guidance on information your identity provider may require. +For identity providers not listed, you can refer to the [instance SAML notes on configuring an identity provider](../../../integration/saml.md#configure-saml-on-your-idp) +for additional guidance on information your provider may require. GitLab provides the following information for guidance only. If you have any questions on configuring the SAML app, contact your provider's support. -### Set up Azure +If you are having issues setting up your identity provider, see the +[troubleshooting documentation](#troubleshooting). -To set up SSO with Azure as your identification provider: +### Azure + +To set up SSO with Azure as your identity provider: 1. In GitLab, on the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > SAML SSO**. @@ -69,28 +53,35 @@ To set up SSO with Azure as your identification provider: 1. You should set the following attributes: - **Unique User Identifier (Name identifier)** to `user.objectID`. - - **nameid-format** to `persistent`. + - **nameid-format** to `persistent`. For more information, see how to [manage user SAML identity](#manage-user-saml-identity). - **Additional claims** to [supported attributes](#user-attributes). -1. Optional. If you use [Group Sync](#group-sync), customize the name of the +1. Make sure the identity provider is set to have provider-initiated calls + to link existing GitLab accounts. + +1. Optional. If you use [Group Sync](group_sync.md), customize the name of the group claim to match the required attribute. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> View a demo of [SCIM provisioning on Azure using SAML SSO for groups](https://youtu.be/24-ZxmTeEBU). The `objectID` mapping is outdated in this video. Follow the [SCIM documentation](scim_setup.md#configure-azure-active-directory) instead. -View an [example configuration page](example_saml_config.md#azure-active-directory). +For more information, see an [example configuration page](example_saml_config.md#azure-active-directory). + +### Google Workspace -### Set up Google Workspace +To set up Google Workspace as your identity provider: -1. [Set up SSO with Google as your identity provider](https://support.google.com/a/answer/6087519?hl=en). - The following GitLab settings correspond to the Google Workspace fields. +1. In GitLab, on the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > SAML SSO**. +1. Note the information on this page. +1. Follow the instructions for [setting up SSO with Google as your identity provider](https://support.google.com/a/answer/6087519?hl=en). The following GitLab settings correspond to the Google Workspace fields. - | GitLab setting | Google Workspace field | - |:-------------------------------------|:-----------------------| - | Identifier | **Entity ID** | - | Assertion consumer service URL | **ACS URL** | - | GitLab single sign-on URL | **Start URL** | - | Identity provider single sign-on URL | **SSO URL** | + | GitLab setting | Google Workspace field | + |:-----------------------------------------|:-----------------------| + | **Identifier** | **Entity ID** | + | **Assertion consumer service URL** | **ACS URL** | + | **GitLab single sign-on URL** | **Start URL** | + | **Identity provider single sign-on URL** | **SSO URL** | 1. Google Workspace displays a SHA256 fingerprint. To retrieve the SHA1 fingerprint required by GitLab to [configure SAML](#configure-gitlab): @@ -102,62 +93,88 @@ View an [example configuration page](example_saml_config.md#azure-active-directo ``` 1. Set these values: - - For **Primary email**: `email` - - For **First name**: `first_name` - - For **Last name**: `last_name` - - For **Name ID format**: `EMAIL` - - For **NameID**: `Basic Information > Primary email` + - For **Primary email**: `email`. + - For **First name**: `first_name`. + - For **Last name**: `last_name`. + - For **Name ID format**: `EMAIL`. + - For **NameID**: `Basic Information > Primary email`. + For more information, see [manage user SAML identity](#manage-user-saml-identity). + +1. Make sure the identity provider is set to have provider-initiated calls + to link existing GitLab accounts. On the GitLab SAML SSO page, when you select **Verify SAML Configuration**, disregard the warning that recommends setting the **NameID** format to `persistent`. -For details, see the [example configuration page](example_saml_config.md#google-workspace). - -### Set up Okta +For more information, see an [example configuration page](example_saml_config.md#google-workspace). <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For a demo of the Okta SAML setup including SCIM, see [Demo: Okta Group SAML & SCIM setup](https://youtu.be/0ES9HsZq0AQ). +View a demo of [how to configure SAML with Google Workspaces and set up Group Sync](https://youtu.be/NKs0FSQVfCY). + +### Okta + +To set up SSO with Okta as your identity provider: + +1. In GitLab, on the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > SAML SSO**. +1. Note the information on this page. +1. Follow the instructions for [setting up a SAML application in Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/main/). -1. [Set up a SAML application in Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/main/). The following GitLab settings correspond to the Okta fields. - | GitLab setting | Okta field | - | ------------------------------------ | ---------------------------------------------------------- | - | Identifier | **Audience URI** | - | Assertion consumer service URL | **Single sign-on URL** | - | GitLab single sign-on URL | **Login page URL** (under **Application Login Page** settings) | - | Identity provider single sign-on URL | **Identity Provider Single Sign-On URL** | + | GitLab setting | Okta field | + | ---------------------------------------- | -------------------------------------------------------------- | + | **Identifier** | **Audience URI** | + | **Assertion consumer service URL** | **Single sign-on URL** | + | **GitLab single sign-on URL** | **Login page URL** (under **Application Login Page** settings) | + | **Identity provider single sign-on URL** | **Identity Provider Single Sign-On URL** | 1. Under the Okta **Single sign-on URL** field, select the **Use this for Recipient URL and Destination URL** checkbox. 1. Set these values: - - For **Application username (NameID)**: **Custom** `user.getInternalProperty("id")` - - For **Name ID Format**: `Persistent` + - For **Application username (NameID)**: **Custom** `user.getInternalProperty("id")`. + - For **Name ID Format**: `Persistent`. For more information, see [manage user SAML identity](#manage-user-saml-identity). + +1. Make sure the identity provider is set to have provider-initiated calls + to link existing GitLab accounts. The Okta GitLab application available in the App Catalog only supports [SCIM](scim_setup.md). Support for SAML is proposed in [issue 216173](https://gitlab.com/gitlab-org/gitlab/-/issues/216173). -### Set up OneLogin +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo of the Okta SAML setup including SCIM, see [Demo: Okta Group SAML & SCIM setup](https://youtu.be/0ES9HsZq0AQ). + +For more information, see an [example configuration page](example_saml_config.md#okta) + +### OneLogin OneLogin supports its own [GitLab (SaaS) application](https://onelogin.service-now.com/support?id=kb_article&sys_id=92e4160adbf16cd0ca1c400e0b961923&kb_category=50984e84db738300d5505eea4b961913). +To set up OneLogin as your identity provider: + +1. In GitLab, on the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > SAML SSO**. +1. Note the information on this page. 1. If you use the OneLogin generic [SAML Test Connector (Advanced)](https://onelogin.service-now.com/support?id=kb_article&sys_id=b2c19353dbde7b8024c780c74b9619fb&kb_category=93e869b0db185340d5505eea4b961934), you should [use the OneLogin SAML Test Connector](https://onelogin.service-now.com/support?id=kb_article&sys_id=93f95543db109700d5505eea4b96198f). The following GitLab settings correspond to the OneLogin fields: - | GitLab setting | OneLogin field | - | ------------------------------------------------ | -------------------------------- | - | Identifier | **Audience** | - | Assertion consumer service URL | **Recipient** | - | Assertion consumer service URL | **ACS (Consumer) URL** | - | Assertion consumer service URL (escaped version) | **ACS (Consumer) URL Validator** | - | GitLab single sign-on URL | **Login URL** | - | Identity provider single sign-on URL | **SAML 2.0 Endpoint** | + | GitLab setting | OneLogin field | + | ---------------------------------------------------- | -------------------------------- | + | **Identifier** | **Audience** | + | **Assertion consumer service URL** | **Recipient** | + | **Assertion consumer service URL** | **ACS (Consumer) URL** | + | **Assertion consumer service URL (escaped version)** | **ACS (Consumer) URL Validator** | + | **GitLab single sign-on URL** | **Login URL** | + | **Identity provider single sign-on URL** | **SAML 2.0 Endpoint** | -1. For **NameID**, use `OneLogin ID`. +1. For **NameID**, use `OneLogin ID`. For more information, see [manage user SAML identity](#manage-user-saml-identity). -### Set up identity provider using metadata +1. Make sure the identity provider is set to have provider-initiated calls + to link existing GitLab accounts. + +### Use metadata To configure some identity providers, you need a GitLab metadata URL. To find this URL: @@ -169,38 +186,32 @@ To find this URL: Check your identity provider's documentation to see if it supports the GitLab metadata URL. -### NameID +### Manage the identity provider -GitLab.com uses the SAML NameID to identify users. The NameID element: +After you have set up your identity provider, you can: -- Is a required field in the SAML response. -- Must be unique to each user. -- Must be a persistent value that never changes, such as a randomly generated unique user ID. -- Is case sensitive. The NameID must match exactly on subsequent login attempts, so should not rely on user input that could change between upper and lower case. -- Should not be an email address or username. We strongly recommend against these as it's hard to - guarantee it doesn't ever change, for example, when a person's name changes. Email addresses are - also case-insensitive, which can result in users being unable to sign in. +- Change the identity provider. +- Change email domains. -The relevant field name and recommended value for supported providers are in the [provider specific notes](#set-up-identity-provider). +#### Change the identity provider -WARNING: -Once users have signed into GitLab using the SSO SAML setup, changing the `NameID` breaks the configuration and potentially locks users out of the GitLab group. +You can change to a different identity provider. During the change process, +users cannot access any of the SAML groups. To mitigate this, you can disable +[SSO enforcement](#sso-enforcement). -#### NameID Format +To change identity providers: -We recommend setting the NameID format to `Persistent` unless using a field (such as email) that requires a different format. -Most NameID formats can be used, except `Transient` due to the temporary nature of this format. +1. [Configure](#set-up-your-identity-provider) the group with the new identity provider. +1. Optional. If the **NameID** is not identical, [change the **NameID** for users](#manage-user-saml-identity). -### User attributes +#### Change email domains -To create users with the correct information for improved [user access and management](#user-access-and-management), -the user's details must be passed to GitLab as attributes in the SAML assertion. At a minimum, the user's email address -must be specified as an attribute named `email` or `mail`. +To migrate users to a new email domain, tell users to: -You can configure the following attributes with GitLab.com Group SAML: +1. Add their new email as the primary email to their accounts and verify it. +1. Optional. Remove their old email from the account. -- `username` or `nickname`. We recommend you configure only one of these. -- The [attributes available](../../../integration/saml.md#configure-assertions) to self-managed GitLab instances. +If the **NameID** is configured with the email address, [change the **NameID** for users](#manage-user-saml-identity). ## Configure GitLab @@ -208,139 +219,25 @@ After you set up your identity provider to work with GitLab, you must configure 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > SAML SSO**. -1. Find the SSO URL from your identity provider and enter it the **Identity provider single sign-on URL** field. -1. Find and enter the fingerprint for the SAML token signing certificate in the **Certificate** field. -1. Select the access level to be applied to newly added users in the **Default membership role** field. The default access level is 'Guest'. +1. Complete the fields: + - In the **Identity provider single sign-on URL** field, enter the SSO URL from your identity provider. + - In the **Certificate fingerprint** field, enter the fingerprint for the SAML token signing certificate. +1. In the **Default membership role** field, select the role to assign to new users. + The default role is **Guest**. In [GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/issues/214523) + and later, group owners can set a default membership role other than **Guest**. + To do so, [configure the SAML SSO for the group](#configure-gitlab). That role + becomes the starting role of all users added to the group. 1. Select the **Enable SAML authentication for this group** checkbox. -1. Select the **Save changes** button. - - +1. Optional. Select: + - **Enforce SSO-only authentication for web activity for this group**. + - **Enforce SSO-only authentication for Git activity for this group**. + For more information, see the [SSO enforcement documentation](#sso-enforcement). +1. Select **Save changes**. NOTE: -The certificate [fingerprint algorithm](../../../integration/saml.md#configure-saml-on-your-idp) must be in SHA1. When configuring the identity provider (such as [Google Workspace](#set-up-google-workspace)), use a secure signature algorithm. - -### Additional configuration information - -Many SAML terms can vary between providers. It is possible that the information you are looking for is listed under another name. - -For more information, start with your identity provider's documentation. Look for their options and examples to see how they configure SAML. This can provide hints on what you need to configure GitLab to work with these providers. - -It can also help to look at our [more detailed docs for self-managed GitLab](../../../integration/saml.md). -SAML configuration for GitLab.com is mostly the same as for self-managed instances. -However, self-managed GitLab instances use a configuration file that supports more options as described in the external [OmniAuth SAML documentation](https://github.com/omniauth/omniauth-saml/). -Internally that uses the [`ruby-saml` library](https://github.com/onelogin/ruby-saml), so we sometimes check there to verify low level details of less commonly used options. +The certificate [fingerprint algorithm](../../../integration/saml.md#configure-saml-on-your-idp) must be in SHA1. When configuring the identity provider (such as [Google Workspace](#google-workspace)), use a secure signature algorithm. -It can also help to compare the XML response from your provider with our [example XML used for internal testing](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/fixtures/saml/response.xml). - -### SSO enforcement - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5291) in GitLab 11.8. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9255) in GitLab 11.11 with ongoing enforcement in the GitLab UI. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/292811) in GitLab 13.8, with an updated timeout experience. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/211962) in GitLab 13.8 with allowing group owners to not go through SSO. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9152) in GitLab 13.11 with enforcing open SSO session to use Git if this setting is switched on. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/339888) in GitLab 14.7 to not enforce SSO checks for Git activity originating from CI/CD jobs. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/215155) in GitLab 15.5 [with a flag](../../../administration/feature_flags.md) named `transparent_sso_enforcement` to include transparent enforcement even when SSO enforcement is not enabled. Disabled on GitLab.com. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/375788) in GitLab 15.8 by enabling transparent SSO by default on GitLab.com. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/389562) in GitLab 15.10. Feature flag `transparent_sso_enforcement` removed. - -On GitLab.com, SSO is enforced: - -- When SAML SSO is enabled. -- For users with an existing SAML identity when accessing groups and projects in the organization's - group hierarchy. Users can view other groups and projects as well as their user settings without SSO sign in by using their GitLab.com credentials. - -A user has a SAML identity if one or both of the following are true: - -- They have signed in to GitLab by using their GitLab group's single sign-on URL. -- They were provisioned by SCIM. - -Users are not prompted to sign in through SSO on each visit. GitLab checks -whether a user has authenticated through SSO. If the user last signed in more -than 24 hours ago, GitLab prompts the user to sign in again through SSO. - -SSO is enforced as follows: - -| Project/Group visibility | Enforce SSO setting | Member with identity | Member without identity | Non-member or not signed in | -|--------------------------|---------------------|--------------------| ------ |------------------------------| -| Private | Off | Enforced | Not enforced | No access | -| Private | On | Enforced | Enforced | No access | -| Public | Off | Enforced | Not enforced | Not enforced | -| Public | On | Enforced | Enforced | Not enforced | - -An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/297389) to add a similar SSO requirement for API activity. - -#### SSO-only for web activity enforcement - -When the **Enforce SSO-only authentication for web activity for this group** option is enabled: - -- All users must access GitLab by using their GitLab group's single sign-on URL - to access group resources, regardless of whether they have an existing SAML - identity. -- SSO is enforced when users access groups and projects in the organization's - group hierarchy. Users can view other groups and projects without SSO sign in. -- Users cannot be added as new members manually. -- Users with the Owner role can use the standard sign in process to make - necessary changes to top-level group settings. - -SSO enforcement for web activity has the following effects when enabled: - -- For groups, users cannot share a project in the group outside the top-level - group, even if the project is forked. -- Git activity originating from CI/CD jobs do not have the SSO check enforced. -- Credentials that are not tied to regular users (for example, project and group - access tokens, and deploy keys) do not have the SSO check enforced. -- Users must be signed-in through SSO before they can pull images using the - [Dependency Proxy](../../packages/dependency_proxy/index.md). -- When the **Enforce SSO-only authentication for Git and Dependency Proxy - activity for this group** option is enabled, any API endpoint that involves - Git activity is under SSO enforcement. For example, creating or deleting a - branch, commit, or tag. For Git activity over SSH and HTTPS, users must - have at least one active session signed-in through SSO before they can push to or - pull from a GitLab repository. - -When SSO for web activity is enforced, non-SSO group members do not lose access -immediately. If the user: - -- Has an active session, they can continue accessing the group for up to 24 - hours until the identity provider session times out. -- Is signed out, they cannot access the group after being removed from the - identity provider. - -### Change the SAML app - -After you have configured your identity provider, you can: - -- Change the identity provider users sign in with. -- Migrate to a different identity provider. -- Change email domains. - -### Change the identity provider - -To change the identity provider: - -- If the `NameID` is not identical in the existing and new identity providers, [change the NameID for users](#change-nameid-for-one-or-more-users). -- If the `NameID` is identical, users do not have to make any changes. - -### Migrate to a different identity provider - -You can migrate to a different identity provider. During the migration process, -users cannot access any of the SAML groups. To mitigate this, you can disable -[SSO enforcement](#sso-enforcement). - -To migrate identity providers: - -1. [Configure](#configure-your-identity-provider) the group with the new identity provider. -1. [Change the NameID for users](#change-nameid-for-one-or-more-users). - -### Change email domains - -To migrate users to a new email domain, tell users to: - -1. Add their new email as the primary email to their accounts and verify it. -1. Optional. Remove their old email from the account. - -If the NameID is configured with the email address, [change the NameID for users](#change-nameid-for-one-or-more-users). +If you are having issues configuring GitLab, see the [troubleshooting documentation](#troubleshooting). ## User access and management @@ -358,54 +255,108 @@ When a user tries to sign in with Group SSO, GitLab attempts to find or create a - Create a new account with another email address. - Sign-in to their existing account to link the SAML identity. -### Linking SAML to your existing GitLab.com account +### User attributes + +You can pass user information to GitLab as attributes in the SAML assertion. + +- The user's email address can be an **email** or **mail** attribute. +- The username can be either a **username** or **nickname** attribute. You should specify only + one of these. + +For more information, see the [attributes available for self-managed GitLab instances](../../../integration/saml.md#configure-assertions). + +### Link SAML to your existing GitLab.com account > **Remember me** checkbox [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/121569) in GitLab 15.7. To link SAML to your existing GitLab.com account: -1. Sign in to your GitLab.com account. [Reset your password](https://gitlab.com/users/password/new) if necessary. -1. Locate and visit the **GitLab single sign-on URL** for the group you're signing in to. A group owner can find this on the group's **Settings > SAML SSO** page. If the sign-in URL is configured, users can connect to the GitLab app from the identity provider. -1. Optional. Select the **Remember me** checkbox to stay signed in to GitLab for 2 weeks. You may still be asked to re-authenticate with your SAML provider - more frequently. +1. Sign in to your GitLab.com account. [Reset your password](https://gitlab.com/users/password/new) + if necessary. +1. Locate and visit the **GitLab single sign-on URL** for the group you're signing + in to. A group owner can find this on the group's **Settings > SAML SSO** page. + If the sign-in URL is configured, users can connect to the GitLab app from the identity provider. +1. Optional. Select the **Remember me** checkbox to stay signed in to GitLab for 2 weeks. + You may still be asked to re-authenticate with your SAML provider more frequently. 1. Select **Authorize**. 1. Enter your credentials on the identity provider if prompted. -1. You are then redirected back to GitLab.com and should now have access to the group. In the future, you can use SAML to sign in to GitLab.com. +1. You are then redirected back to GitLab.com and should now have access to the group. + In the future, you can use SAML to sign in to GitLab.com. + +If a user is already a member of the group, linking the SAML identity does not +change their role. -On subsequent visits, you should be able to go [sign in to GitLab.com with SAML](#signing-in-to-gitlabcom-with-saml) or by visiting links directly. If the **enforce SSO** option is turned on, you are then redirected to sign in through the identity provider. +On subsequent visits, you should be able to [sign in to GitLab.com with SAML](#sign-in-to-gitlabcom-with-saml) +or by visiting links directly. If the **enforce SSO** option is turned on, you +are then redirected to sign in through the identity provider. -### Signing in to GitLab.com with SAML +### Sign in to GitLab.com with SAML 1. Sign in to your identity provider. 1. From the list of apps, select the "GitLab.com" app. (The name is set by the administrator of the identity provider.) 1. You are then signed in to GitLab.com and redirected to the group. -### Change NameID for one or more users +### Manage user SAML identity > Update of SAML identities using the SAML API [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227841) in GitLab 15.5. -Group owners can update the SAML identities for their group members using the [SAML API](../../../api/saml.md#update-extern_uid-field-for-a-saml-identity). +GitLab.com uses the SAML **NameID** to identify users. The **NameID** is: + +- A required field in the SAML response. +- Case sensitive. + +The **NameID** must: + +- Be unique to each user. +- Be a persistent value that never changes, such as a randomly generated unique user ID. +- Match exactly on subsequent sign-in attempts, so it should not rely on user input + that could change between upper and lower case. + +The **NameID** should not be an email address or username because: + +- Email addresses and usernames are more likely to change over time. For example, + when a person's name changes. +- Email addresses are case-insensitive, which can result in users being unable to + sign in. + +The **NameID** format must be `Persistent`, unless you are using a field, like email, that +requires a different format. You can use any format except `Transient`. + +#### Change user **NameID** + +Group owners can use the [SAML API](../../../api/saml.md#update-extern_uid-field-for-a-saml-identity) to change their group members' **NameID** and update their SAML identities . + If [SCIM](scim_setup.md) is configured, group owners can update the SCIM identities using the [SCIM API](../../../api/scim.md#update-extern_uid-field-for-a-scim-identity). Alternatively, ask the users to reconnect their SAML account. -1. Ask relevant users to [unlink their account from the group](#unlinking-accounts). -1. Ask relevant users to [link their account to the new SAML app](#linking-saml-to-your-existing-gitlabcom-account). +1. Ask relevant users to [unlink their account from the group](#unlink-accounts). +1. Ask relevant users to [link their account to the new SAML app](#link-saml-to-your-existing-gitlabcom-account). + +WARNING: +After users have signed into GitLab using SSO SAML, changing the **NameID** value +breaks the configuration and could lock users out of the GitLab group. + +For more information on the recommended value and format for specific identity +providers, see [set up your identity provider](#set-up-your-identity-provider). ### Configure user settings from SAML response > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263661) in GitLab 13.7. GitLab allows setting certain user attributes based on values from the SAML response. -Existing users will have these attributes updated if the user was originally -provisioned by the group. Users are provisioned by the group when the account was -created via [SCIM](scim_setup.md) or by first sign-in with SAML SSO for GitLab.com groups. +An existing user's attributes are updated from the SAML response values if that +user was originally provisioned by the group. Users are provisioned by the group +when the account was created either: + +- Through [SCIM](scim_setup.md). +- By first sign-in with SAML SSO for GitLab.com groups. #### Supported user attributes -- `can_create_group` - 'true' or 'false' to indicate whether the user can create +- **can_create_group** - `true` or `false` to indicate whether the user can create new groups. Default is `true`. -- `projects_limit` - The total number of personal projects a user can create. +- **projects_limit** - The total number of personal projects a user can create. A value of `0` means the user cannot create new projects in their personal namespace. Default is `10000`. @@ -451,35 +402,25 @@ bypassed for users: - That are provisioned with SAML or SCIM. - That have an email address that belongs to the verified domain. -### Role - -Starting from [GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/issues/214523), group owners can set a -"Default membership role" other than Guest. To do so, [configure the SAML SSO for the group](#configure-gitlab). -That role becomes the starting access level of all users added to the group. - -Existing members with appropriate privileges can promote or demote users, as needed. - -If a user is already a member of the group, linking the SAML identity does not change their role. - -Users given a "minimal access" role have [specific restrictions](../../permissions.md#users-with-minimal-access). - -### Blocking access +### Block user access To rescind a user's access to the group when only SAML SSO is configured, either: - Remove (in order) the user from: 1. The user data store on the identity provider or the list of users on the specific app. 1. The GitLab.com group. -- Use [Group Sync](group_sync.md#automatic-member-removal) at the top-level of your group with the [default role](#role) set to [minimal access](../../permissions.md#users-with-minimal-access) to automatically block access to all resources within the group. Users may continue to [use a seat](../../permissions.md#minimal-access-users-take-license-seats). +- Use [Group Sync](group_sync.md#automatic-member-removal) at the top-level of + your group with the default role set to [minimal access](../../permissions.md#users-with-minimal-access) + to automatically block access to all resources in the group. To rescind a user's access to the group when also using SCIM, refer to [Remove access](scim_setup.md#remove-access). -### Unlinking accounts +### Unlink accounts Users can unlink SAML for a group from their profile page. This can be helpful if: - You no longer want a group to be able to sign you in to GitLab.com. -- Your SAML NameID has changed and so GitLab can no longer find your user. +- Your SAML **NameID** has changed and so GitLab can no longer find your user. WARNING: Unlinking an account removes all roles assigned to that user in the group. @@ -496,25 +437,106 @@ For example, to unlink the `MyOrg` account: 1. On the left sidebar, select **Account**. 1. In the **Service sign-in** section, select **Disconnect** next to the connected account. -## Group Sync +## SSO enforcement + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5291) in GitLab 11.8. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9255) in GitLab 11.11 with ongoing enforcement in the GitLab UI. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/292811) in GitLab 13.8, with an updated timeout experience. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/211962) in GitLab 13.8 with allowing group owners to not go through SSO. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9152) in GitLab 13.11 with enforcing open SSO session to use Git if this setting is switched on. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/339888) in GitLab 14.7 to not enforce SSO checks for Git activity originating from CI/CD jobs. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/215155) in GitLab 15.5 [with a flag](../../../administration/feature_flags.md) named `transparent_sso_enforcement` to include transparent enforcement even when SSO enforcement is not enabled. Disabled on GitLab.com. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/375788) in GitLab 15.8 by enabling transparent SSO by default on GitLab.com. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/389562) in GitLab 15.10. Feature flag `transparent_sso_enforcement` removed. + +On GitLab.com, SSO is enforced: -For information on automatically managing GitLab group membership, see [SAML Group Sync](group_sync.md). +- When SAML SSO is enabled. +- For users with an existing SAML identity when accessing groups and projects in the organization's + group hierarchy. Users can view other groups and projects as well as their user settings without SSO sign in by using their GitLab.com credentials. + +A user has a SAML identity if one or both of the following are true: -## Passwords for users created via SAML SSO for Groups +- They have signed in to GitLab by using their GitLab group's single sign-on URL. +- They were provisioned by SCIM. -The [Generated passwords for users created through integrated authentication](../../../security/passwords_for_integrated_authentication_methods.md) guide provides an overview of how GitLab generates and sets passwords for users created via SAML SSO for Groups. +Users are not prompted to sign in through SSO on each visit. GitLab checks +whether a user has authenticated through SSO. If the user last signed in more +than 24 hours ago, GitLab prompts the user to sign in again through SSO. -## Related topics +SSO is enforced as follows: + +| Project/Group visibility | Enforce SSO setting | Member with identity | Member without identity | Non-member or not signed in | +|--------------------------|---------------------|----------------------|-------------------------|-----------------------------| +| Private | Off | Enforced | Not enforced | Not enforced | +| Private | On | Enforced | Enforced | Enforced | +| Public | Off | Enforced | Not enforced | Not enforced | +| Public | On | Enforced | Enforced | Not enforced | + +An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/297389) to add a similar SSO requirement for API activity. + +### SSO-only for web activity enforcement + +When the **Enforce SSO-only authentication for web activity for this group** option is enabled: + +- All members must access GitLab by using their GitLab group's single sign-on URL + to access group resources, regardless of whether they have an existing SAML + identity. +- SSO is enforced when users access groups and projects in the organization's + group hierarchy. Users can view other groups and projects without SSO sign in. +- Users cannot be added as new members manually. +- Users with the Owner role can use the standard sign in process to make + necessary changes to top-level group settings. +- For non-members or users who are not signed in: + - SSO is not enforced when they access public group resources. + - SSO is enforced when they access private group resources. + +SSO enforcement for web activity has the following effects when enabled: -For more information on: +- For groups, users cannot share a project in the group outside the top-level + group, even if the project is forked. +- Git activity originating from CI/CD jobs do not have the SSO check enforced. +- Credentials that are not tied to regular users (for example, project and group + access tokens, and deploy keys) do not have the SSO check enforced. +- Users must be signed-in through SSO before they can pull images using the + [Dependency Proxy](../../packages/dependency_proxy/index.md). +- When the **Enforce SSO-only authentication for Git and Dependency Proxy + activity for this group** option is enabled, any API endpoint that involves + Git activity is under SSO enforcement. For example, creating or deleting a + branch, commit, or tag. For Git activity over SSH and HTTPS, users must + have at least one active session signed-in through SSO before they can push to or + pull from a GitLab repository. + +When SSO for web activity is enforced, non-SSO group members do not lose access +immediately. If the user: + +- Has an active session, they can continue accessing the group for up to 24 + hours until the identity provider session times out. +- Is signed out, they cannot access the group after being removed from the + identity provider. + +## Related topics -- Setting up SAML on self-managed GitLab instances, see - [SAML SSO for self-managed GitLab instances](../../../integration/saml.md). -- Commonly-used terms, see the - [glossary of common terms](../../../integration/saml.md#glossary-of-common-terms). -- The differences between SaaS and self-managed authentication and authorization, - see the [SaaS vs. Self-Managed comparison](../../../administration/auth/index.md#saas-vs-self-managed-comparison). +- [SAML SSO for self-managed GitLab instances](../../../integration/saml.md) +- [Glossary](../../../integration/saml.md#glossary) +- [Authentication comparison between SaaS and self-managed](../../../administration/auth/index.md#saas-vs-self-managed-comparison) +- [Passwords for users created through integrated authentication](../../../security/passwords_for_integrated_authentication_methods.md) +- [SAML Group Sync](group_sync.md) ## Troubleshooting -See our [troubleshooting SAML guide](troubleshooting.md). +If you find it difficult to match the different SAML terms between GitLab and the +identity provider: + +1. Check your identity provider's documentation. Look at their example SAML + configurations for information on the terms they use. +1. Check the [SAML SSO for self-managed GitLab instances documentation](../../../integration/saml.md). + The self-managed GitLab instance SAML configuration file supports more options + than the GitLab.com file. You can find information on the self-managed instance + file in the: + - External [OmniAuth SAML documentation](https://github.com/omniauth/omniauth-saml/). + - [`ruby-saml` library](https://github.com/onelogin/ruby-saml). +1. Compare the XML response from your provider with our + [example XML used for internal testing](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/fixtures/saml/response.xml). + +For other troubleshooting information, see the [troubleshooting SAML guide](troubleshooting.md). diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index a61615104c1..38a1c4125aa 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -116,7 +116,7 @@ For each attribute: 1. Select the required settings. 1. Select **Ok**. -If your SAML configuration differs from [the recommended SAML settings](index.md#set-up-azure), select the mapping +If your SAML configuration differs from [the recommended SAML settings](index.md#azure), select the mapping attributes and modify them accordingly. In particular, the `objectId` source attribute must map to the `externalId` target attribute. @@ -133,7 +133,7 @@ Prerequisites: product tier is required to use SCIM on Okta. - [GitLab is configured](#configure-gitlab). - SAML application for [Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/main/) set up as - described in the [Okta setup notes](index.md#set-up-okta). + described in the [Okta setup notes](index.md#okta). - Your Okta SAML setup matches the [configuration steps exactly](index.md), especially the NameID configuration. To configure Okta for SCIM: @@ -214,7 +214,7 @@ To link your SCIM and SAML identities: 1. Update the [primary email](../../profile/index.md#change-your-primary-email) address in your GitLab.com user account to match the user profile email address in your identity provider. -1. [Link your SAML identity](index.md#linking-saml-to-your-existing-gitlabcom-account). +1. [Link your SAML identity](index.md#link-saml-to-your-existing-gitlabcom-account). ### Remove access diff --git a/doc/user/group/saml_sso/troubleshooting.md b/doc/user/group/saml_sso/troubleshooting.md index eadf385feb3..62c13e8909b 100644 --- a/doc/user/group/saml_sso/troubleshooting.md +++ b/doc/user/group/saml_sso/troubleshooting.md @@ -167,7 +167,7 @@ you must set `attribute_statements` in the SAML configuration to This error suggests you are signed in as a GitLab user but have already linked your SAML identity to a different GitLab user. Sign out and then try to sign in again using SAML, which should log you into GitLab with the linked user account. -If you do not wish to use that GitLab user with the SAML login, you can [unlink the GitLab account from the SAML app](index.md#unlinking-accounts). +If you do not wish to use that GitLab user with the SAML login, you can [unlink the GitLab account from the SAML app](index.md#unlink-accounts). ### Message: "SAML authentication failed: User has already been taken" @@ -176,7 +176,7 @@ Here are possible causes and solutions: | Cause | Solution | | ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| You've tried to link multiple SAML identities to the same user, for a given identity provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](index.md#unlinking-accounts) from this GitLab account before attempting to sign in again. | +| You've tried to link multiple SAML identities to the same user, for a given identity provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](index.md#unlink-accounts) from this GitLab account before attempting to sign in again. | | The `NameID` changes every time the user requests SSO identification | [Check the `NameID`](#verify-nameid) is not set with `Transient` format, or the `NameID` is not changing on subsequent requests.| ### Message: "SAML authentication failed: Email has already been taken" @@ -196,7 +196,7 @@ User accounts are created in one of the following ways: Getting both of these errors at the same time suggests the `NameID` capitalization provided by the identity provider didn't exactly match the previous value for that user. -This can be prevented by configuring the `NameID` to return a consistent value. Fixing this for an individual user involves changing the identifier for the user. For GitLab.com, the user needs to [unlink their SAML from the GitLab account](index.md#unlinking-accounts). +This can be prevented by configuring the `NameID` to return a consistent value. Fixing this for an individual user involves changing the identifier for the user. For GitLab.com, the user needs to [unlink their SAML from the GitLab account](index.md#unlink-accounts). ### Message: "Request to link SAML account must be authorized" @@ -209,7 +209,7 @@ initiated by the service provider and not only the identity provider. ### Message: "There is already a GitLab account associated with this email address. Sign in with your existing credentials to connect your organization's account" **(PREMIUM SAAS)** -A user can see this message when they are trying to [manually link SAML to their existing GitLab.com account](index.md#linking-saml-to-your-existing-gitlabcom-account). +A user can see this message when they are trying to [manually link SAML to their existing GitLab.com account](index.md#link-saml-to-your-existing-gitlabcom-account). To resolve this problem, the user should check they are using the correct GitLab password to sign in. The user first needs to [reset their password](https://gitlab.com/users/password/new) if both: @@ -233,7 +233,7 @@ This can then be compared to the `NameID` being sent by the identity provider by Ensure that the **GitLab single sign-on URL** (for GitLab.com) or the instance URL (for self-managed) has been configured as "Login URL" (or similarly named field) in the identity provider's SAML app. -For GitLab.com, alternatively, when users need to [link SAML to their existing GitLab.com account](index.md#linking-saml-to-your-existing-gitlabcom-account), provide the **GitLab single sign-on URL** and instruct users not to use the SAML app on first sign in. +For GitLab.com, alternatively, when users need to [link SAML to their existing GitLab.com account](index.md#link-saml-to-your-existing-gitlabcom-account), provide the **GitLab single sign-on URL** and instruct users not to use the SAML app on first sign in. ### Users receive a 404 **(PREMIUM SAAS)** @@ -244,8 +244,8 @@ If all users are receiving a `404` when attempting to sign in using SAML, confir If you receive a `404` during setup when using "verify configuration", make sure you have used the correct [SHA-1 generated fingerprint](../../../integration/saml.md#configure-saml-on-your-idp). -If a user is trying to sign in for the first time and the GitLab single sign-on URL has not [been configured](index.md#configure-your-identity-provider), they may see a 404. -As outlined in the [user access section](index.md#linking-saml-to-your-existing-gitlabcom-account), a group Owner needs to provide the URL to users. +If a user is trying to sign in for the first time and the GitLab single sign-on URL has not [been configured](index.md#set-up-your-identity-provider), they may see a 404. +As outlined in the [user access section](index.md#link-saml-to-your-existing-gitlabcom-account), a group Owner needs to provide the URL to users. If all users are receiving a `404` after signing in to the identity provider (IdP): @@ -317,4 +317,4 @@ This error appears when you try to invite a user to a GitLab.com group (or subgr If you see this message after trying to invite a user to a group: 1. Ensure the user has been [added to the SAML identity provider](index.md#user-access-and-management). -1. Ask the user to [link SAML to their existing GitLab.com account](index.md#linking-saml-to-your-existing-gitlabcom-account), if they have one. Otherwise, ask the user to create a GitLab.com account by [accessing GitLab.com through the identity provider's dashboard](index.md#user-access-and-management), or by [signing up manually](https://gitlab.com/users/sign_up) and linking SAML to their new account. +1. Ask the user to [link SAML to their existing GitLab.com account](index.md#link-saml-to-your-existing-gitlabcom-account), if they have one. Otherwise, ask the user to create a GitLab.com account by [accessing GitLab.com through the identity provider's dashboard](index.md#user-access-and-management), or by [signing up manually](https://gitlab.com/users/sign_up) and linking SAML to their new account. diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md index 57d4fd513ff..a1eead1563f 100644 --- a/doc/user/group/value_stream_analytics/index.md +++ b/doc/user/group/value_stream_analytics/index.md @@ -5,13 +5,9 @@ group: Optimize info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Value stream analytics for groups **(PREMIUM)** +# Value stream analytics **(FREE)** -Value stream analytics measures the time it takes to go from an idea to production. It also helps businesses: - -- Visualize their end-to-end DevSecOps workstreams. -- Identify and solve inefficiencies. -- Optimize their workstreams to deliver more value, faster. +Value stream analytics measures the time it takes to go from an idea to production. A **value stream** is the entire work process that delivers value to customers. For example, the [DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/) is a value stream that starts @@ -25,11 +21,81 @@ Use value stream analytics to identify: - Long-running issues or merge requests. - Factors that cause your software development lifecycle to slow down. -Value stream analytics is also available for [projects](../../analytics/value_stream_analytics.md). +Value stream analytics helps businesses: + +- Visualize their end-to-end DevSecOps workstreams. +- Identify and solve inefficiencies. +- Optimize their workstreams to deliver more value, faster. + +Value stream analytics is available for projects and groups. + +## Feature availability + +Value stream analytics offers different features at the project and group level for FOSS and licensed versions. + +- On GitLab Free, value stream analytics does not aggregate data. It queries the database directly where the date range filter is applied to the creation date of issues and merge request. You can view value stream analytics with pre-defined default stages. +- On GitLab Premium, value stream analytics aggregates data and applies the date range filter on the end event. You can also create, edit, and delete value streams. + +|Feature|Group level (licensed)|Project level (licensed)|Project level (FOSS)| +|-|-|-|-| +|Create custom value streams|Yes|Yes|no, only one value stream (default) is present with the default stages| +|Create custom stages|Yes|Yes|No| +|Filtering (for example, by author, label, milestone)|Yes|Yes|Yes| +|Stage time chart|Yes|Yes|No| +|Total time chart|Yes|Yes|No| +|Task by type chart|Yes|No|No| +|DORA Metrics|Yes|Yes|No| +|Cycle time and lead time summary (Key metrics)|Yes|Yes|No| +|New issues, commits, and deploys (Key metrics)|Yes, excluding commits|Yes|Yes| +|Uses aggregated backend|Yes|Yes|No| +|Date filter behavior|Filters items [finished within the date range](https://gitlab.com/groups/gitlab-org/-/epics/6046)|Filters items by creation date.|Filters items by creation date.| +|Authorization|At least reporter|At least reporter|Can be public| ## How value stream analytics works -### How value stream analytics aggregates data +Value stream analytics calculates the duration of every stage of your software development process. + +Value stream analytics is made of three core objects: + +- A **value stream** contains a value stream stage list. +- Each value stream stage list contains one or more **stages**. +- Each stage has two **events**: start and stop. + +### Value stream stages + +A stage represents an event pair (start and end events) with additional metadata, such as the name of the stage. You can configure the stages in the pairing rules defined in the backend. + +### Value streams + +Value streams are container objects for the stages. You can have multiple value streams per group, to focus on different aspects of the DevOps lifecycle. + +### Value stream stage events + +Events are the smallest building blocks of the value stream analytics feature. A stage consists of a start event and an end event. + +The following stage events are available: + +- Issue closed +- Issue created +- Issue first added to board +- Issue first associated with milestone +- Issue first mentioned +- Issue label added +- Issue label removed +- MR closed +- MR merged +- MR created +- MR first commit time +- MR first deployed +- MR label added +- MR label removed +- MR last pipeline duration + +These events play a key role in the duration calculation, which is calculated by the formula: duration = end event time - start event time. + +To learn what start and end events can be paired, see [Validating start and end events](../../../development/value_stream_analytics.md#validating-start-and-end-events). + +### How value stream analytics aggregates data **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335391) in GitLab 14.5. > - Filter by stop date toggle [added](https://gitlab.com/gitlab-org/gitlab/-/issues/352428) in GitLab 14.9 @@ -133,12 +199,14 @@ You can change the name of a project environment in your GitLab CI/CD configurat Prerequisites: -- You must have at least the Reporter role to view value stream analytics for groups. -- You must create a [custom value stream](#create-a-value-stream-with-gitlab-default-stages). Value stream analytics only shows custom value streams created for your group. +- You must have at least the Reporter role. +- You must create a [custom value stream](#create-a-value-stream-with-gitlab-default-stages). Value stream analytics only shows custom value streams created for your group or project. -To view value stream analytics for your group: +To view value stream analytics for your group or project: -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value stream**. 1. To view metrics for a particular stage, select a stage below the **Filter results** text box. 1. Optional. Filter the results: @@ -166,30 +234,11 @@ The table shows a list of related workflow items for the selected stage. Based o > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210315) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in GitLab 13.12. -The **Overview** page in value stream analytics shows key metrics and DORA metrics of group performance. Based on the filter you select, -the dashboard automatically aggregates DORA metrics and displays the current status of the value stream. Select a DORA metric to view its chart. - -Prerequisite: - -- To view deployment metrics, you must have a -[production environment configured](#how-value-stream-analytics-identifies-the-production-environment). - -To view the DORA metrics and key metrics: - -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. -Key metrics and DORA metrics display below the **Filter results** text box. +The **Overview** page in value stream analytics displays key metrics and DORA metrics of group performance. ### Key metrics -The **Overview** page shows the following key metrics that measure team performance: +Value stream analytics includes the following key metrics: - **Lead time**: Median time from when the issue was created to when it was closed. - **Cycle time**: Median time from first commit to issue closed. GitLab measures cycle time from the earliest commit of a @@ -205,20 +254,49 @@ The **Overview** page shows the following key metrics that measure team performa > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355304) time to restore service tile in GitLab 15.0. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357071) change failure rate tile in GitLab 15.0. -The value stream analytics **Overview** dashboard displays the following [DORA](../../../user/analytics/dora_metrics.md) metrics: +Value stream analytics includes the following [DORA](../../../user/analytics/dora_metrics.md) metrics: -- Deployment Frequency. -- Lead time for changes. -- Time to restore service. -- Change failure rate. +- Deployment frequency +- Lead time for changes +- Time to restore service +- Change failure rate DORA metrics are calculated based on data from the [DORA API](../../../api/dora/metrics.md#devops-research-and-assessment-dora-key-metrics-api). +If you have a GitLab Premium or Ultimate subscription: + +- The number of successful deployments is calculated with DORA data. +- The data is filtered based on environment and environment tier. + NOTE: In GitLab 13.9 and later, deployment frequency metrics are calculated based on when the deployment was finished. In GitLab 13.8 and earlier, deployment frequency metrics are calculated based on when the deployment was created. +## View key and DORA metrics + +Prerequisite: + +- To view deployment metrics, you must have a +[production environment configured](#how-value-stream-analytics-identifies-the-production-environment). + +To view the DORA metrics and key metrics: + +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. + Key metrics and DORA metrics display below the **Filter results** text box. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + Based on the filter you select, the dashboard automatically aggregates DORA metrics and displays the status of the value stream. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. +1. Select a DORA metric to view its chart. + ## View metrics for each development stage > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210315) in GitLab 13.0. @@ -228,7 +306,9 @@ Value stream analytics shows the median time spent by issues or merge requests i To view the median time spent in each stage by a group: -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value stream**. 1. Optional. Filter the results: 1. Select the **Filter results** text box. @@ -243,7 +323,24 @@ NOTE: The date range selector filters items by the event time. The event time is when the selected stage finished for the given item. -## Create a value stream +## View tasks by type **(PREMIUM)** + +The **Tasks by type** chart displays the cumulative number of issues and merge requests per day for your group. + +The chart uses the global page filters to display data based on the selected +group and time frame. + +To view tasks by type: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Below the **Filter results** text box, select **Overview**. The **Tasks by type** chart displays below the **Total time** chart. +1. To switch between the task type, select the **Settings** (**{settings}**) dropdown list + and select **Issues** or **Merge Requests**. +1. To add or remove labels, select the **Settings** (**{settings}**) dropdown list + and select or search for a label. By default the top group-level labels (maximum 10) are selected. You can select a maximum of 15 labels. + +## Create a value stream **(PREMIUM)** ### Create a value stream with GitLab default stages @@ -252,7 +349,9 @@ selected stage finished for the given item. When you create a value stream, you can use GitLab default stages and hide or re-order them. You can also create custom stages in addition to those provided in the default template. -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. Select **Create new Value Stream**. 1. Enter a name for the value stream. @@ -276,7 +375,9 @@ If you have recently upgraded to GitLab Premium, it can take up to 30 minutes fo When you create a value stream, you can create and add custom stages that align with your own development workflows. -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. Select **Create value stream**. 1. For each stage: @@ -304,13 +405,15 @@ In the example above, two independent value streams are set up for two teams tha The first value stream uses standard timestamp-based events for defining the stages. The second value stream uses label events. -## Edit a value stream +## Edit a value stream **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267537) in GitLab 13.10. After you create a value stream, you can customize it to suit your purposes. To edit a value stream: -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. In the upper-right corner, select the dropdown list, then select a value stream. 1. Next to the value stream dropdown list, select **Edit**. @@ -323,21 +426,22 @@ After you create a value stream, you can customize it to suit your purposes. To 1. Optional. To undo any modifications, select **Restore value stream defaults**. 1. Select **Save Value Stream**. -## Delete a value stream +## Delete a value stream **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221205) in GitLab 13.4. To delete a custom value stream: -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Analytics > Value Stream**. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. In the upper-right corner, select the dropdown list, then select the value stream you would like to delete. 1. Select **Delete (name of value stream)**. 1. To confirm, select **Delete**.  -## View number of days for a cycle to complete +## View number of days for a cycle to complete **(PREMIUM)** > - Chart median line [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/235455) in GitLab 13.4. > - Totals [replaced](https://gitlab.com/gitlab-org/gitlab/-/issues/262070) with averages in GitLab 13.12. @@ -345,7 +449,9 @@ To delete a custom value stream: The **Total time chart** shows the average number of days it takes for development cycles to complete. The chart shows data for the last 500 workflow items. -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value stream**. 1. Above the **Filter results** box, select a stage: - To view a summary of the cycle time for all stages, select **Overview**. @@ -358,23 +464,42 @@ The chart shows data for the last 500 workflow items. - In the **From** field, select a start date. - In the **To** field, select an end date. -## View tasks by type +## Access permissions for value stream analytics -The **Tasks by type** chart displays the cumulative number of issues and merge requests per day for your group. +Access permissions for value stream analytics depend on the project type. -The chart uses the global page filters to display data based on the selected -group, projects, and time frame. +| Project type | Permissions | +|--------------|----------------------------------------| +| Public | Anyone can access. | +| Internal | Any authenticated user can access. | +| Private | Any member Guest and above can access. | -To view tasks by type: +## Troubleshooting -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Analytics > Value stream**. -1. Below the **Filter results** text box, select **Overview**. The **Tasks by type** chart displays below the **Total time** chart. -1. To switch between the task type, select the **Settings** (**{settings}**) dropdown list - and select **Issues** or **Merge Requests**. -1. To add or remove labels, select the **Settings** (**{settings}**) dropdown list - and select or search for a label. By default the top group-level labels (maximum 10) are selected. You can select a maximum of 15 labels. +### 100% CPU utilization by Sidekiq `cronjob:analytics_cycle_analytics` -## Troubleshooting +It is possible that value stream analytics background jobs +strongly impact performance by monopolizing CPU resources. + +To recover from this situation: + +1. Disable the feature for all projects in [the Rails console](../../../administration/operations/rails_console.md), + and remove existing jobs: + + ```ruby + Project.find_each do |p| + p.analytics_access_level='disabled'; + p.save! + end + + Analytics::CycleAnalytics::GroupStage.delete_all + Analytics::CycleAnalytics::Aggregation.delete_all + ``` -See [Value stream analytics for projects](../../analytics/value_stream_analytics.md#troubleshooting). +1. Configure a [Sidekiq routing](../../../administration/sidekiq/processing_specific_job_classes.md) + with for example a single `feature_category=value_stream_management` + and multiple `feature_category!=value_stream_management` entries. + Find other relevant queue metadata in the + [Enterprise Edition list](../../../administration/sidekiq/processing_specific_job_classes.md#list-of-available-job-classes). +1. Enable value stream analytics for one project after another. + You might need to tweak the Sidekiq routing further according to your performance requirements. diff --git a/doc/user/img/observability_copy_shortened_link.png b/doc/user/img/observability_copy_shortened_link.png Binary files differdeleted file mode 100644 index 217e56972cc..00000000000 --- a/doc/user/img/observability_copy_shortened_link.png +++ /dev/null diff --git a/doc/user/infrastructure/clusters/connect/index.md b/doc/user/infrastructure/clusters/connect/index.md index 655bc774477..b571a65b50a 100644 --- a/doc/user/infrastructure/clusters/connect/index.md +++ b/doc/user/infrastructure/clusters/connect/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md index 569f876f36c..45445c0a0f5 100644 --- a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md index befd793c949..4516ea538a9 100644 --- a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md index ecfd086d254..c8d2fb674b2 100644 --- a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/deploy/inventory_object.md b/doc/user/infrastructure/clusters/deploy/inventory_object.md index 6bd44c7a0f7..50185966267 100644 --- a/doc/user/infrastructure/clusters/deploy/inventory_object.md +++ b/doc/user/infrastructure/clusters/deploy/inventory_object.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/index.md b/doc/user/infrastructure/clusters/index.md index 9f56b92dabc..d0419e08f95 100644 --- a/doc/user/infrastructure/clusters/index.md +++ b/doc/user/infrastructure/clusters/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/manage/clusters_health.md b/doc/user/infrastructure/clusters/manage/clusters_health.md index f522cb5bff6..620430a449a 100644 --- a/doc/user/infrastructure/clusters/manage/clusters_health.md +++ b/doc/user/infrastructure/clusters/manage/clusters_health.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md index 8b75d54d352..f07b70dd8e0 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md index 14d3a7996e0..1d56e7c1ba1 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md index c2190ad7cfa..bbf8391e8a0 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md index b2b5da61a81..a21c7271e7a 100644 --- a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md +++ b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/iac/gitlab_terraform_helpers.md b/doc/user/infrastructure/iac/gitlab_terraform_helpers.md index 8be949c40cd..d5376bfbcb1 100644 --- a/doc/user/infrastructure/iac/gitlab_terraform_helpers.md +++ b/doc/user/infrastructure/iac/gitlab_terraform_helpers.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md index f9891934067..d4fc345f494 100644 --- a/doc/user/infrastructure/iac/index.md +++ b/doc/user/infrastructure/iac/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -100,7 +100,7 @@ For GitLab-curated template recipes, see [Terraform template recipes](terraform_ ## Related topics - View [the images that contain the `gitlab-terraform` shell script](https://gitlab.com/gitlab-org/terraform-images). -- Use GitLab as a [Terraform module registry](../../packages/terraform_module_registry/index.md). +- Use GitLab as a [Terraform Module Registry](../../packages/terraform_module_registry/index.md). - To store state files in local storage or in a remote store, use the [GitLab-managed Terraform state](terraform_state.md). - To collaborate on Terraform code changes and Infrastructure-as-Code workflows, use the [Terraform integration in merge requests](mr_integration.md). diff --git a/doc/user/infrastructure/iac/mr_integration.md b/doc/user/infrastructure/iac/mr_integration.md index 7b4a7cd6b95..b2e02cd7375 100644 --- a/doc/user/infrastructure/iac/mr_integration.md +++ b/doc/user/infrastructure/iac/mr_integration.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/iac/terraform_state.md b/doc/user/infrastructure/iac/terraform_state.md index 62772c5b379..7b95a7267af 100644 --- a/doc/user/infrastructure/iac/terraform_state.md +++ b/doc/user/infrastructure/iac/terraform_state.md @@ -1,16 +1,14 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # GitLab-managed Terraform state **(FREE)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2673) in GitLab 13.0. -> - Support for state names that contain periods introduced in GitLab 15.7 [with a flag](../../../administration/feature_flags.md) named `allow_dots_on_tf_state_names`. Disabled by default. [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106861) in GitLab 15.7. - -FLAG: -On self-managed GitLab, by default support for state names that contain periods is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `allow_dots_on_tf_state_names`. On GitLab.com, support for state names that contain periods is available. Requests for state files might generate HTTP 404 errors after enabling this feature. For more information, see [Troubleshooting the Terraform integration with GitLab](troubleshooting.md#state-not-found-if-the-state-name-contains-a-period). +> - Support for state names that contain periods introduced in GitLab 15.7 [with a flag](../../../administration/feature_flags.md) named `allow_dots_on_tf_state_names`. Disabled by default. +> - Support for state names that contain periods [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/385597) in GitLab 16.0. Feature flag `allow_dots_on_tf_state_names` removed. Terraform uses state files to store details about your infrastructure configuration. With Terraform remote [backends](https://www.terraform.io/language/settings/backends/configuration), @@ -338,6 +336,5 @@ You can also use [the GraphQL API](../../../api/graphql/reference/index.md#mutat ## Related topics -- [Troubleshooting GitLab-managed Terraform state](troubleshooting.md). -- To use GitLab and Terraform to deploy an AWS EC2 instance in a custom VPC, - see [this sample project](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-aws). +- [Troubleshooting GitLab-managed Terraform state](troubleshooting.md) +- [Sample project: Terraform deployment of AWS EC2 instance in a custom VPC](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-aws) diff --git a/doc/user/infrastructure/iac/terraform_template_recipes.md b/doc/user/infrastructure/iac/terraform_template_recipes.md index a79f2758441..0011e7c9242 100644 --- a/doc/user/infrastructure/iac/terraform_template_recipes.md +++ b/doc/user/infrastructure/iac/terraform_template_recipes.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/iac/troubleshooting.md b/doc/user/infrastructure/iac/troubleshooting.md index 2aa1e5d3284..d770c0111d0 100644 --- a/doc/user/infrastructure/iac/troubleshooting.md +++ b/doc/user/infrastructure/iac/troubleshooting.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -160,12 +160,3 @@ If your `TF_HTTP_ADDRESS`, `TF_HTTP_LOCK_ADDRESS` and `TF_HTTP_UNLOCK_ADDRESS` a to update the state names there. Alternatively, you can [migrate your terraform state](terraform_state.md#migrate-to-a-gitlab-managed-terraform-state). - -#### Self-managed GitLab instances - -By default, support for state names with periods is not enabled on self-managed GitLab. -You can enable it from the Rails console: - -```ruby -Feature.enable(:allow_dots_on_tf_state_names) -``` diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index 87633932e0d..d60f20a3713 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 30432b8b492..fd48e7244d6 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -211,27 +211,44 @@ For more information, see the [Kroki integration](../administration/integration/ [View this topic in GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md#emojis). -```markdown -Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: +::Tabs -:zap: You can use emoji anywhere GitLab Flavored Markdown is supported. :v: +:::TabTitle Rendered Markdown -You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People :heart: you for that. +<!-- vale gitlab.Markdown_emoji = NO --> -If you're new to this, don't be :fearful:. You can join the emoji :family:. Just look up one of the supported codes. +Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/monkey.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":monkey:" alt=":monkey:"> around a bit and add some <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/star2.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":star2:" alt=":star2:"> to your <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/speech_balloon.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":speech_balloon:" alt=":speech_balloon:">. Well we have a gift for you: -Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. :thumbsup: -``` +<img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/zap.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":zap:" alt=":zap:">You can use emoji anywhere GitLab Flavored Markdown is supported. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/v.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":v:" alt=":v:"> + +You can use it to point out a <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/bug.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":bug:" alt=":bug:"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/speak_no_evil.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":speak_no_evil:" alt=":speak_no_evil:"> patches. If someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/snail.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":snail:" alt=":snail:"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/birthday.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":birthday:" alt=":birthday:">. People <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/heart.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":heart:" alt=":heart:"> you for that. + +If you're new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/fearful.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":fearful:" alt=":fearful:">. You can join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/family.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":family:" alt=":family:">. Just look up one of the supported codes. + +Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":thumbsup:" alt=":thumbsup:"> + +<!-- vale gitlab.Markdown_emoji = YES --> -Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/monkey.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> around a bit and add some <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/star2.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> to your <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/speech_balloon.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">. Well we have a gift for you: +:::TabTitle Code -<img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/zap.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">You can use emoji anywhere GitLab Flavored Markdown is supported. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/v.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> +```markdown +Sometimes you want to :monkey: around a bit and add some :star2: to your +:speech_balloon:. Well we have a gift for you: + +:zap: You can use emoji anywhere GitLab Flavored Markdown is supported. :v: -You can use it to point out a<img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/bug.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/speak_no_evil.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> patches. If someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/snail.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/birthday.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">. People <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/heart.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> you for that. +You can use it to point out a :bug: or warn about :speak_no_evil: patches. +And if someone improves your really :snail: code, send them some :birthday:. +People :heart: you for that. -If you're new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/fearful.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">. You can join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/family.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">. Just look up one of the supported codes. +If you're new to this, don't be :fearful:. You can join the emoji :family:. +Just look up one of the supported codes. -Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> +Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) +for a list of all supported emoji codes. :thumbsup: +``` + +::EndTabs #### Emojis and your operating system @@ -249,6 +266,8 @@ this font installed by default. <!-- vale gitlab.Spelling = YES --> +To learn more about adding custom emojis, see [Custom emoji](award_emojis.md#custom-emojis). + ### Front matter Front matter is metadata included at the beginning of a Markdown document, preceding @@ -362,6 +381,10 @@ _KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX. This syntax also works for the Asciidoctor `:stem: latexmath`. For details, see the [Asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activating-stem-support). +To prevent malicious activity, GitLab renders only the first 50 inline math instances. +The number of math blocks is also limited based on render time. If the limit is exceeded, +GitLab renders the excess math instances as text. + Math written between dollar signs with backticks (``$`...`$``) or single dollar signs (`$...$`) is rendered inline with the text. @@ -371,15 +394,15 @@ the language declared as `math` is rendered on a separate line: ````markdown This math is inline: $`a^2+b^2=c^2`$. -This math is on a separate line: +This math is on a separate line using a ```` ```math ```` block: ```math a^2+b^2=c^2 ``` -This math is on a separate line: $$a^2+b^2=c^2$$ +This math is on a separate line using inline `$$`: $$a^2+b^2=c^2$$ -This math is on a separate line: +This math is on a separate line using a `$$...$$` block: $$ a^2+b^2=c^2 @@ -388,23 +411,15 @@ $$ This math is inline: $`a^2+b^2=c^2`$. -This math is on a separate line: +This math is on a separate line using a ```` ```math ```` block: ```math a^2+b^2=c^2 ``` -This math is on a separate line: $$a^2+b^2=c^2$$ +This math is on a separate line using inline `$$`: $$a^2+b^2=c^2$$ -This math is on a separate line: - -$$ -a^2+b^2=c^2 -$$ - -This math is on a separate line: $$a^2+b^2=c^2$$ - -This math is on a separate line: +This math is on a separate line using a `$$...$$` block: $$ a^2+b^2=c^2 @@ -736,13 +751,17 @@ you can quote that without having to manually prepend `>` to every line! ``` ->>> -If you paste a message from somewhere else - -that spans multiple lines, +<!-- +Use a standard blockquote here until https://gitlab.com/gitlab-org/gitlab/-/issues/390290 +gets properly fixed. The mixture of HTML comments and HTML tags +trigger this problem. +--> -you can quote that without having to manually prepend `>` to every line! ->>> +> If you paste a message from somewhere else +> +> that spans multiple lines, +> +> you can quote that without having to manually prepend `>` to every line! ### Code spans and blocks @@ -807,7 +826,7 @@ Tildes are OK too. If this section isn't rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md#colored-code-and-syntax-highlighting). -GitLab uses the [Rouge Ruby library](http://rouge.jneen.net/) for more colorful syntax +GitLab uses the [Rouge Ruby library](https://github.com/rouge-ruby/rouge) for more colorful syntax highlighting in code blocks. For a list of supported languages visit the [Rouge project wiki](https://github.com/rouge-ruby/rouge/wiki/List-of-supported-languages-and-lexers). Syntax highlighting is supported only in code blocks, so you can't highlight inline code. diff --git a/doc/user/okrs.md b/doc/user/okrs.md index 7025dc916ac..a61c395ab00 100644 --- a/doc/user/okrs.md +++ b/doc/user/okrs.md @@ -8,8 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103355) in GitLab 15.6 [with a flag](../administration/feature_flags.md) named `okrs_mvc`. Disabled by default. -WARNING: -OKRs are in [**Alpha**](../policy/alpha-beta-support.md#alpha-features). +OKRs are an [Experiment](../policy/alpha-beta-support.md#experiment). For the OKR feature roadmap, see [epic 7864](https://gitlab.com/groups/gitlab-org/-/epics/7864). FLAG: @@ -17,7 +16,7 @@ On self-managed GitLab, by default this feature is not available. To make it ava On GitLab.com, this feature is not available. The feature is not ready for production use. -[Objectives and key results](https:://en.wikipedia.org/wiki/OKR) (OKRs) are a framework for setting +[Objectives and key results](https://en.wikipedia.org/wiki/OKR) (OKRs) are a framework for setting and tracking goals that are aligned with your organization's overall strategy and vision. The objective and the key result in GitLab share many features. In the documentation, the term @@ -119,7 +118,6 @@ To edit an OKR: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) in GitLab 15.7 [with a flag](../administration/feature_flags.md) named `work_items_mvc_2`. Disabled by default. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) to feature flag named `work_items_mvc` in GitLab 15.8. Disabled by default. -> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/334812) in GitLab 15.10. > - Changing activity sort order [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) in GitLab 15.8. > - Filtering activity [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/389971) in GitLab 15.10. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/334812) in GitLab 15.10. diff --git a/doc/user/operations_dashboard/index.md b/doc/user/operations_dashboard/index.md index 493771fa30a..d013e2813f7 100644 --- a/doc/user/operations_dashboard/index.md +++ b/doc/user/operations_dashboard/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/packages/container_registry/reduce_container_registry_storage.md b/doc/user/packages/container_registry/reduce_container_registry_storage.md index 9419451fd5d..f873453e049 100644 --- a/doc/user/packages/container_registry/reduce_container_registry_storage.md +++ b/doc/user/packages/container_registry/reduce_container_registry_storage.md @@ -21,7 +21,7 @@ The Usage Quotas page (**Settings > Usage Quotas > Storage**) displays storage u This page includes the Container Registry usage, which is only available on GitLab.com. Measuring usage is only possible on the new version of the GitLab Container Registry backed by a metadata database. Support for improvements is proposed in epic [5523](https://gitlab.com/groups/gitlab-org/-/epics/5523). -You cannot use the Container Registry in self-managed instances, but epic [5521](https://gitlab.com/groups/gitlab-org/-/epics/5521) proposes to change this behavior. +On self-managed instances, you cannot use the Container Registry with a metadata database. For the plan to change this behavior, see [epic 5521](https://gitlab.com/groups/gitlab-org/-/epics/5521). Image layers stored in the Container Registry are deduplicated at the root namespace level. @@ -112,6 +112,43 @@ to work with all third-party registries in the same predictable way. If you use Registry, this workaround is not required because we implemented a special tag delete operation. In this case, you can expect cleanup policies to be consistent and predictable. +#### Example cleanup policy workflow + +The interaction between the keep and remove rules for the cleanup policy can be complex. +For example, with a project with this cleanup policy configuration: + +- **Keep the most recent**: 1 tag per image name. +- **Keep tags matching**: `production-.*` +- **Remove tags older than**: 7 days. +- **Remove tags matching**: `.*`. + +And a container repository with these tags: + +- `latest`, published 2 hours ago. +- `production-v44`, published 3 days ago. +- `production-v43`, published 6 days ago. +- `production-v42`, published 11 days ago. +- `dev-v44`, published 2 days ago. +- `dev-v43`, published 5 day ago. +- `dev-v42`, published 10 days ago. +- `v44`, published yesterday. +- `v43`, published 12 days ago. +- `v42`, published 20 days ago. + +In this example, the tags that would be deleted in the next cleanup run are `dev-v42`, `v43`, and `v42`. +You can interpret the rules as applying with this precedence: + +1. The keep rules have highest precedence. Tags must be kept when they match **any** rule. + - The `latest` tag must be kept, because `latest` tags are always kept. + - The `production-v44`, `production-v43`, and `production-v42` tags must be kept, + because they match the **Keep tags matching** rule. + - The `v44` tag must be kept because it's the most recent, matching the **Keep the most recent** rule. +1. The remove rules have lower precedence, and tags are only deleted if **all** rules match. + For the tags not matching any keep rules (`dev-44`, `dev-v43`, `dev-v42`, `v43`, and `v42`): + - `dev-44` and `dev-43` do **not** match the **Remove tags older than**, and are kept. + - `dev-v42`, `v43`, and `v42` match both **Remove tags older than** and **Remove tags matching** + rules, so these three tags can be deleted. + ### Create a cleanup policy You can create a cleanup policy in [the API](#use-the-cleanup-policy-api) or the UI. @@ -119,8 +156,8 @@ You can create a cleanup policy in [the API](#use-the-cleanup-policy-api) or the To create a cleanup policy in the UI: 1. For your project, go to **Settings > Packages and registries**. -1. Expand the **Clean up image tags** section. -1. Complete the fields. +1. In the **Cleanup policies** section, select **Set cleanup rules**. +1. Complete the fields: | Field | Description | |----------------------------|-------------------------------------------------| diff --git a/doc/user/packages/debian_repository/index.md b/doc/user/packages/debian_repository/index.md index 05574c54112..5ce65bfd8aa 100644 --- a/doc/user/packages/debian_repository/index.md +++ b/doc/user/packages/debian_repository/index.md @@ -98,8 +98,11 @@ with one of the following: ## Create a Distribution -On the project-level, Debian packages are published using *Debian Distributions*. To publish -packages on the group level, create a distribution with the same `codename`. +At the project level, Debian packages are published with **Debian distributions**. At the +group level, Debian packages are aggregated from the projects in the group provided that: + +- The project visibility is set to `public`. +- The Debian `codename` for the group matches the Debian `codename` for the project. To create a project-level distribution using a personal access token: @@ -162,9 +165,9 @@ EOF dput --config=dput.cf --unchecked --no-upload-log gitlab <your_package>.changes ``` -## Directly upload a package +## Upload a package with explicit distribution and component -> Direct upload [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101838) in GitLab 15.9. +> Upload with explicit distribution and component [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101838) in GitLab 15.9. When you don't have access to `.changes` file, you can directly upload a `.deb` by passing distribution `codename` and target `component` as parameters with @@ -173,7 +176,8 @@ For example, to upload to component `main` of distribution `sid` using a persona ```shell curl --request PUT --user "<username>:<personal_access_token>" \ - "https://gitlab.example.com/api/v4/projects/<project_id>/packages/debian/?distribution=sid&component=main" \ + --get --data "distribution=sid" --data "component=main" \ + "https://gitlab.example.com/api/v4/projects/<project_id>/packages/debian/" \ --upload-file /path/to/your.deb ``` diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index 8dc3c98795b..d7cf33cc2a4 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -342,13 +342,21 @@ Authenticating with credentials from job payload (GitLab Registry) Make sure you are using the expected authentication mechanism. -### "Not Found" error when pulling image +### `Not Found` or `404` error when pulling image -Docker errors similar to the following may indicate that the user running the build job doesn't have -a minimum of the Guest role in the specified Dependency Proxy group: +Errors like these might indicate that the user running the job doesn't have +a minimum of the Guest role in the Dependency Proxy group: -```plaintext -ERROR: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found +- ```plaintext + ERROR: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found -failed to solve with frontend dockerfile.v0: failed to create LLB definition: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found -``` + failed to solve with frontend dockerfile.v0: failed to create LLB definition: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found + ``` + +- ```plaintext + ERROR: Job failed: failed to pull image "gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest" with specified policies [always]: + Error response from daemon: error parsing HTTP 404 response body: unexpected end of JSON input: "" (manager.go:237:1s) + ``` + +For more information about the work to improve the error messages in similar cases to `Access denied`, +see [issue 354826](https://gitlab.com/gitlab-org/gitlab/-/issues/354826). diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index 932de0bcde6..bad099e6733 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -22,6 +22,8 @@ do not support the other available mechanisms. The `user-id` is not checked and may be any value, and the `password` must be either a [personal access token](../../../api/rest/index.md#personalprojectgroup-access-tokens), a [CI/CD job token](../../../ci/jobs/ci_job_token.md), or a [deploy token](../../project/deploy_tokens/index.md). +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. + ## Publish a package file When you publish a package file, if the package does not exist, it is created. diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 7418977e0d1..2084de58bdb 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -14,15 +14,9 @@ packages, which can be easily consumed as a dependency in downstream projects. The GitLab [Container Registry](container_registry/index.md) is a secure and private registry for container images. It's built on open source software and completely integrated within GitLab. Use GitLab CI/CD to create and publish images. Use the GitLab [API](../../api/container_registry.md) to manage the registry across groups and projects. -## Infrastructure Registry +## Terraform Module Registry -The GitLab [Infrastructure Registry](infrastructure_registry/index.md) is a secure and private registry for infrastructure packages. You can use GitLab CI/CD to create and publish infrastructure packages. - -The Infrastructure Registry supports the following formats: - -| Package type | GitLab version | -| ------------ | -------------- | -| [Terraform Module](terraform_module_registry/index.md) | 14.0+ | +The GitLab [Terraform Module Registry](terraform_module_registry/index.md) is a secure and private registry for Terraform modules. You can use GitLab CI/CD to create and publish modules. ## Dependency Proxy diff --git a/doc/user/packages/infrastructure_registry/index.md b/doc/user/packages/infrastructure_registry/index.md index d06c5e7fb1e..8c1e8a2ad8a 100644 --- a/doc/user/packages/infrastructure_registry/index.md +++ b/doc/user/packages/infrastructure_registry/index.md @@ -1,102 +1,11 @@ --- -stage: Package -group: Package Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../terraform_module_registry/index.md' +remove_date: '2023-06-13' --- -# Infrastructure Registry **(FREE)** +This document was moved to [another location](../terraform_module_registry/index.md). -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3221) in GitLab 14.0. - -With the GitLab Infrastructure Registry, you can use GitLab projects as a -private registry for infrastructure packages. You can create and publish -packages with GitLab CI/CD, which can then be consumed from other private -projects. - -## View packages - -To view packages within your project: - -1. Go to the project. -1. Go to **Packages and registries > Infrastructure Registry**. - -You can search, sort, and filter packages on this page. - -For information on how to create and upload a package, view the GitLab -documentation for your package type: - -- [Terraform modules](../terraform_module_registry/index.md) - -## Use GitLab CI/CD to build packages - -To use [GitLab CI/CD](../../../ci/index.md) to build packages, you can -authenticate with the [`CI_JOB_TOKEN` predefined variable](../../../ci/variables/predefined_variables.md). - -CI/CD templates, which you can use to get started, are in [this repository](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). - -For more information about using CI/CD to build Terraform modules, -see [Publish a Terraform module by using CI/CD](../terraform_module_registry/index.md#publish-a-terraform-module-by-using-cicd). - -If you use CI/CD to build a package, you can find extended activity information -when you view the package details: - - - -You can see the pipeline that published the package as well as the commit and the user who triggered it. However, the history is limited to five updates per package. - -## Download a package - -To download a package: - -1. Go to **Packages and registries > Infrastructure Registry**. -1. Select the name of the package you want to download. -1. In the **Activity** section, select the name of the package you want to download. - -## Delete a package - -You cannot edit a package after you publish it in the Infrastructure Registry. Instead, you -must delete and recreate it. - -To delete a package, you must have suitable [permissions](../../permissions.md). - -You can delete packages by using [the API](../../../api/packages.md#delete-a-project-package) or the UI. - -To delete a package in the UI, from your project: - -1. Go to **Packages and registries > Infrastructure Registry**. -1. Find the name of the package you want to delete. -1. Select **Delete**. - -The package is permanently deleted. - -## Disable the Infrastructure Registry - -The Infrastructure Registry is automatically enabled. - -For self-managed instances, a GitLab administrator can -[disable](../../../administration/packages/index.md) **Packages and registries**, -which removes this menu item from the sidebar. - -You can also remove the Infrastructure Registry for a specific project: - -1. In your project, go to **Settings > General**. -1. Expand the **Visibility, project features, permissions** section and toggle **Packages** off (in gray). -1. Select **Save changes**. - -To enable it back, follow the same steps above and toggle it on (in blue). - -## How module resolution works - -When you upload a new module, GitLab generates a path for the module, for example, `https://gitlab.example.com/parent-group/my-infra-package`. - -- This path conforms with [the Terraform spec](https://www.terraform.io/internals/module-registry-protocol). -- The name of the path must be unique within the namespace. - -For projects in subgroups, GitLab checks that the module name does not already exist anywhere in the namespace, including all subgroups and the parent group. - -For example, if: - -- The project is `gitlab.example.com/parent-group/sub-group/my-project`. -- The infrastructure package is `my-infra-package`. - -The project name must be unique in all projects in all groups under `parent-group`. +<!-- This redirect file can be deleted after <2023-06-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 1899cdc213f..7a451ca283f 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -22,6 +22,8 @@ You need an token to publish a package. There are different tokens available dep Create a token and save it to use later in the process. +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. + ### Edit the `settings.xml` Add the following section to your @@ -64,6 +66,13 @@ The three endpoints are: - **Group-level**: Use when you want to install packages from many different projects in the same GitLab group. GitLab does not guarantee the uniqueness of package names within the group. You can have two projects with the same package name and package version. As a result, GitLab serves whichever one is more recent. - **Instance-level**: Use when you have many packages in different GitLab groups or in their own namespace. +For the instance-level endpoint, ensure the relevant section of your `pom.xml` in Maven looks like this: + +```xml + <groupId>group-slug.subgroup-slug</groupId> + <artifactId>project-slug</artifactId> +``` + **Only packages that have the same path as the project** are exposed by the instance-level endpoint. | Project | Package | Instance-level endpoint available | diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 6ce4aab930e..52fdda0d523 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -23,6 +23,8 @@ You need an token to publish a package. There are different tokens available dep Create a token and save it to use later in the process. +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. + ### Naming convention Depending on how the package is installed, you may need to adhere to the naming convention. @@ -122,21 +124,50 @@ You can install a package from a GitLab project or instance: - **Instance-level**: Use when you have many npm packages in different GitLab groups or in their own namespace. - **Project-level**: Use when you have few npm packages and they are not in the same GitLab group. -### Install from the instance level +### Authenticate to the Package Registry -WARNING: -To install a package from the instance level, the package must have been published following the scoped [naming convention](#naming-convention). +You must authenticate to the Package Registry to install a package from a private project. +No authentication is needed if the project is public. -1. Authenticate to the Package Registry +To authenticate with `npm`: - If you would like to install a package from a private project, you would have to authenticate to the Package Registry. Skip this step if the project is not private. +```shell +npm config set -- //your_domain_name/:_authToken=your_token +``` - ```shell - npm config set -- //your_domain_name/api/v4/packages/npm/:_authToken=your_token - ``` +With npm version 7 or earlier, use the full URL to the endpoint. - - Replace `your_domain_name` with your domain name, for example, `gitlab.com`. - - Replace `your_token` with a deploy token, group access token, project access token, or personal access token. +If you're installing: + +- From the instance level: + + ```shell + npm config set -- //your_domain_name/api/v4/packages/npm/:_authToken=your_token + ``` + + From the project level: + + ```shell + npm config set -- //your_domain_name/api/v4/projects/your_project_id/packages/npm/:_authToken=your_token + ``` + +In these examples: + +- Replace `your_domain_name` with your domain name, for example, `gitlab.com`. +- Replace `your_project_id` is your project ID, found on the project's home page. +- Replace `your_token` with a deploy token, group access token, project access token, or personal access token. + +NOTE: +Starting with npm version 8, you can [use a URI fragment instead of a full URL](https://docs.npmjs.com/cli/v8/configuring-npm/npmrc?v=true#auth-related-configuration) +in the `_authToken` parameter. However, [group-level endpoints](https://gitlab.com/gitlab-org/gitlab/-/issues/299834) +are not supported. + +### Install from the instance level + +WARNING: +To install a package from the instance level, the package must have been published following the scoped [naming convention](#naming-convention). + +1. [Authenticate to the Package Registry](#authenticate-to-the-package-registry). 1. Set the registry @@ -156,17 +187,7 @@ To install a package from the instance level, the package must have been publish ### Install from the project level -1. Authenticate to the Package Registry - - If you would like to install a package from a private project, you would have to authenticate to the Package Registry. Skip this step if the project is not private. - - ```shell - npm config set -- //your_domain_name/api/v4/projects/your_project_id/packages/npm/:_authToken=your_token - ``` - - - Replace `your_domain_name` with your domain name, for example, `gitlab.com`. - - Replace `your_project_id` is your project ID, found on the project's home page. - - Replace `your_token` with a deploy token, group access token, project access token, or personal access token. +1. [Authenticate to the Package Registry](#authenticate-to-the-package-registry). 1. Set the registry @@ -184,6 +205,39 @@ To install a package from the instance level, the package must have been publish npm install @scope/my-package ``` +## Deprecate a package + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/396763) in GitLab 16.0. + +You can deprecate a package so that a deprecation warning displays when the package is fetched. + +Pre-requisites: + +- The same [permissions](../../permissions.md) as deleting a package. +- [Authenticated to the package registry](#authentication-to-the-package-registry). + +From the command line, run: + +```shell +npm deprecate @scope/package "Deprecation message" +``` + +The CLI also accepts version ranges for `@scope/package`. For example: + +```shell +npm deprecate @scope/package "All package versions are deprecated" +npm deprecate @scope/package@1.0.1 "Only version 1.0.1 is deprecated" +npm deprecate @scope/package@"< 1.0.5" "All package versions less than 1.0.5 are deprecated" +``` + +### Remove deprecation warning + +To remove a package's deprecation warning, specify `""` (an empty string) for the message. For example: + +```shell +npm deprecate @scope/package "" +``` + ## Helpful hints ### Package forwarding to npmjs.com @@ -272,6 +326,8 @@ The GitLab npm repository supports the following commands for the npm CLI (`npm` - `npm dist-tag rm`: Delete a dist-tag. - `npm ci`: Install npm packages directly from your `package-lock.json` file. - `npm view`: Show package metadata. +- `npm pack`: Create a tarball from a package. +- `npm deprecate`: Deprecate a version of a package. ## Troubleshooting diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index 4595e0eebb9..c97ce9ec593 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -39,6 +39,8 @@ Some features such as [publishing](#publish-a-nuget-package) a package are only When asking for versions of a given NuGet package name, the GitLab Package Registry returns a maximum of 300 most recent versions. +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. + WARNING: Because of how NuGet handles credentials, the Package Registry rejects anonymous requests on the group-level endpoint. To work around this limitation, set up [authentication](#add-the-package-registry-as-a-source-for-nuget-packages). diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index cd60a229479..eb3fd082d33 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -71,9 +71,13 @@ NOTE: If you have not activated the "Package registry" feature for your project at **Settings > General > Visibility, project features, permissions**, you receive a 403 Forbidden response. Accessing package registry via deploy token is not available when external authorization is enabled. -## Use GitLab CI/CD to build packages +## Use GitLab CI/CD + +You can use [GitLab CI/CD](../../../ci/index.md) to build or import packages into +a package registry. + +### To build packages -You can use [GitLab CI/CD](../../../ci/index.md) to build packages. For Maven, NuGet, npm, Conan, Helm, and PyPI packages, and Composer dependencies, you can authenticate with GitLab by using the `CI_JOB_TOKEN`. @@ -97,6 +101,16 @@ when you view the package details: You can view which pipeline published the package, and the commit and user who triggered it. However, the history is limited to five updates of a given package. +### To import packages + +If you already have packages built in a different registry, you can import them +into your GitLab package registry with the [Packages Importer](https://gitlab.com/gitlab-org/ci-cd/package-stage/pkgs_importer) + +The Packages Importer runs a CI/CD pipeline that [can import these package types](https://gitlab.com/gitlab-org/ci-cd/package-stage/pkgs_importer#formats-supported): + +- NPM +- NuGet + ## Reduce storage usage For information on reducing your storage use for the Package Registry, see diff --git a/doc/user/packages/package_registry/supported_functionality.md b/doc/user/packages/package_registry/supported_functionality.md index 0c7813beae0..0f359c70525 100644 --- a/doc/user/packages/package_registry/supported_functionality.md +++ b/doc/user/packages/package_registry/supported_functionality.md @@ -35,7 +35,7 @@ Packages can be pulled from your project, group, or instance. | Package type | Project | Group | Instance | |-----------------------------------------------------|---------|-------|----------| | [Maven](../maven_repository/index.md) | Y | Y | Y | -| [npm](../npm_registry/index.md) | Y | Y | Y | +| [npm](../npm_registry/index.md) | Y | N | Y | | [NuGet](../nuget_repository/index.md) | Y | Y | N | | [PyPI](../pypi_repository/index.md) | Y | Y | N | | [Generic packages](../generic_packages/index.md) | Y | N | N | diff --git a/doc/user/packages/package_registry/supported_package_managers.md b/doc/user/packages/package_registry/supported_package_managers.md index 75b8c95a0fa..0c33bef5e1c 100644 --- a/doc/user/packages/package_registry/supported_package_managers.md +++ b/doc/user/packages/package_registry/supported_package_managers.md @@ -11,24 +11,20 @@ Not all package manager formats are ready for production use. The Package Registry supports the following package manager types: -| Package type | GitLab version | Status | -| ------------------------------------------------ | -------------- | ---------------------------------------------------------- | -| [Maven](../maven_repository/index.md) | 11.3+ | GA | -| [npm](../npm_registry/index.md) | 11.7+ | GA | -| [NuGet](../nuget_repository/index.md) | 12.8+ | GA | -| [PyPI](../pypi_repository/index.md) | 12.10+ | GA | -| [Generic packages](../generic_packages/index.md) | 13.5+ | GA | -| [Composer](../composer_repository/index.md) | 13.2+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6817) | -| [Conan](../conan_repository/index.md) | 12.6+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6816) | -| [Helm](../helm_repository/index.md) | 14.1+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6366) | -| [Debian](../debian_repository/index.md) | 14.2+ | [Alpha](https://gitlab.com/groups/gitlab-org/-/epics/6057) | -| [Go](../go_proxy/index.md) | 13.1+ | [Alpha](https://gitlab.com/groups/gitlab-org/-/epics/3043) | -| [Ruby gems](../rubygems_registry/index.md) | 13.10+ | [Alpha](https://gitlab.com/groups/gitlab-org/-/epics/3200) | +| Package type | GitLab version | Status | +| ------------------------------------------------ | -------------- | --------------------------------------------------------------- | +| [Maven](../maven_repository/index.md) | 11.3+ | GA | +| [npm](../npm_registry/index.md) | 11.7+ | GA | +| [NuGet](../nuget_repository/index.md) | 12.8+ | GA | +| [PyPI](../pypi_repository/index.md) | 12.10+ | GA | +| [Generic packages](../generic_packages/index.md) | 13.5+ | GA | +| [Composer](../composer_repository/index.md) | 13.2+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6817) | +| [Conan](../conan_repository/index.md) | 12.6+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6816) | +| [Helm](../helm_repository/index.md) | 14.1+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6366) | +| [Debian](../debian_repository/index.md) | 14.2+ | [Experiment](https://gitlab.com/groups/gitlab-org/-/epics/6057) | +| [Go](../go_proxy/index.md) | 13.1+ | [Experiment](https://gitlab.com/groups/gitlab-org/-/epics/3043) | +| [Ruby gems](../rubygems_registry/index.md) | 13.10+ | [Experiment](https://gitlab.com/groups/gitlab-org/-/epics/3200) | -[Status](../../../policy/alpha-beta-support.md): - -- Alpha: behind a feature flag and not officially supported. -- Beta: several known issues that may prevent expected use. -- GA (Generally Available): ready for production use at any scale. +[View what each status means](../../../policy/alpha-beta-support.md). You can also use the [API](../../../api/packages.md) to administer the Package Registry. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index e5b7f06f6ca..d23966f26fe 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -34,6 +34,8 @@ To do this, you can use: `read_package_registry`, `write_package_registry`, or both. - A [CI job token](#authenticate-with-a-ci-job-token). +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. + ### Authenticate with a personal access token To authenticate with a personal access token, edit the `~/.pypirc` file and add: diff --git a/doc/user/packages/terraform_module_registry/index.md b/doc/user/packages/terraform_module_registry/index.md index 66a42f398b0..d7f33dd9e79 100644 --- a/doc/user/packages/terraform_module_registry/index.md +++ b/doc/user/packages/terraform_module_registry/index.md @@ -4,30 +4,53 @@ group: Package Registry info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Terraform module registry **(FREE)** +# Terraform Module Registry **(FREE)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3221) in GitLab 14.0. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3221) in GitLab 14.0. +> - Infrastructure registry and Terraform Module Registry [merged](https://gitlab.com/gitlab-org/gitlab/-/issues/404075) into a single Terraform Module Registry feature in GitLab 15.11. -Publish Terraform modules in your project's Infrastructure Registry, then reference them using GitLab -as a Terraform module registry. +With the Terraform Module Registry, you can use GitLab projects as a +private registry for terraform modules. You can create and publish +modules with GitLab CI/CD, which can then be consumed from other private +projects. -## Authenticate to the Terraform module registry +## View Terraform modules -To authenticate to the Terraform module registry, you need either: +To view Terraform modules in your project: + +1. Go to the project. +1. On the left sidebar, select **Packages and registries > Terraform modules**. + +You can search, sort, and filter modules on this page. + +For information on how to create and upload a package, view the GitLab +documentation for your package type: + +- [Terraform modules](../terraform_module_registry/index.md) + +## Authenticate to the Terraform Module Registry + +To authenticate to the Terraform Module Registry, you need either: - A [personal access token](../../../api/rest/index.md#personalprojectgroup-access-tokens) with at least `read_api` rights. - A [CI/CD job token](../../../ci/jobs/ci_job_token.md). -## Publish a Terraform Module +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. -When you publish a Terraform Module, if it does not exist, it is created. +## Publish a Terraform module + +When you publish a Terraform module, if it does not exist, it is created. + +### Using the API + +You can publish Terraform modules by using the [Terraform Module Registry API](../../../api/packages/terraform-modules.md). Prerequisites: -- The package name and version [must be unique in the top-level namespace](../infrastructure_registry/index.md#how-module-resolution-works). +- The package name and version [must be unique in the top-level namespace](#how-module-resolution-works). - Your project and group names must not include a dot (`.`). For example, `source = "gitlab.example.com/my.group/project.name"`. - You must [authenticate with the API](../../../api/rest/index.md#authentication). If authenticating with a deploy token, it must be configured with the `write_package_registry` scope. -- The name of a module [must be unique within the scope of its group](../infrastructure_registry/index.md#how-module-resolution-works), otherwise an +- The name of a module [must be unique in the scope of its group](#how-module-resolution-works), otherwise an [error occurs](#troubleshooting). ```plaintext @@ -37,9 +60,9 @@ PUT /projects/:id/packages/terraform/modules/:module-name/:module-system/:module | Attribute | Type | Required | Description | | -------------------| --------------- | ---------| -------------------------------------------------------------------------------------------------------------------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/rest/index.md#namespaced-path-encoding). | -| `module-name` | string | yes | The package name. **Supported syntax**: One to 64 ASCII characters, including lowercase letters (a-z) and digits (0-9). The package name can't exceed 64 characters. -| `module-system` | string | yes | The package system. **Supported syntax**: One to 64 ASCII characters, including lowercase letters (a-z) and digits (0-9). The package system can't exceed 64 characters. More information can be found in the [Terraform Module Registry Protocol documentation](https://www.terraform.io/internals/module-registry-protocol). -| `module-version` | string | yes | The package version. It must be valid according to the [Semantic Versioning Specification](https://semver.org/). +| `module-name` | string | yes | The module name. **Supported syntax**: One to 64 ASCII characters, including lowercase letters (a-z) and digits (0-9). The module name can't exceed 64 characters. +| `module-system` | string | yes | The module system. **Supported syntax**: One to 64 ASCII characters, including lowercase letters (a-z) and digits (0-9). The module system can't exceed 64 characters. More information can be found in the [Terraform Module Registry protocol documentation](https://www.terraform.io/internals/module-registry-protocol). +| `module-version` | string | yes | The module version. It must be valid according to the [semantic versioning specification](https://semver.org/). Provide the file content in the request body. @@ -55,14 +78,6 @@ curl --fail-with-body --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/terraform/modules/my-module/my-system/0.0.1/file" ``` -Example response: - -```json -{ - "message":"201 Created" -} -``` - Example request using a deploy token: ```shell @@ -79,41 +94,13 @@ Example response: } ``` -## Reference a Terraform Module +### Using a CI/CD template (recommended) -Prerequisites: - -- You need to [authenticate with the API](../../../api/rest/index.md#authentication). If authenticating with a personal access token, it must be configured with the `read_api` scope. - -Authentication tokens (Job Token or Personal Access Token) can be provided for `terraform` in your `~/.terraformrc` file: - -```plaintext -credentials "gitlab.com" { - token = "<TOKEN>" -} -``` - -Where `gitlab.com` can be replaced with the hostname of your self-managed GitLab instance. - -You can then refer to your Terraform Module from a downstream Terraform project: - -```plaintext -module "<module>" { - source = "gitlab.com/<namespace>/<module-name>/<module-system>" -} -``` - -Where `<namespace>` is the [namespace](../../../user/namespace/index.md) of the Terraform module registry. - -## Publish a Terraform module by using CI/CD - -> CI/CD template [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110493) in GitLab 15.9. - -### Use a CI/CD template (recommended) +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110493) in GitLab 15.9. You can use the [`Terraform-Module.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform-Module.gitlab-ci.yml) or the advanced [`Terraform/Module-Base.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Module-Base.gitlab-ci.yml) -CI/CD template to publish a Terraform module to the GitLab Terraform Registry: +CI/CD template to publish a Terraform module to the GitLab terraform registry: ```yaml include: @@ -124,7 +111,7 @@ The pipeline contains the following jobs: - `fmt` - Validate the formatting of the Terraform module. - `kics-iac-sast` - Test the Terraform module for security issues. -- `deploy` - For tag pipelines only. Deploy the Terraform module to the GitLab Terraform Registry. +- `deploy` - For tag pipelines only. Deploy the Terraform module to the Terraform Module Registry. #### Pipeline variables @@ -137,7 +124,7 @@ You can configure the pipeline with the following variables: | `TERRAFORM_MODULE_SYSTEM` | `local` | The system or provider of your Terraform module targets. For example, `local`, `aws`, `google`. | | `TERRAFORM_MODULE_VERSION` | `${CI_COMMIT_TAG}` | The Terraform module version. You should follow the semantic versioning specification. | -### Deploy manually via CI/CD +### Using CI/CD manually To work with Terraform modules in [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` in place of the personal access token in your commands. @@ -166,14 +153,97 @@ upload: - if: $CI_COMMIT_TAG ``` -To trigger this upload job, add a Git tag to your commit. Ensure the tag follows the [Semantic Versioning Specification](https://semver.org/) that Terraform requires. The `rules:if: $CI_COMMIT_TAG` ensures that only tagged commits to your repository trigger the module upload job. +To trigger this upload job, add a Git tag to your commit. Ensure the tag follows the [Semantic versioning specification](https://semver.org/) that Terraform requires. The `rules:if: $CI_COMMIT_TAG` ensures that only tagged commits to your repository trigger the module upload job. For other ways to control jobs in your CI/CD pipeline, refer to the [`.gitlab-ci.yml`](../../../ci/yaml/index.md) keyword reference. +## Reference a Terraform module + +Prerequisites: + +- You need to [authenticate with the API](../../../api/rest/index.md#authentication). If authenticating with a personal access token, it must be configured with the `read_api` scope. + +Authentication tokens (Job Token or Personal Access Token) can be provided for `terraform` in your `~/.terraformrc` file: + +```terraform +credentials "gitlab.com" { + token = "<TOKEN>" +} +``` + +Where `gitlab.com` can be replaced with the hostname of your self-managed GitLab instance. + +You can then refer to your Terraform module from a downstream Terraform project: + +```terraform +module "<module>" { + source = "gitlab.com/<namespace>/<module-name>/<module-system>" +} +``` + +Where `<namespace>` is the [namespace](../../../user/namespace/index.md) of the Terraform Module Registry. + +## Download a Terraform module + +To download a Terraform module: + +1. On the left sidebar, select **Packages and registries > Terraform modules**. +1. Select the name of the module you want to download. +1. In the **Activity** section, select the name of the module you want to download. + +## How module resolution works + +When you upload a new module, GitLab generates a path for the module, for example, `https://gitlab.example.com/parent-group/my-infra-package`. + +- This path conforms with [the Terraform spec](https://www.terraform.io/internals/module-registry-protocol). +- The name of the path must be unique in the namespace. + +For projects in subgroups, GitLab checks that the module name does not already exist anywhere in the namespace, including all subgroups and the parent group. + +For example, if: + +- The project is `gitlab.example.com/parent-group/sub-group/my-project`. +- The Terraform module is `my-infra-package`. + +The project name must be unique in all projects in all groups under `parent-group`. + +## Delete a Terraform module + +You cannot edit a Terraform module after you publish it in the Terraform Module Registry. Instead, you +must delete and recreate it. + +To delete a module, you must have suitable [permissions](../../permissions.md). + +You can delete modules by using [the packages API](../../../api/packages.md#delete-a-project-package) or the UI. + +To delete a module in the UI, from your project: + +1. On the left sidebar, select **Packages and registries > Terraform modules**. +1. Find the name of the package you want to delete. +1. Select **Delete**. + +The package is permanently deleted. + +## Disable the Terraform Module Registry + +The Terraform Module Registry is automatically enabled. + +For self-managed instances, a GitLab administrator can +[disable](../../../administration/packages/index.md) **Packages and registries**, +which removes this menu item from the sidebar. + +You can also remove the Terraform Module Registry for a specific project: + +1. In your project, go to **Settings > General**. +1. Expand the **Visibility, project features, permissions** section and toggle **Packages** off (in gray). +1. Select **Save changes**. + +To enable it back, follow the same steps above and toggle it on (in blue). + ## Example projects -For examples of the Terraform module registry, check the projects below: +For examples of the Terraform Module Registry, check the projects below: -- The [_GitLab local file_ project](https://gitlab.com/mattkasa/gitlab-local-file) creates a minimal Terraform module and uploads it into the Terraform module registry using GitLab CI/CD. +- The [_GitLab local file_ project](https://gitlab.com/mattkasa/gitlab-local-file) creates a minimal Terraform module and uploads it into the Terraform Module Registry using GitLab CI/CD. - The [_Terraform module test_ project](https://gitlab.com/mattkasa/terraform-module-test) uses the module from the previous example. ## Troubleshooting diff --git a/doc/user/packages/yarn_repository/index.md b/doc/user/packages/yarn_repository/index.md index e756a912928..c8f4985a518 100644 --- a/doc/user/packages/yarn_repository/index.md +++ b/doc/user/packages/yarn_repository/index.md @@ -302,10 +302,8 @@ Then you can use `yarn add` to install your packages. ## Related topics -- For full helpful hints information, see the - [npm documentation](../npm_registry/index.md#helpful-hints). -- For Yarn 1 to Yarn 2+ migration information see the - [Yarn Migration Guide](https://yarnpkg.com/getting-started/migration). +- [npm documentation](../npm_registry/index.md#helpful-hints) +- [Yarn Migration Guide](https://yarnpkg.com/getting-started/migration) ## Troubleshooting diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 43029e37047..e4a3f60834f 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -23,6 +23,7 @@ The available roles are: - Developer - Maintainer - Owner +- Minimal Access (available for the top-level group only) A user assigned the Guest role has the least permissions, and the Owner has the most. @@ -58,7 +59,7 @@ The following table lists project permissions available for each role: |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|----------|-----------|------------|----------| | [Analytics](analytics/index.md):<br>View [issue analytics](analytics/issue_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [merge request analytics](analytics/merge_request_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Analytics](analytics/index.md):<br>View [value stream analytics](analytics/value_stream_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Analytics](analytics/index.md):<br>View [value stream analytics](group/value_stream_analytics/index.md) | ✓ | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [DORA metrics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [CI/CD analytics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [code review analytics](analytics/code_review_analytics.md) | | ✓ | ✓ | ✓ | ✓ | @@ -137,7 +138,7 @@ The following table lists project permissions available for each role: | [Package registry](packages/index.md):<br>Delete a package | | | | ✓ | ✓ | | [Package registry](packages/index.md):<br>Delete a file associated with a package | | | | ✓ | ✓ | | [Project operations](../operations/index.md):<br>View [Error Tracking](../operations/error_tracking.md) list | | ✓ | ✓ | ✓ | ✓ | -| [Project operations](../operations/index.md):<br>Manage [Feature Flags](../operations/feature_flags.md) | | | ✓ | ✓ | ✓ | +| [Project operations](../operations/index.md):<br>Manage [Feature flags](../operations/feature_flags.md) | | | ✓ | ✓ | ✓ | | [Project operations](../operations/index.md):<br>Manage [Error Tracking](../operations/error_tracking.md) | | | | ✓ | ✓ | | [Projects](project/index.md):<br>Download project | ✓ (1) | ✓ | ✓ | ✓ | ✓ | | [Projects](project/index.md):<br>Leave comments | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -327,7 +328,7 @@ This table shows granted privileges for jobs triggered by specific types of user | Push source and LFS | | | | | 1. Only if the triggering user is not an external one. -1. Only if the triggering user is a member of the project. See also [Usage of private Docker images with `if-not-present` pull policy](http://docs.gitlab.com/runner/security/index.html#usage-of-private-docker-images-with-if-not-present-pull-policy). +1. Only if the triggering user is a member of the project. See also [Usage of private Docker images with `if-not-present` pull policy](https://docs.gitlab.com/runner/security/index.html#usage-of-private-docker-images-with-if-not-present-pull-policy). ## Group members permissions @@ -423,39 +424,38 @@ nested groups if you have membership in one of its parents. For more information, see [subgroup memberships](group/subgroups/index.md#subgroup-membership). -## Users with minimal access **(PREMIUM)** +## Users with Minimal Access **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40942) in GitLab 13.4. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40942) in GitLab 13.4. +> - Support for inviting users with Minimal Access role [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106438) in GitLab 15.9. -Owners can add members with a "minimal access" role to a root group. Such users don't automatically have access to -projects and subgroups underneath. Owners must explicitly add these "minimal access" users to the specific subgroups and +Users with the Minimal Access role do not: + +- Count as licensed seats on self-managed Ultimate subscriptions or any GitLab.com subscriptions. +- Automatically have access to projects and subgroups in that root group. + +Owners must explicitly add these users to the specific subgroups and projects. -You can use minimal access to give the same member more than one role in a group: +You can use the Minimal Access role to give the same member more than one role in a group: -1. Add the member to the root group with a minimal access role. +1. Add the member to the root group with a Minimal Access role. 1. Invite the member as a direct member with a specific role in any subgroup or project in that group. -Because of an [outstanding issue](https://gitlab.com/gitlab-org/gitlab/-/issues/267996), when minimal access users: +Because of an [outstanding issue](https://gitlab.com/gitlab-org/gitlab/-/issues/267996), when a user with the Minimal Access role: -- Sign in with standard web authentication, they receive a `404` error when accessing the parent group. -- Sign in with Group SSO, they receive a `404` error immediately because they are redirected to the parent group page. +- Signs in with standard web authentication, they receive a `404` error when accessing the parent group. +- Signs in with Group SSO, they receive a `404` error immediately because they are redirected to the parent group page. To work around the issue, give these users the Guest role or higher to any project or subgroup within the parent group. -### Minimal access users take license seats - -Users with even a "minimal access" role are counted against your number of license seats. This -requirement does not apply for [GitLab Ultimate](https://about.gitlab.com/pricing/) -subscriptions. - ## Related topics - [The GitLab principles behind permissions](https://about.gitlab.com/handbook/product/gitlab-the-product/#permissions-in-gitlab) - [Members](project/members/index.md) - Customize permissions on [protected branches](project/protected_branches.md) - [LDAP user permissions](group/access_and_permissions.md#manage-group-memberships-via-ldap) -- [Value stream analytics permissions](analytics/value_stream_analytics.md#access-permissions-for-value-stream-analytics) +- [Value stream analytics permissions](group/value_stream_analytics/index.md#access-permissions-for-value-stream-analytics) - [Project aliases](../user/project/import/index.md#project-aliases) - [Auditor users](../administration/auditor_users.md) - [Confidential issues](project/issues/confidential_issues.md) @@ -539,8 +539,5 @@ the Owner role: ### Known issues - Additional permissions can only be applied to users with the Guest role. -- There is no visual distinction in the UI between the Guest role and the Guest role with additional permission. For more information, see [issue 384099](https://gitlab.com/gitlab-org/gitlab/-/issues/384099). - If a user with a custom role is shared with a group or project, their custom role is not transferred over with them. The user has the regular Guest role in the new group or project. -- If a custom role is deleted, the users associated with that custom role are also removed from the group. For more information, see [issue 370352](https://gitlab.com/gitlab-org/gitlab/-/issues/370352). -- The API endpoint for associating a custom role with a user only works for users with the Guest role in a group. A project member can be associated with a custom role, but not through the API yet. For more information, see [issue 385495](https://gitlab.com/gitlab-org/gitlab/-/issues/385495). -- The only way to remove a custom role from a user's membership to a Group is to delete the custom role, which deletes the user membership entirely. See [issue 387769](https://gitlab.com/gitlab-org/gitlab/-/issues/387769). +- You cannot use an [Auditor user](../administration/auditor_users.md) as a template for a custom role. diff --git a/doc/user/product_analytics/index.md b/doc/user/product_analytics/index.md index 1a6ad4edf02..4c5733ed5ec 100644 --- a/doc/user/product_analytics/index.md +++ b/doc/user/product_analytics/index.md @@ -4,21 +4,30 @@ group: Product Analytics info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Product analytics **(ULTIMATE)** +# Product analytics (Experiment) **(ULTIMATE)** -> - Introduced in GitLab 15.4 as an [Alpha](../../policy/alpha-beta-support.md#alpha-features) feature [with a flag](../../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default. +> - Introduced in GitLab 15.4 as an [Experiment](../../policy/alpha-beta-support.md#experiment) feature [with a flag](../../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default. > - `cube_api_proxy` revised to only reference the [Product Analytics API](../../api/product_analytics.md) in GitLab 15.6. > - `cube_api_proxy` removed and replaced with `product_analytics_internal_preview` in GitLab 15.10. +> - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_internal_preview`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_dashboards`. On GitLab.com, this feature is not available. This feature is not ready for production use. This page is a work in progress, and we're updating the information as we add more features. For more information, see the [group direction page](https://about.gitlab.com/direction/analytics/product-analytics/). -## How Product Analytics works +## How product analytics works + +Product analytics uses several tools: + +- [**Jitsu**](https://jitsu.com/docs) - A web and app event collection platform that provides a consistent API to collect user data and pass it through to ClickHouse. +- [**ClickHouse**](https://clickhouse.com/docs) - A database suited to store, query, and retrieve analytical data. +- [**Cube.js**](https://cube.dev/docs/) - An analytical graphing library that provides an API to run queries against the data stored in Clickhouse. + +The following diagram illustrates the product analytics flow: ```mermaid --- @@ -41,25 +50,20 @@ flowchart TB end ``` -Product Analytics uses several tools: - -- [**Jitsu**](https://jitsu.com/docs) - A web and app event collection platform that provides a consistent API to collect user data and pass it through to Clickhouse. -- [**Clickhouse**](https://clickhouse.com/docs) - A database suited to store, query, and retrieve analytical data. -- [**Cube.js**](https://cube.dev/docs/) - An analytical graphing library that provides an API to run queries against the data stored in Clickhouse. - ## Enable product analytics > - Introduced in GitLab 15.6 behind the [feature flag](../../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default. > - Moved to be behind the [feature flag](../../administration/feature_flags.md) named `product_analytics_admin_settings` in GitLab 15.7. Disabled by default. > - `cube_api_proxy` removed and replaced with `product_analytics_internal_preview` in GitLab 15.10. +> - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_admin_settings`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flags](../../administration/feature_flags.md) named `product_analytics_dashboards` and `product_analytics_admin_settings`. On GitLab.com, this feature is not available. This feature is not ready for production use. -You can enable and configure product analytics to track events -within your project applications on a self-managed instance. +To track events in your project applications on a self-managed instance, +you must enable and configure product analytics. Prerequisite: @@ -71,31 +75,37 @@ Prerequisite: 1. Select **Enable product analytics** and enter the configuration values. The following table shows the required configuration parameters and example values: - | Name | Value | - |------------------------------|------------------------------------------------------------| - | Jitsu host | `https://jitsu.gitlab.com` | - | Jitsu project ID | `g0maofw84gx5sjxgse2k` | - | Jitsu administrator email | `jitsu.admin@gitlab.com` | - | Jitsu administrator password | `<your_password>` | - | Clickhouse URL | `https://<username>:<password>@clickhouse.gitlab.com:8123` | - | Cube API URL | `https://cube.gitlab.com` | - | Cube API key | `25718201b3e9...ae6bbdc62dbb` | + | Name | Value | + |--------------------------------|------------------------------------------------------------| + | Configurator connection string | `https://test:test@configurator.gitlab.com` | + | Jitsu host | `https://jitsu.gitlab.com` | + | Jitsu project ID | `g0maofw84gx5sjxgse2k` | + | Jitsu administrator email | `jitsu.admin@gitlab.com` | + | Jitsu administrator password | `<your_password>` | + | Collector host | `https://collector.gitlab.com` | + | ClickHouse URL | `https://<username>:<password>@clickhouse.gitlab.com:8123` | + | Cube API URL | `https://cube.gitlab.com` | + | Cube API key | `25718201b3e9...ae6bbdc62dbb` | 1. Select **Save changes**. ## Product analytics dashboards -> Introduced in GitLab 15.5 behind the [feature flag](../../administration/feature_flags.md) named `product_analytics_internal_preview`. Disabled by default. +> - Introduced in GitLab 15.5 behind the [feature flag](../../administration/feature_flags.md) named `product_analytics_internal_preview`. Disabled by default. +> - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_internal_preview`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_dashboards`. On GitLab.com, this feature is not available. This feature is not ready for production use. -Each project can define an unlimited number of dashboards. These dashboards are defined using our YAML schema and stored -in the `.gitlab/product_analytics/dashboards/` directory of a project repository. The name of the file is the name of the dashboard, and visualizations are shared across dashboards. +Each project can have an unlimited number of dashboards. +These dashboards are defined using the GitLab YAML schema, and stored in the `.gitlab/product_analytics/dashboards/` directory of a project repository. +The name of the file is the name of the dashboard. +Each dashboard can contain one or more visualizations (charts), which are shared across dashboards. -Project maintainers can enforce approval rules on dashboard changes using features such as code owners and approval rules. Dashboards are versioned in source control with the rest of a project's code. +Project maintainers can enforce approval rules on dashboard changes using features such as code owners and approval rules. +Dashboards are versioned in source control with the rest of a project's code. ### View project dashboards @@ -110,17 +120,25 @@ To view a list of product analytics dashboards for a project: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Dashboards**. +1. From the list of available dashboards, select the dashboard you want to view. ### Define a dashboard To define a dashboard: -1. In `.gitlab/product_analytics/dashboards/`, create a directory named like the dashboard. Each dashboard should have its own directory. -1. In the new directory, create a `.yaml` file with the same name as the directory. This file contains the dashboard definition, and must conform to the JSON schema defined in `ee/app/validators/json_schemas/product_analytics_dashboard.json`. -1. In the `.gitlab/product_analytics/dashboards/visualizations/` directory, create a `yaml` file. This file defines the visualization type for the dashboard, and must conform to the schema in +1. In `.gitlab/product_analytics/dashboards/`, create a directory named like the dashboard. + + Each dashboard should have its own directory. +1. In the new directory, create a `.yaml` file with the same name as the directory. + + This file contains the dashboard definition. It must conform to the JSON schema defined in `ee/app/validators/json_schemas/product_analytics_dashboard.json`. +1. In the `.gitlab/product_analytics/dashboards/visualizations/` directory, create a `.yaml` file. + + This file defines the visualization type for the dashboard. It must conform to the schema in `ee/app/validators/json_schemas/product_analytics_visualization.json`. -The example below includes three dashboards and one visualization that applies to all dashboards. +For example, if you want to create three dashboards (Conversion funnels, Demographic breakdown, and North star metrics) +and one visualization (line chart) that applies to all dashboards, the file structure would be: ```plaintext .gitlab/product_analytics/dashboards @@ -136,13 +154,13 @@ The example below includes three dashboards and one visualization that applies t ## Funnel analysis -Funnel analysis can be used to understand the flow of users through your application and where +Use funnel analysis to understand the flow of users through your application, and where users drop out of a predefined flow (for example, a checkout process or ticket purchase). Each product can also define an unlimited number of funnels. -These funnels are defined using our YAML schema and stored in the `.gitlab/product_analytics/funnels/` directory of a project repository. +Like dashboards, funnels are defined using the GitLab YAML schema, and stored in the `.gitlab/product_analytics/funnels/` directory of a project repository. -Funnel definitions must include the keys `name`, `seconds_to_convert`, and an array of `steps`. +Funnel definitions must include the keys `name` and `seconds_to_convert`, and an array of `steps`. | Key | Description | |----------------------|----------------------------------------------------------| @@ -160,6 +178,8 @@ Each step must include the keys `name`, `target`, and `action`. ### Example funnel definition +The following example defines a funnel that tracks users who completed a purchase within one hour by going through three target pages: + ```yaml name: completed_purchase seconds_to_convert: 3600 @@ -212,23 +232,33 @@ The `afterDate` filter is not supported. Please use `beforeDate` or `inDateRange Exporting the raw event data from the underlying storage engine can help you debug and create datasets for data analysis. +Because Cube acts as an abstraction layer between the raw data and the API, the exported raw data has some caveats: + +- Data is grouped by the selected dimensions. Therefore, the exported data might be incomplete, unless including both `utcTime` and `userAnonymousId`. +- Data is by default limited to 10,000 rows, but you can increase the limit to maximum 50,000 rows. If your dataset has more than 50,000 rows, you must paginate through the results by using the `limit` and `offset` parameters. +- Data is always returned in JSON format. If you need it in a different format, you need to convert the JSON to the required format using a scripting language of your choice. + +[Issue 391683](https://gitlab.com/gitlab-org/gitlab/-/issues/391683) tracks efforts to implement a more scalable export solution. + ### Export raw data with Cube queries -You can [query the raw data with the REST API](../../api/product_analytics.md#send-query-request-to-cube) and convert the JSON output to any required format. +You can [query the raw data with the REST API](../../api/product_analytics.md#send-query-request-to-cube), +and convert the JSON output to any required format. -You can export the raw data for a specific dimension by passing a list of dimensions to the `dimensions` key. For example, the following query outputs the raw data for the attributes listed: +To export the raw data for a specific dimension, pass a list of dimensions to the `dimensions` key. +For example, the following query outputs the raw data for the attributes listed: ```json POST /api/v4/projects/PROJECT_ID/product_analytics/request/load?queryType=multi { + "query":{ "dimensions": [ "TrackedEvents.docEncoding", "TrackedEvents.docHost", "TrackedEvents.docPath", "TrackedEvents.docSearch", "TrackedEvents.eventType", - "TrackedEvents.idsAjsAnonymousId", "TrackedEvents.localTzOffset", "TrackedEvents.pageTitle", "TrackedEvents.src", @@ -238,16 +268,8 @@ POST /api/v4/projects/PROJECT_ID/product_analytics/request/load?queryType=multi "order": { "TrackedEvents.apiKey": "asc" } + } } ``` If the request is successful, the returned JSON includes an array of rows of results. - -### Caveats - -Because Cube acts as an abstraction layer between the raw data and the API, the exported raw data has some caveats: - -- Data is grouped by the selected dimensions. Therefore, the exported data might be incomplete, unless including both `utcTime` and `userAnonymousId`. -- Data is by default limited to 10,000 rows, but you can increase the limit to maximum 50,000 rows. If your dataset has more than 50,000 rows, you need to paginate through the results by using the `limit` and `offset` parameters. -- Data is always returned in JSON format. If you need it in a different format, you need to convert the JSON to the required format using a scripting language of your choice. -- [Issue 391683](https://gitlab.com/gitlab-org/gitlab/-/issues/391683) tracks the implementation of a more scalable export solution. diff --git a/doc/user/profile/account/create_accounts.md b/doc/user/profile/account/create_accounts.md index 08d23902095..f405dc42f46 100644 --- a/doc/user/profile/account/create_accounts.md +++ b/doc/user/profile/account/create_accounts.md @@ -36,9 +36,27 @@ To create a user manually: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users** (`/admin/users`). 1. Select **New user**. -1. Complete the fields. +1. Complete the required fields, such as name, username, and email. 1. Select **Create user**. +A reset link is sent to the user's email and they are forced to set their +password on first sign in. + +To set a user's password without relying on the email confirmation, after you +create a user following the previous steps: + +1. Select the user. +1. Select **Edit**. +1. Complete the password and password confirmation fields. +1. Select **Save changes**. + +The user can now sign in with the new username and password, and they are asked +to change the password you set up for them. + +NOTE: +If you wanted to create a test user, you could follow the previous steps +by providing a fake email and using the same password in the final confirmation. + ## Create users through authentication integrations Users are: diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index fe2e2acaae3..a26f027f329 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -273,7 +273,7 @@ feature flag after WebAuthn devices have been registered, these devices are not On GitLab.com, WebAuthn devices are available. FLAG: -On self-managed GitLab, by default, optional one-time password authentication for WebAuthn devices is available. To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `webauthn_without_topt`. +On self-managed GitLab, by default, optional one-time password authentication for WebAuthn devices is not available. To enable the feature, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `webauthn_without_totp`. On GitLab.com, this feature is available. WebAuthn is [supported by](https://caniuse.com/#search=webauthn) the following: diff --git a/doc/user/profile/achievements.md b/doc/user/profile/achievements.md new file mode 100644 index 00000000000..c8456a80e69 --- /dev/null +++ b/doc/user/profile/achievements.md @@ -0,0 +1,235 @@ +--- +stage: Data Stores +group: Tenant Scale +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Achievements (Experiment) **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113156) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `achievements`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `achievements`. +The feature is not ready for production use. + +Achievements are a way to reward users for their activity on GitLab. +As a namespace maintainer or owner, you can create custom achievements for specific contributions, which you can award to or revoke from users based on your criteria. + +As a user, you can collect achievements to highlight your contributions to different projects or groups on your profile. +An achievement consists of a name, a description, and an avatar. + + + +This feature is an Experiment. +For more information about planned work, see [epic 9429](https://gitlab.com/groups/gitlab-org/-/epics/9429). +Tell us about your use cases by leaving comments in the epic. + +## Types of achievement + +Programmatically, there is only one way to create, award, revoke, or delete achievements. + +Practically, you can differentiate between achievements that are awarded: + +- Once and irrevocable. For example, a "First contribution merged" achievement. +- Once and revocable. For example, a "Core team member" achievement. +- Multiple times. For example, a "Contributor of the month" achievement. + +## View a user's achievements + +You can view a user's achievements on their profile page. + +Prerequisites: + +- The user profile must be public. +- You must be a member of the namespace awarding the achievement, or the namespace must be public. + +To view a user's achievements: + +1. Go to the user's profile page. +1. Below the user's avatar, see their achievements. +1. To view details about an achievement, hover over it. + It displays the following information: + + - Name of the achievement + - Description of the achievement + - Date when the achievement was awarded to the user + - Namespace that awarded the achievement + +To retrieve a list of a user's achievements, query the [`user` GraphQL type](../../api/graphql/reference/index.md#user). + +```graphql +query { + user(username: "<username>") { + userAchievements { + nodes { + achievement { + name + description + avatarUrl + namespace { + fullPath + name + } + } + } + } + } +} +``` + +## Create an achievement + +You can create custom achievements to award for specific contributions. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To create an achievement, call the [`achievementsCreate` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementscreate). + +```graphql +mutation achievementsCreate($file: Upload!) { + achievementsCreate( + input: { + namespaceId: "gid://gitlab/Namespace/<namespace id>", + name: "<name>", + description: "<description>", + avatar: $file} + ) { + errors + achievement { + id + name + description + avatarUrl + } + } +} +``` + +## Update an achievement + +You can change the name, description, and avatar of an achievement at any time. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To update an achievement, call the [`achievementsUpdate` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementsupdate). + +```graphql +mutation achievementsUpdate($file: Upload!) { + achievementsUpdate( + input: { + achievementId: "gid://gitlab/Achievements::Achievement/<achievement id>", + name: "<new name>", + description: "<new description>", + avatar: $file} + ) { + errors + achievement { + id + name + description + avatarUrl + } + } +} +``` + +## Award an achievement + +You can award an achievement to a user to recognize their contributions. +The user receives an email notification when they are awarded an achievement. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To award an achievement to a user, call the [`achievementsAward` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementsaward). + +```graphql +mutation { + achievementsAward(input: { + achievementId: "gid://gitlab/Achievements::Achievement/<achievement id>", + userId: "gid://gitlab/User/<user id>" }) { + userAchievement { + id + achievement { + id + name + } + user { + id + username + } + } + errors + } +} +``` + +## Revoke an achievement + +You can revoke a user's achievement if you consider the user no longer meets the awarding criteria. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To revoke an achievement, call the [`achievementsRevoke` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementsrevoke). + +```graphql +mutation { + achievementsRevoke(input: { + userAchievementId: "gid://gitlab/Achievements::UserAchievement/<user achievement id>" }) { + userAchievement { + id + achievement { + id + name + } + user { + id + username + } + revokedAt + } + errors + } +} +``` + +## Delete an achievement + +If you consider you no longer need an achievement, you can delete it. +This will delete all related awarded and revoked instances of the achievement. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To delete an achievement, call the [`achievementsDelete` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementsdelete). + +```graphql +mutation { + achievementsDelete(input: { + achievementId: "gid://gitlab/Achievements::Achievement/<achievement id>" }) { + achievement { + id + name + } + errors + } +} +``` + +## Hide achievements + +If you don't want to display achievements on your profile, you can opt out. To do this: + +1. On the top bar, in the upper-right corner, select your avatar. +1. Select **Edit profile**. +1. In the **Main settings** section, clear the **Display achievements on your profile** checkbox. +1. Select **Update profile settings**. diff --git a/doc/user/profile/comment_templates.md b/doc/user/profile/comment_templates.md new file mode 100644 index 00000000000..b9cd595916c --- /dev/null +++ b/doc/user/profile/comment_templates.md @@ -0,0 +1,68 @@ +--- +stage: Create +group: Code Review +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: howto +--- + +# Comment templates **(FREE)** + +> - GraphQL support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352956) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `saved_replies`. Disabled by default. +> - User interface [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113232) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `saved_replies`. Disabled by default. Enabled for GitLab team members only. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `saved_replies`. +On GitLab.com, this feature is not available by default, but enabled for GitLab team members. + +With comment templates, create and reuse text for any text area in: + +- Merge requests, including diffs. +- Issues, including design management comments. +- Epics. +- Work items. + +Comment templates can be small, like approving a merge request and unassigning yourself from it, +or large, like chunks of boilerplate text you use frequently: + + + +For more information about the rollout plan for this feature, see [issue 352956](https://gitlab.com/gitlab-org/gitlab/-/issues/352956). + +## Use comment templates in a text area + +To include the text of a comment template in your comment: + +1. In the editor toolbar for your comment, select **Comment templates** (**{symlink}**). +1. Select your desired comment template. + +## Create comment templates + +To create a comment template for future use: + +1. On the top bar, in the upper-right corner, select your avatar. +1. From the dropdown list, select **Preferences**. +1. On the left sidebar, select **Comment templates** (**{symlink}**). +1. Provide a **Name** for your comment template. +1. Enter the **Content** of your reply. You can use any formatting you use in + other GitLab text areas. +1. Select **Save**, and the page reloads with your comment template shown. + +## View your comment templates + +To go to your comment templates: + +1. On the top bar, in the upper-right corner, select your avatar. +1. From the dropdown list, select **Preferences**. +1. On the left sidebar, select **Comment templates** (**{symlink}**). +1. Scroll to **My comment templates**. + +## Edit or delete comment templates + +To edit or delete a previously comment template: + +1. On the top bar, in the upper-right corner, select your avatar. +1. From the dropdown list, select **Preferences**. +1. On the left sidebar, select **Comment templates** (**{symlink}**). +1. Scroll to **My comment templates**, and identify the comment template you want to edit. +1. To edit, select **Edit** (**{pencil}**). +1. To delete, select **Delete** (**{remove}**), then select **Delete** again from the modal window. diff --git a/doc/user/profile/img/busy_indicator_note_header_v13_9.png b/doc/user/profile/img/busy_indicator_note_header_v13_9.png Binary files differdeleted file mode 100644 index 63301ebdc14..00000000000 --- a/doc/user/profile/img/busy_indicator_note_header_v13_9.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_notes_v13_9.png b/doc/user/profile/img/busy_indicator_notes_v13_9.png Binary files differdeleted file mode 100644 index 2efe075c72b..00000000000 --- a/doc/user/profile/img/busy_indicator_notes_v13_9.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_profile_page_v13_6.png b/doc/user/profile/img/busy_indicator_profile_page_v13_6.png Binary files differdeleted file mode 100644 index c8e969f38db..00000000000 --- a/doc/user/profile/img/busy_indicator_profile_page_v13_6.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_settings_menu_v13_6.png b/doc/user/profile/img/busy_indicator_settings_menu_v13_6.png Binary files differdeleted file mode 100644 index 711638541dd..00000000000 --- a/doc/user/profile/img/busy_indicator_settings_menu_v13_6.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_sidebar_collapsed_v13_9.png b/doc/user/profile/img/busy_indicator_sidebar_collapsed_v13_9.png Binary files differdeleted file mode 100644 index 3dca88ec8cc..00000000000 --- a/doc/user/profile/img/busy_indicator_sidebar_collapsed_v13_9.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_sidebar_v13_9.png b/doc/user/profile/img/busy_indicator_sidebar_v13_9.png Binary files differdeleted file mode 100644 index 83024b319bf..00000000000 --- a/doc/user/profile/img/busy_indicator_sidebar_v13_9.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_user_popovers_v13_6.png b/doc/user/profile/img/busy_indicator_user_popovers_v13_6.png Binary files differdeleted file mode 100644 index 16fb0e6556b..00000000000 --- a/doc/user/profile/img/busy_indicator_user_popovers_v13_6.png +++ /dev/null diff --git a/doc/user/profile/img/profile-preferences-syntax-themes.png b/doc/user/profile/img/profile-preferences-syntax-themes.png Binary files differdeleted file mode 100644 index 719df9410fc..00000000000 --- a/doc/user/profile/img/profile-preferences-syntax-themes.png +++ /dev/null diff --git a/doc/user/profile/img/profile-preferences-syntax-themes_v15_11.png b/doc/user/profile/img/profile-preferences-syntax-themes_v15_11.png Binary files differnew file mode 100644 index 00000000000..39cd35f3b5b --- /dev/null +++ b/doc/user/profile/img/profile-preferences-syntax-themes_v15_11.png diff --git a/doc/user/profile/img/user_profile_achievements_v15_11.png b/doc/user/profile/img/user_profile_achievements_v15_11.png Binary files differnew file mode 100644 index 00000000000..cf675cd6b06 --- /dev/null +++ b/doc/user/profile/img/user_profile_achievements_v15_11.png diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index d52f119ba09..191695bc7ab 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -135,11 +135,13 @@ To add links to other accounts: 1. On the top bar, in the upper-right corner, select your avatar. 1. Select **Edit profile**. -1. In the **Main settings** section, add your information from: - - Discord ([User ID](https://support.discord.com/hc/en-us/articles/206346498-Where-can-I-find-my-User-Server-Message-ID-)) - - LinkedIn - - Skype - - Twitter +1. In the **Main settings** section, add your: + - Discord [user ID](https://support.discord.com/hc/en-us/articles/206346498-Where-can-I-find-my-User-Server-Message-ID-). + - LinkedIn profile name. + - Skype username. + - Twitter @username. + + Your user ID or username must be 500 characters or less. 1. Select **Update profile settings**. ## Show private contributions on your user profile page @@ -164,7 +166,7 @@ To specify your pronouns: 1. On the top bar, in the upper-right corner, select your avatar. 1. Select **Edit profile**. -1. In the **Pronouns** text box, enter your pronouns. +1. In the **Pronouns** text box, enter your pronouns. The text must be 50 characters or less. 1. Select **Update profile settings**. ## Add your name pronunciation @@ -178,7 +180,7 @@ To add your name pronunciation: 1. On the top bar, in the upper-right corner, select your avatar. 1. Select **Edit profile**. -1. In the **Pronunciation** text box, enter how your name is pronounced. +1. In the **Pronunciation** text box, enter how your name is pronounced. The pronunciation must be plain text and 255 characters or less. 1. Select **Update profile settings**. ## Set your current status @@ -224,25 +226,7 @@ To set the busy status indicator, either: 1. Select **Edit profile**. 1. In the **Current status** section, select the **Set yourself as busy** checkbox. - The busy status is displayed in the user interface. - - Username: - - | Profile page | Settings menu | User popovers | - | --- | --- | --- | - |  |  |  | - - Issue and merge request sidebar: - - | Sidebar| Collapsed sidebar | - | --- | --- | - |  |  | - - Notes: - - | Notes | Note headers | - | --- | --- | - |  |  | + The busy status is displayed next to your name, every time your name is shown in the user interface. ## Set your time zone diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index e6675b4eff2..c0c967a3f18 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -1,5 +1,4 @@ --- -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/notifications.html' stage: Plan group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index 0f46c3d6d25..696a5f4b42e 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -39,11 +39,11 @@ The default theme is Indigo. You can choose between 10 themes: ## Dark mode -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28252) in GitLab 13.1 as an [Alpha](../../policy/alpha-beta-support.md#alpha-features) release. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28252) in GitLab 13.1 as an [Experiment](../../policy/alpha-beta-support.md#experiment) release. -GitLab has started work on dark mode! The dark mode Alpha release is available in the +GitLab has started work on dark mode! The dark mode Experiment release is available in the spirit of iteration and the lower expectations of -[Alpha versions](../../policy/alpha-beta-support.md#alpha-features). +[Experiment features](../../policy/alpha-beta-support.md#experiment). Progress on dark mode is tracked in the [Dark theme epic](https://gitlab.com/groups/gitlab-org/-/epics/2902). See the epic for: @@ -65,7 +65,7 @@ Dark theme only works with the **Dark** syntax highlighting theme. > Changing the default syntax highlighting theme for new users and users who are not signed in [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25129) in GitLab 15.10. -GitLab uses the [rouge Ruby library](http://rouge.jneen.net/ "Rouge website") +GitLab uses the [Rouge Ruby library](https://github.com/rouge-ruby/rouge) for syntax highlighting outside of any Editor context. The WebIDE (like Snippets) uses [Monaco Editor](https://microsoft.github.io/monaco-editor/) and it's provided [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) library for @@ -75,19 +75,7 @@ the respective libraries. Changing this setting allows you to customize the color theme when viewing any syntax highlighted code on GitLab. -The default syntax theme is White, and you can choose among 5 different themes: - -<!-- vale gitlab.Spelling = NO --> - -- White -- Dark -- Solarized light -- Solarized dark -- Monokai - -<!-- vale gitlab.Spelling = YES --> - - + Introduced in GitLab 13.6, the themes [Solarized](https://gitlab.com/gitlab-org/gitlab/-/issues/221034) and [Monokai](https://gitlab.com/gitlab-org/gitlab/-/issues/221034) also apply to the [Web IDE](../project/web_ide/index.md) and [Snippets](../snippets.md). @@ -199,16 +187,15 @@ Open an issue if you notice that using absolute times breaks a layout. ## Web IDE -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370139) in GitLab 15.7 [with a flag](../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370139) in GitLab 15.7 [with a flag](../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/371084) in GitLab 15.7. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115741) in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The [Web IDE Beta](../project/web_ide_beta/index.md) is -the default editing environment when the `vscode_web_ide` feature -flag is enabled. - -To stop using the Web IDE Beta: +the default editing environment. To stop using the Web IDE Beta: 1. In the **Web IDE** section, select the **Opt out of the Web IDE Beta** checkbox. 1. Select **Save changes**. diff --git a/doc/user/profile/saved_replies.md b/doc/user/profile/saved_replies.md index 302daceb31d..1f4e4f5fa51 100644 --- a/doc/user/profile/saved_replies.md +++ b/doc/user/profile/saved_replies.md @@ -1,61 +1,11 @@ --- -stage: Create -group: Code Review -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: howto +redirect_to: 'comment_templates.md' +remove_date: '2023-06-22' --- -# Saved replies **(FREE)** +This document was moved to [another location](comment_templates.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352956) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `saved_replies`. Disabled by default. - -With saved replies, create and reuse text for any text area in: - -- Merge requests, including diffs. -- Issues, including design management comments. -- Epics. -- Work items. - -Saved replies can be small, like approving a merge request and unassigning yourself from it, -or large, like chunks of boilerplate text you use frequently: - - - -## Use saved replies in a text area - -To include the text of a saved reply in your comment: - -1. In the editor toolbar for your comment, select **Saved replies** (**{symlink}**). -1. Select your desired saved reply. - -## Create saved replies - -To create a saved reply for future use: - -1. On the top bar, in the upper-right corner, select your avatar. -1. From the dropdown list, select **Preferences**. -1. On the left sidebar, select **Saved replies** (**{symlink}**). -1. Provide a **Name** for your saved reply. -1. Enter the **Content** of your reply. You can use any formatting you use in - other GitLab text areas. -1. Select **Save**, and the page reloads with your saved reply shown. - -## View your saved replies - -To go to your saved replies: - -1. On the top bar, in the upper-right corner, select your avatar. -1. From the dropdown list, select **Preferences**. -1. On the left sidebar, select **Saved replies** (**{symlink}**). -1. Scroll to **My saved replies**. - -## Edit or delete saved replies - -To edit or delete a previously saved reply: - -1. On the top bar, in the upper-right corner, select your avatar. -1. From the dropdown list, select **Preferences**. -1. On the left sidebar, select **Saved replies** (**{symlink}**). -1. Scroll to **My saved replies**, and identify the saved reply you want to edit. -1. To edit, select **Edit** (**{pencil}**). -1. To delete, select **Delete** (**{remove}**), then select **Delete** again from the modal window. +<!-- This redirect file can be deleted after <2023-06-22>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/badges.md b/doc/user/project/badges.md index 2d36a378b82..76933255820 100644 --- a/doc/user/project/badges.md +++ b/doc/user/project/badges.md @@ -62,7 +62,7 @@ You can access a test coverage report badge image by using the following link: https://gitlab.example.com/<namespace>/<project>/badges/<branch>/coverage.svg ``` -You can define the regular expression for the [coverage report](../../ci/pipelines/settings.md#merge-request-test-coverage-results) +You can define the regular expression for the [coverage report](../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) that each job log is matched against. This means that each job in the pipeline can have the test coverage percentage value defined. @@ -279,7 +279,7 @@ To add a new badge with a custom image to a group or project: 1. Select **Add badge**. To learn how to use custom images generated through a pipeline, see the documentation on -[accessing the latest job artifacts by URL](../../ci/pipelines/job_artifacts.md#access-the-latest-job-artifacts). +[accessing the latest job artifacts by URL](../../ci/jobs/job_artifacts.md#from-a-url). ## Placeholders diff --git a/doc/user/project/changelogs.md b/doc/user/project/changelogs.md index a64edd053b1..8410655c244 100644 --- a/doc/user/project/changelogs.md +++ b/doc/user/project/changelogs.md @@ -313,5 +313,5 @@ an error is produced when generating a changelog. ## Related topics -- [Changelog-related endpoints](../../api/repositories.md) in the Repositories API. -- Developer documentation for [changelog entries](../../development/changelog.md) in GitLab. +- [Changelog-related endpoints](../../api/repositories.md) in the Repositories API +- Developer documentation for [changelog entries](../../development/changelog.md) in GitLab diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index 3ed8a4a67cf..c4f4f220c9b 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md index a7fac2a1cea..bb85662d442 100644 --- a/doc/user/project/clusters/add_existing_cluster.md +++ b/doc/user/project/clusters/add_existing_cluster.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/clusters/add_gke_clusters.md b/doc/user/project/clusters/add_gke_clusters.md index e00c4370da9..c6561fffa0b 100644 --- a/doc/user/project/clusters/add_gke_clusters.md +++ b/doc/user/project/clusters/add_gke_clusters.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -77,7 +77,7 @@ cluster certificates: - **Zone** - Choose the [region zone](https://cloud.google.com/compute/docs/regions-zones/) under which to create the cluster. - **Number of nodes** - Enter the number of nodes you wish the cluster to have. - - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types) + - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-resource) of the Virtual Machine instance to base the cluster on. - **Enable Cloud Run for Anthos** - Check this if you want to use Cloud Run for Anthos for this cluster. See the [Cloud Run for Anthos section](#cloud-run-for-anthos) for more information. diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index bf2f6e28d9a..bba05f1e724 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/clusters/cluster_access.md b/doc/user/project/clusters/cluster_access.md index 0eea3ebb8cd..ff30da2ca98 100644 --- a/doc/user/project/clusters/cluster_access.md +++ b/doc/user/project/clusters/cluster_access.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/clusters/deploy_to_cluster.md b/doc/user/project/clusters/deploy_to_cluster.md index e2d0a010293..31d5a73757a 100644 --- a/doc/user/project/clusters/deploy_to_cluster.md +++ b/doc/user/project/clusters/deploy_to_cluster.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/clusters/gitlab_managed_clusters.md b/doc/user/project/clusters/gitlab_managed_clusters.md index 46e74eb8460..b321646d8d8 100644 --- a/doc/user/project/clusters/gitlab_managed_clusters.md +++ b/doc/user/project/clusters/gitlab_managed_clusters.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 490188b27b0..5ce1994ba36 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/clusters/multiple_kubernetes_clusters.md b/doc/user/project/clusters/multiple_kubernetes_clusters.md index 9167e6197d6..c8f49b1917d 100644 --- a/doc/user/project/clusters/multiple_kubernetes_clusters.md +++ b/doc/user/project/clusters/multiple_kubernetes_clusters.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index b1f2a9dbb79..a985436d67d 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index 0994bff4aa2..f9244177aa5 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -1,473 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'codeowners/index.md' +remove_date: '2023-07-07' --- -# Code Owners **(PREMIUM)** +This document was moved to [another location](codeowners/index.md). -> Moved to GitLab Premium in 13.9. - -Use the Code Owners feature to define who has expertise for specific parts of your project's codebase. -Define the owners of files and directories in a repository to: - -- **Require owners to approve changes.** Combine protected branches with Code Owners to require - experts to approve merge requests before they merge into a protected branch. -- **Identify owners.** Code Owner names are displayed on the files and directories they own: -  - -Use Code Owners in combination with merge request -[approval rules](merge_requests/approvals/rules.md) (either optional or required) -to build a flexible approval workflow: - -- Use **Code Owners** to ensure quality. Define the users who have domain expertise - for specific paths in your repository. -- Use **Approval rules** to define areas of expertise that don't correspond to specific - file paths in your repository. Approval rules help guide merge request creators to - the correct set of reviewers, such as frontend developers or a security team. - -For example: - -| Type | Name | Scope | Comment | -|------|------|--------|------------| -| Approval rule | UX | All files | A user experience (UX) team member reviews the user experience of all changes made in your project. -| Approval rule | Security | All files | A security team member reviews all changes for vulnerabilities. -| Code Owner approval rule | Frontend: Code Style | `*.css` files | A frontend engineer reviews CSS file changes for adherence to project style standards. -| Code Owner approval rule | Backend: Code Review | `*.rb` files | A backend engineer reviews the logic and code style of Ruby files. - -<div class="video-fallback"> - Video introduction: <a href="https://www.youtube.com/watch?v=RoyBySTUSB0">Code Owners</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube-nocookie.com/embed/RoyBySTUSB0" frameborder="0" allowfullscreen> </iframe> -</figure> - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> - -## View a file's Code Owner - -To view the Code Owners for a file: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Repository > Files**. -1. Go to the file or directory you want to see the Code Owners for. -1. Optional. Select your desired branch. - -GitLab shows the Code Owners at the top of the page. - -## Set up Code Owners - -1. Create a `CODEOWNERS` file in your [preferred location](#code-owners-file). -1. In the file, enter text that follows one of these patterns: - - ```plaintext - # Code Owners for a file - filename @username1 @username2 - - # Code Owners for a directory - directoryname/ @username1 @username2 - - # All group members as Code Owners for a file - filename @groupname - - # All group members as Code Owners for a directory - directoryname/ @groupname - ``` - -### Code Owners file - -A `CODEOWNERS` file (with no extension) specifies the users or -[shared groups](members/share_project_with_groups.md) responsible for -specific files and directories in a repository. - -Each repository uses a single `CODEOWNERS` file. GitLab checks these locations -in your repository in this order. The first `CODEOWNERS` file found is used, and -all others are ignored: - -1. In the root directory: `./CODEOWNERS`. -1. In the `docs` directory: `./docs/CODEOWNERS`. -1. In the `.gitlab` directory: `./.gitlab/CODEOWNERS`. - -### Make Code Owners eligible approvers or require their approval of MRs - -- [Add Code Owners as merge request approvers](merge_requests/approvals/rules.md#code-owners-as-eligible-approvers). -- Set up [Code Owner approval on a protected branch](protected_branches.md#require-code-owner-approval-on-a-protected-branch). - -### Groups as Code Owners - -> Group and subgroup hierarchy support was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32432) in GitLab 13.0. - -You can use members of groups and subgroups as Code Owners for projects: - -```mermaid -graph TD - A[Parent group X] -->|owns| B[Project A] - A -->|contains| C[Subgroup Y] - C -->|owns| D[Project B] - A-. inherits ownership .-> D -``` - -In this example: - -- **Parent group X** (`group-x`) owns **Project A**. -- **Parent group X** also contains a subgroup, **Subgroup Y**. (`group-x/subgroup-y`) -- **Subgroup Y** owns **Project B**. - -The eligible Code Owners are: - -- **Project A**: the members of **Group X** only, because **Project A** doesn't belong to **Subgroup Y**. -- **Project B**: the members of both **Group X** and **Subgroup Y**. - -You can [invite](members/share_project_with_groups.md) **Subgroup Y** to **Project A** -so that their members also become eligible Code Owners. - -```mermaid -graph LR - A[Parent group X] -->|owns| B[Project A] - A -->|also contains| C[Subgroup Y] - C -.->D{Invite Subgroup Y<br/>to Project A?} -.->|yes| F[Approvals can be<br/> required] -.-> B - D{Invite Subgroup Y<br/>to Project A?} -.->|no| I[Subgroup Y cannot be<br /> an approver] -.-> B - C -.->E{Add Subgroup Y<br/>as Code Owners<br/>to Project A?} -.->|yes| H[Approvals can only<br/>be optional] -.-> B -``` - -If you do not invite **Subgroup Y** to **Project A**, but make them Code Owners, their approval -of the merge request becomes optional. - -Inviting **Subgroup Y** to a parent group of **Project A** -[is not supported](https://gitlab.com/gitlab-org/gitlab/-/issues/288851). To set **Subgroup Y** as -Code Owners, add this group directly to the project itself. - -NOTE: -For approval to be required, groups as Code Owners must have a direct membership -(not inherited membership) in the project. Approval can only be optional for groups -that inherit membership. Members in the Code Owners group also must be direct members, -and not inherit membership from any parent groups. - -#### Add a group as a Code Owner - -To set a group as a Code Owner: - -In the `CODEOWNERS` file, enter text that follows one of these patterns: - -```plaintext -# All group members as Code Owners for a file -file.md @group-x - -# All subgroup members as Code Owners for a file -file.md @group-x/subgroup-y - -# All group and subgroup members as Code Owners for a file -file.md @group-x @group-x/subgroup-y -``` - -### Define more specific owners for more specifically defined files or directories - -When a file or directory matches multiple entries in the `CODEOWNERS` file, -the users from last pattern matching the file or directory are used. This enables you -to define more specific owners for more specifically defined files or directories, when -you order the entries in a sensible way. - -For example, in the following `CODEOWNERS` file: - -```plaintext -# This line would match the file terms.md -*.md @doc-team - -# This line would also match the file terms.md -terms.md @legal-team -``` - -The Code Owner for `terms.md` would be `@legal-team`. - -If you use sections, the last pattern matching the file or directory for each section is used. -For example, in a `CODEOWNERS` file using sections: - -```plaintext -[README Owners] -README.md @user1 @user2 -internal/README.md @user4 - -[README other owners] -README.md @user3 -``` - -The Code Owners for the `README.md` in the root directory are `@user1`, `@user2`, -and `@user3`. The Code Owners for `internal/README.md` are `@user4` and `@user3`. - -Only one CODEOWNERS pattern per section is matched to a file path. - -### Organize Code Owners by putting them into sections - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in GitLab 13.2 [with a flag](../../administration/feature_flags.md) named `sectional_codeowners`. Disabled by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42389) in GitLab 13.4. Feature flag `sectional_codeowners` removed. - -You can organize Code Owners by putting them into named sections. - -You can use sections for shared directories, so that multiple -teams can be reviewers. - -To add a section to the `CODEOWNERS` file, enter a section name in brackets, -followed by the files or directories, and users, groups, or subgroups: - -```plaintext -[README Owners] -README.md @user1 @user2 -internal/README.md @user2 -``` - -Each Code Owner in the merge request widget is listed under a label. -The following image shows a **Groups** and **Documentation** section: - - - -#### Set default owner for a section **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371711) in GitLab 15.11 [with a flag](../../administration/feature_flags.md) named `codeowners_default_owners`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `codeowners_default_owners`. -The feature is not ready for production use. - -WARNING: -To disable this feature flag after setting default owners per section, edit your -CODEOWNERS file to [list Code Owners per line](#set-up-code-owners). - -If multiple file paths inside a section share the same ownership, define a default -Code Owner for the section. All paths in that section inherit this default, unless -you override the section default on a specific line. - -Default owners are applied when specific owners are not specified for file paths. -Specific owners defined beside the file path override default owners: - -```plaintext -[Documentation] @docs-team -docs/ -README.md - -[Database] @database-team -model/db/ -config/db/database-setup.md @docs-team -``` - -In this example: - -- `@docs-team` owns all items in the `Documentation` section. -- `@database-team` owns all items in the `Database` section except - `config/db/database-setup.md`, which has an override assigning it to `@docs-team`. - -To combine the syntax for default owners with [optional sections](#make-a-code-owners-section-optional) -and required approvals, place default owners at the end: - -```plaintext -[Documentation][2] @docs-team -docs/ -README.md - -^[Database] @database-team -model/db/ -config/db/database-setup.md @docs-team -``` - -#### Sections with duplicate names - -If multiple sections have the same name, they are combined. -Also, section headings are not case-sensitive. For example: - -```plaintext -[Documentation] -ee/docs/ @docs -docs/ @docs - -[Database] -README.md @database -model/db/ @database - -[DOCUMENTATION] -README.md @docs -``` - -This code results in three entries under the **Documentation** section header, and two -entries under **Database**. The entries defined under the sections **Documentation** and -**DOCUMENTATION** are combined, using the case of the first section. - -#### Make a Code Owners section optional - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/232995) in GitLab 13.8. - -You can designate optional sections in your Code Owners file. Prepend the -section name with the caret `^` character to treat the entire section as optional. -Optional sections enable you to designate responsible parties for various parts -of your codebase, but not require approval from them. This approach provides -a more relaxed policy for parts of your project that are frequently updated, -but don't require stringent reviews. - -In this example, the `[Go]` section is optional: - -```plaintext -[Documentation] -*.md @root - -[Ruby] -*.rb @root - -^[Go] -*.go @root -``` - -The optional Code Owners section displays in merge requests under the **Approval Rules** area: - - - -If a section is duplicated in the file, and one of them is marked as optional and the other isn't, the section is required. - -Optional sections in the `CODEOWNERS` file are treated as optional only -when changes are submitted by using merge requests. If a change is submitted directly -to the protected branch, approval from Code Owners is still required, even if the -section is marked as optional. - -### Require multiple approvals from Code Owners - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335451) in GitLab 15.9. - -You can require multiple approvals for the Code Owners sections under the Approval Rules area in merge requests. -Append the section name with a number `n` in brackets. This requires `n` approvals from the Code Owners in this section. -Please note valid entries for `n` are integers `≥ 1`. `[1]` is optional as it is the default. Invalid values for `n` are treated as `1`. - -WARNING: -[Issue #384881](https://gitlab.com/gitlab-org/gitlab/-/issues/385881) proposes changes -to the behavior of this setting. Do not intentionally set invalid values. They may -become valid in the future, and cause unexpected behavior. - -Please confirm you enabled `Require approval from code owners` in `Settings > Repository > Protected branches`, otherwise the Code Owner approvals will be optional. - -In this example, the `[Documentation]` section requires 2 approvals: - -```plaintext -[Documentation][2] -*.md @tech-writer-team - -[Ruby] -*.rb @dev-team -``` - -The `Documentation` Code Owners section under the **Approval Rules** area displays 2 approvals are required: - - - -### Allowed to Push - -The Code Owner approval and protected branch features do not apply to users who -are **Allowed to push**. - -## Example `CODEOWNERS` file - -```plaintext -# This is an example of a CODEOWNERS file. -# Lines that start with `#` are ignored. - -# app/ @commented-rule - -# Specify a default Code Owner by using a wildcard: -* @default-codeowner - -# Specify multiple Code Owners by using a tab or space: -* @multiple @code @owners - -# Rules defined later in the file take precedence over the rules -# defined before. -# For example, for all files with a filename ending in `.rb`: -*.rb @ruby-owner - -# Files with a `#` can still be accessed by escaping the pound sign: -\#file_with_pound.rb @owner-file-with-pound - -# Specify multiple Code Owners separated by spaces or tabs. -# In the following case the CODEOWNERS file from the root of the repo -# has 3 Code Owners (@multiple @code @owners): -CODEOWNERS @multiple @code @owners - -# You can use both usernames or email addresses to match -# users. Everything else is ignored. For example, this code -# specifies the `@legal` and a user with email `janedoe@gitlab.com` as the -# owner for the LICENSE file: -LICENSE @legal this_does_not_match janedoe@gitlab.com - -# Use group names to match groups, and nested groups to specify -# them as owners for a file: -README @group @group/with-nested/subgroup - -# End a path in a `/` to specify the Code Owners for every file -# nested in that directory, on any level: -/docs/ @all-docs - -# End a path in `/*` to specify Code Owners for every file in -# a directory, but not nested deeper. This code matches -# `docs/index.md` but not `docs/projects/index.md`: -/docs/* @root-docs - -# Include `/**` to specify Code Owners for all subdirectories -# in a directory. This rule matches `docs/projects/index.md` or -# `docs/development/index.md` -/docs/**/*.md @root-docs - -# This code makes matches a `lib` directory nested anywhere in the repository: -lib/ @lib-owner - -# This code match only a `config` directory in the root of the repository: -/config/ @config-owner - -# If the path contains spaces, escape them like this: -path\ with\ spaces/ @space-owner - -# Code Owners section: -[Documentation] -ee/docs @docs -docs @docs - -# Use of default owners for a section. In this case, all files (*) are owned by -the dev team except the README.md and data-models which are owned by other teams. -[Development] @dev-team -* -README.md @docs-team -data-models/ @data-science-team - -# This section is combined with the previously defined [Documentation] section: -[DOCUMENTATION] -README.md @docs -``` - -## Technical Resources - -[Code Owners development guidelines](../../../ee/development/code_owners/index.md) - -## Troubleshooting - -### Approvals shown as optional - -A Code Owner approval rule is optional if any of these conditions are true: - -- The user or group are not a member of the project. Code Owners [cannot inherit from parent groups](https://gitlab.com/gitlab-org/gitlab/-/issues/288851/). -- [Code Owner approval on a protected branch](protected_branches.md#require-code-owner-approval-on-a-protected-branch) has not been set up. -- The section is [marked as optional](#make-a-code-owners-section-optional). - -### Approvals do not show - -Code Owner approval rules only update when the merge request is created. -If you update the `CODEOWNERS` file, close the merge request and create a new one. - -### User not shown as possible approver - -A user might not show as an approver on the Code Owner merge request approval rules -if any of these conditions are true: - -- A rule prevents the specific user from approving the merge request. - Check the project [merge request approval](merge_requests/approvals/settings.md#edit-merge-request-approval-settings) settings. -- A Code Owner group has a visibility of **private**, and the current user is not a - member of the Code Owner group. -- Current user is an external user who does not have permission to the internal Code Owner group. - -### Approval rule is invalid. GitLab has approved this rule automatically to unblock the merge request - -This message may appear if an approval rule uses a Code Owner that is not a direct member of the project. -Check that the group or user has been invited to the project. +<!-- This redirect file can be deleted after <2023-07-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/codeowners/index.md b/doc/user/project/codeowners/index.md new file mode 100644 index 00000000000..bbb39179975 --- /dev/null +++ b/doc/user/project/codeowners/index.md @@ -0,0 +1,376 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Code Owners **(PREMIUM)** + +> Moved to GitLab Premium in 13.9. + +Use the Code Owners feature to define who has expertise for specific parts of your project's codebase. +Define the owners of files and directories in a repository to: + +- **Require owners to approve changes.** Combine protected branches with Code Owners to require + experts to approve merge requests before they merge into a protected branch. +- **Identify owners.** Code Owner names are displayed on the files and directories they own: +  + +Use Code Owners in combination with merge request +[approval rules](../merge_requests/approvals/rules.md) (either optional or required) +to build a flexible approval workflow: + +- Use **Code Owners** to ensure quality. Define the users who have domain expertise + for specific paths in your repository. +- Use **Approval rules** to define areas of expertise that don't correspond to specific + file paths in your repository. Approval rules help guide merge request creators to + the correct set of reviewers, such as frontend developers or a security team. + +For example: + +| Type | Name | Scope | Comment | +|------|------|--------|------------| +| Approval rule | UX | All files | A user experience (UX) team member reviews the user experience of all changes made in your project. +| Approval rule | Security | All files | A security team member reviews all changes for vulnerabilities. +| Code Owner approval rule | Frontend: Code Style | `*.css` files | A frontend engineer reviews CSS file changes for adherence to project style standards. +| Code Owner approval rule | Backend: Code Review | `*.rb` files | A backend engineer reviews the logic and code style of Ruby files. + +<div class="video-fallback"> + Video introduction: <a href="https://www.youtube.com/watch?v=RoyBySTUSB0">Code Owners</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/RoyBySTUSB0" frameborder="0" allowfullscreen> </iframe> +</figure> + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> + +## View Code Owners of a file or directory + +To view the Code Owners of a file or directory: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Files**. +1. Go to the file or directory you want to see the Code Owners for. +1. Optional. Select a branch or tag. + +GitLab shows the Code Owners at the top of the page. + +## Set up Code Owners + +1. Create a `CODEOWNERS` file in your [preferred location](#code-owners-file). +1. Define some rules in the file following the [Code Owners syntax reference](reference.md). + Some suggestions: + - Configure [All eligible approvers](../merge_requests/approvals/rules.md#code-owners-as-eligible-approvers) approval rule. + - [Require Code Owner approval](../protected_branches.md#require-code-owner-approval-on-a-protected-branch) on a protected branch. +1. Commit your changes, and push them up to GitLab. + +### Code Owners file + +A `CODEOWNERS` file (with no extension) specifies the users or +[shared groups](../members/share_project_with_groups.md) responsible for +specific files and directories in a repository. + +Each repository uses a single `CODEOWNERS` file. GitLab checks these locations +in your repository in this order. The first `CODEOWNERS` file found is used, and +all others are ignored: + +1. In the root directory: `./CODEOWNERS`. +1. In the `docs` directory: `./docs/CODEOWNERS`. +1. In the `.gitlab` directory: `./.gitlab/CODEOWNERS`. + +### Groups as Code Owners + +> Group and subgroup hierarchy support was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32432) in GitLab 13.0. + +You can use members of groups and subgroups as Code Owners for projects: + +```mermaid +graph TD + A[Parent group X] -->|owns| B[Project A] + A -->|contains| C[Subgroup Y] + C -->|owns| D[Project B] + A-. inherits ownership .-> D +``` + +In this example: + +- **Parent group X** (`group-x`) owns **Project A**. +- **Parent group X** also contains a subgroup, **Subgroup Y**. (`group-x/subgroup-y`) +- **Subgroup Y** owns **Project B**. + +The eligible Code Owners are: + +- **Project A**: the members of **Group X** only, because **Project A** doesn't belong to **Subgroup Y**. +- **Project B**: the members of both **Group X** and **Subgroup Y**. + +You can [invite](../members/share_project_with_groups.md) **Subgroup Y** to **Project A** +so that their members also become eligible Code Owners. + +```mermaid +graph LR + A[Parent group X] -->|owns| B[Project A] + A -->|also contains| C[Subgroup Y] + C -.->D{Invite Subgroup Y<br/>to Project A?} -.->|yes| F[Approvals can be<br/> required] -.-> B + D{Invite Subgroup Y<br/>to Project A?} -.->|no| I[Subgroup Y cannot be<br /> an approver] -.-> B + C -.->E{Add Subgroup Y<br/>as Code Owners<br/>to Project A?} -.->|yes| H[Approvals can only<br/>be optional] -.-> B +``` + +If you do not invite **Subgroup Y** to **Project A**, but make them Code Owners, their approval +of the merge request becomes optional. + +Inviting **Subgroup Y** to a parent group of **Project A** +[is not supported](https://gitlab.com/gitlab-org/gitlab/-/issues/288851). To set **Subgroup Y** as +Code Owners, add this group directly to the project itself. + +NOTE: +For approval to be required, groups as Code Owners must have a direct membership +(not inherited membership) in the project. Approval can only be optional for groups +that inherit membership. Members in the Code Owners group also must be direct members, +and not inherit membership from any parent groups. + +#### Add a group as a Code Owner + +To set a group as a Code Owner: + +In the `CODEOWNERS` file, enter text that follows one of these patterns: + +```plaintext +# All group members as Code Owners for a file +file.md @group-x + +# All subgroup members as Code Owners for a file +file.md @group-x/subgroup-y + +# All group and subgroup members as Code Owners for a file +file.md @group-x @group-x/subgroup-y +``` + +### Define more specific owners for more specifically defined files or directories + +When a file or directory matches multiple entries in the `CODEOWNERS` file, +the users from last pattern matching the file or directory are used. This enables you +to define more specific owners for more specifically defined files or directories, when +you order the entries in a sensible way. + +For example, in the following `CODEOWNERS` file: + +```plaintext +# This line would match the file terms.md +*.md @doc-team + +# This line would also match the file terms.md +terms.md @legal-team +``` + +The Code Owner for `terms.md` would be `@legal-team`. + +If you use sections, the last pattern matching the file or directory for each section is used. +For example, in a `CODEOWNERS` file using sections: + +```plaintext +[README Owners] +README.md @user1 @user2 +internal/README.md @user4 + +[README other owners] +README.md @user3 +``` + +The Code Owners for the `README.md` in the root directory are `@user1`, `@user2`, +and `@user3`. The Code Owners for `internal/README.md` are `@user4` and `@user3`. + +Only one CODEOWNERS pattern per section is matched to a file path. + +### Organize Code Owners by putting them into sections + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in GitLab 13.2 [with a flag](../../../administration/feature_flags.md) named `sectional_codeowners`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42389) in GitLab 13.4. Feature flag `sectional_codeowners` removed. + +You can organize Code Owners by putting them into named sections. + +You can use sections for shared directories, so that multiple +teams can be reviewers. + +To add a section to the `CODEOWNERS` file, enter a section name in brackets, +followed by the files or directories, and users, groups, or subgroups: + +```plaintext +[README Owners] +README.md @user1 @user2 +internal/README.md @user2 +``` + +Each Code Owner in the merge request widget is listed under a label. +The following image shows a **Groups** and **Documentation** section: + + + +#### Set default owner for a section + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371711) in GitLab 15.11 [with a flag](../../../administration/feature_flags.md) named `codeowners_default_owners`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115888) in GitLab 15.11. Feature flag `codeowners_default_owners` removed. + +If multiple file paths inside a section share the same ownership, define a default +Code Owner for the section. All paths in that section inherit this default, unless +you override the section default on a specific line. + +Default owners are applied when specific owners are not specified for file paths. +Specific owners defined beside the file path override default owners: + +```plaintext +[Documentation] @docs-team +docs/ +README.md + +[Database] @database-team +model/db/ +config/db/database-setup.md @docs-team +``` + +In this example: + +- `@docs-team` owns all items in the `Documentation` section. +- `@database-team` owns all items in the `Database` section except + `config/db/database-setup.md`, which has an override assigning it to `@docs-team`. + +To combine the syntax for default owners with [optional sections](#make-a-code-owners-section-optional) +and required approvals, place default owners at the end: + +```plaintext +[Documentation][2] @docs-team +docs/ +README.md + +^[Database] @database-team +model/db/ +config/db/database-setup.md @docs-team +``` + +#### Sections with duplicate names + +If multiple sections have the same name, they are combined. +Also, section headings are not case-sensitive. For example: + +```plaintext +[Documentation] +ee/docs/ @docs +docs/ @docs + +[Database] +README.md @database +model/db/ @database + +[DOCUMENTATION] +README.md @docs +``` + +This code results in three entries under the **Documentation** section header, and two +entries under **Database**. The entries defined under the sections **Documentation** and +**DOCUMENTATION** are combined, using the case of the first section. + +#### Make a Code Owners section optional + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/232995) in GitLab 13.8. + +You can designate optional sections in your Code Owners file. Prepend the +section name with the caret `^` character to treat the entire section as optional. +Optional sections enable you to designate responsible parties for various parts +of your codebase, but not require approval from them. This approach provides +a more relaxed policy for parts of your project that are frequently updated, +but don't require stringent reviews. + +In this example, the `[Go]` section is optional: + +```plaintext +[Documentation] +*.md @root + +[Ruby] +*.rb @root + +^[Go] +*.go @root +``` + +The optional Code Owners section displays in merge requests under the **Approval Rules** area: + + + +If a section is duplicated in the file, and one of them is marked as optional and the other isn't, the section is required. + +Optional sections in the `CODEOWNERS` file are treated as optional only +when changes are submitted by using merge requests. If a change is submitted directly +to the protected branch, approval from Code Owners is still required, even if the +section is marked as optional. + +### Require multiple approvals from Code Owners + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335451) in GitLab 15.9. + +You can require multiple approvals for the Code Owners sections under the Approval Rules area in merge requests. +Append the section name with a number `n` in brackets. This requires `n` approvals from the Code Owners in this section. +Please note valid entries for `n` are integers `≥ 1`. `[1]` is optional as it is the default. Invalid values for `n` are treated as `1`. + +WARNING: +[Issue #384881](https://gitlab.com/gitlab-org/gitlab/-/issues/385881) proposes changes +to the behavior of this setting. Do not intentionally set invalid values. They may +become valid in the future, and cause unexpected behavior. + +Please confirm you enabled `Require approval from code owners` in `Settings > Repository > Protected branches`, otherwise the Code Owner approvals will be optional. + +In this example, the `[Documentation]` section requires 2 approvals: + +```plaintext +[Documentation][2] +*.md @tech-writer-team + +[Ruby] +*.rb @dev-team +``` + +The `Documentation` Code Owners section under the **Approval Rules** area displays 2 approvals are required: + + + +### Allowed to Push + +The Code Owner approval and protected branch features do not apply to users who +are **Allowed to push**. + +## Technical Resources + +[Code Owners development guidelines](../../../development/code_owners/index.md) + +## Troubleshooting + +For more information about how the Code Owners feature handles errors, see the +[Code Owners reference](reference.md). + +### Approvals shown as optional + +A Code Owner approval rule is optional if any of these conditions are true: + +- The user or group are not a member of the project. Code Owners [cannot inherit from parent groups](https://gitlab.com/gitlab-org/gitlab/-/issues/288851/). +- [Code Owner approval on a protected branch](../protected_branches.md#require-code-owner-approval-on-a-protected-branch) has not been set up. +- The section is [marked as optional](#make-a-code-owners-section-optional). + +### Approvals do not show + +Code Owner approval rules only update when the merge request is created. +If you update the `CODEOWNERS` file, close the merge request and create a new one. + +### User not shown as possible approver + +A user might not show as an approver on the Code Owner merge request approval rules +if any of these conditions are true: + +- A rule prevents the specific user from approving the merge request. + Check the project [merge request approval](../merge_requests/approvals/settings.md#edit-merge-request-approval-settings) settings. +- A Code Owner group has a visibility of **private**, and the current user is not a + member of the Code Owner group. +- Current user is an external user who does not have permission to the internal Code Owner group. + +### Approval rule is invalid. GitLab has approved this rule automatically to unblock the merge request + +This message may appear if an approval rule uses a Code Owner that is not a direct member of the project. +Check that the group or user has been invited to the project. diff --git a/doc/user/project/codeowners/reference.md b/doc/user/project/codeowners/reference.md new file mode 100644 index 00000000000..dbb39fafabe --- /dev/null +++ b/doc/user/project/codeowners/reference.md @@ -0,0 +1,371 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Code Owners syntax and error handling **(PREMIUM)** + +This page describes the syntax and error handling used in Code Owners files, +and provides an example file. + +## Code Owners syntax + +### Comments + +Lines beginning with `#` are ignored: + +```plaintext +# This is a comment +``` + +### Sections + +Sections are groups of entries. A section begins with a section heading in square brackets, followed by the entries. + +```plaintext +[Section name] +/path/of/protected/file.rb @username +/path/of/protected/dir/ @group +``` + +#### Section headings + +Section headings must always have a name. They can also be made optional, or +require a number of approvals. A list of default owners can be added to the section heading line. + +```plaintext +# Required section +[Section name] + +# Optional section +^[Section name] + +# Section requiring 5 approvals +[Section name][5] + +# Section with @username as default owner +[Section name] @username + +# Section with @group and @subgroup as default owners and requiring 2 approvals +[Section name][2] @group @subgroup +``` + +#### Section names + +Sections names are defined between square brackets. Section names are not case-sensitive. +[Sections with duplicate names](index.md#sections-with-duplicate-names) are combined. + +```plaintext +[Section name] +``` + +#### Required sections + +Required sections do not include `^` before the [section name](#section-names). + +```plaintext +[Required section] +``` + +#### Optional sections + +Optional sections include a `^` before the [section name](#section-names). + +```plaintext +^[Optional section] +``` + +#### Sections requiring multiple approvals + +Sections requiring multiple approvals include the number of approvals in square brackets after the [section name](#section-names). + +```plaintext +[Section requiring 5 approvals][5] +``` + +NOTE: +Optional sections ignore the number of approvals required. + +#### Sections with default owners + +You can define a default owner for the entries in a section by appending the owners to the [section heading](#section-headings). + +```plaintext +# Section with @username as default owner +[Section name] @username + +# Section with @group and @subgroup as default owners and requiring 2 approvals +[Section name][2] @group @subgroup +``` + +### Code Owner entries + +Each Code Owner entry includes a path followed by one or more owners. + +```plaintext +README.md @username1 +``` + +NOTE: +If an entry is duplicated in a section, [the last entry is used from each section.](index.md#define-more-specific-owners-for-more-specifically-defined-files-or-directories) + +### Relative paths + +If a path does not start with a `/`, the path is treated as if it starts with +a [globstar](#globstar-paths). `README.md` is treated the same way as `/**/README.md`: + +```plaintext +# This will match /README.md, /internal/README.md, /app/lib/README.md +README.md @username + +# This will match /internal/README.md, /docs/internal/README.md, /docs/api/internal/README.md +internal/README.md +``` + +### Absolute paths + +If a path starts with a `/` it matches the root of the repository. + +```plaintext +# Matches only the file named `README.md` in the root of the repository. +/README.md + +# Matches only the file named `README.md` inside the `/docs` directory. +/docs/README.md +``` + +### Directory paths + +If a path ends with `/`, the path matches any file in the directory. + +```plaintext +# This is the same as `/docs/**/*` +/docs/ +``` + +### Wildcard paths + +Wildcards can be used to match one of more characters of a path. + +```plaintext +# Any markdown files in the docs directory +/docs/*.md @username + +# /docs/index file of any filetype +# For example: /docs/index.md, /docs/index.html, /docs/index.xml +/docs/index.* @username + +# Any file in the docs directory with 'spec' in the name. +# For example: /docs/qa_specs.rb, /docs/spec_helpers.rb, /docs/runtime.spec +/docs/*spec* @username + +# README.md files one level deep within the docs directory +# For example: /docs/api/README.md +/docs/*/README.md @username +``` + +### Globstar paths + +Globstars (`**`) can be used to match zero or more directories and subdirectories. + +```plaintext +# This will match /docs/index.md, /docs/api/index.md, /docs/api/graphql/index.md +/docs/**/index.md +``` + +### Entry owners + +Entries must be followed by one or more owner. These can be groups, subgroups, +and users. Order of owners is not important. + +```plaintext +/path/to/entry.rb @group +/path/to/entry.rb @group/subgroup +/path/to/entry.rb @user +/path/to/entry.rb @group @group/subgroup @user +``` + +#### Groups as entry owners + +Groups and subgroups can be owners of an entry. +Each entry can be owned by [one or more owners](#entry-owners). +For more details see the [Groups as Code Owners section](index.md#groups-as-code-owners). + +```plaintext +/path/to/entry.rb @group +/path/to/entry.rb @group/subgroup +/path/to/entry.rb @group @group/subgroup +``` + +### Users as entry owners + +Users can be owners of an entry. Each entry can be owned by +[one or more owners](#entry-owners). + +```plaintext +/path/to/entry.rb @username1 +/path/to/entry.rb @username1 @username2 +``` + +## Error handling in Code Owners + +### Entries with spaces + +Paths containing whitespace must be escaped with backslashes: `path\ with\ spaces/*.md`. +Without the backslashes, the path after the first whitespace is parsed as an owner. +GitLab the parses `folder with spaces/*.md @group` into +`path: "folder", owners: " with spaces/*.md @group"`. + +### Unparsable sections + +If a section heading cannot be parsed, the section is: + +1. Parsed as an entry. +1. Added to the previous section. +1. If no previous section exists, the section is added to the default section. + +For example, this file is missing a square closing bracket: + +```plaintext +* @group + +[Section name +docs/ @docs_group +``` + +GitLab recognizes the heading `[Section name` as an entry. The default section includes 3 rules: + +- Default section + - `*` owned by `@group` + - `[Section` owned by `name` + - `docs/` owned by `@docs_group` + +This file contains an unescaped space between the words `Section` and `name`. +GitLab recognizes the intended heading as an entry: + +```plaintext +[Docs] +docs/**/* @group + +[Section name]{2} @group +docs/ @docs_group +``` + +The `[Docs]` section then includes 3 rules: + +- `docs/**/*` owned by `@group` +- `[Section` owned by `name]{2} @group` +- `docs/` owned by `@docs_group` + +### Malformed owners + +Each entry must contain 1 or more owners to be valid, malformed owners are ignored. +For example `/path/* @group user_without_at_symbol @user_with_at_symbol` +is owned by `@group` and `@user_with_at_symbol`. + +### Inaccessible or incorrect owners + +Inaccessible or incorrect owners are ignored. For example, if `@group`, `@username`, +and `example@gitlab.com` are accessible on the project and we create an entry: + +```plaintext +* @group @grou @username @i_left @i_dont_exist example@gitlab.com invalid@gitlab.com +``` + +GitLab ignores `@grou`, `@i_left`, `@i_dont_exist`, and `invalid@gitlab.com`. + +For more information on who is accessible, see [Groups as Code Owners](index.md#groups-as-code-owners). + +### Zero owners + +If an entry includes no owners, or zero [accessible owners](#inaccessible-or-incorrect-owners) +exist, the entry is invalid. Because this rule can never be satisfied, GitLab +auto-approves it in merge requests. + +NOTE: +When a protected branch has `Require code owner approval` enabled, rules with +zero owners are still honored. + +### Less than 1 required approval + +When [defining the number of approvals](index.md#require-multiple-approvals-from-code-owners) for a section, +the minimum number of approvals is `1`. Setting the number of approvals to +`0` results in GitLab requiring one approval. + +## Example `CODEOWNERS` file + +```plaintext +# This is an example of a CODEOWNERS file. +# Lines that start with `#` are ignored. + +# app/ @commented-rule + +# Specify a default Code Owner by using a wildcard: +* @default-codeowner + +# Specify multiple Code Owners by using a tab or space: +* @multiple @code @owners + +# Rules defined later in the file take precedence over the rules +# defined before. +# For example, for all files with a filename ending in `.rb`: +*.rb @ruby-owner + +# Files with a `#` can still be accessed by escaping the pound sign: +\#file_with_pound.rb @owner-file-with-pound + +# Specify multiple Code Owners separated by spaces or tabs. +# In the following case the CODEOWNERS file from the root of the repo +# has 3 Code Owners (@multiple @code @owners): +CODEOWNERS @multiple @code @owners + +# You can use both usernames or email addresses to match +# users. Everything else is ignored. For example, this code +# specifies the `@legal` and a user with email `janedoe@gitlab.com` as the +# owner for the LICENSE file: +LICENSE @legal this_does_not_match janedoe@gitlab.com + +# Use group names to match groups, and nested groups to specify +# them as owners for a file: +README @group @group/with-nested/subgroup + +# End a path in a `/` to specify the Code Owners for every file +# nested in that directory, on any level: +/docs/ @all-docs + +# End a path in `/*` to specify Code Owners for every file in +# a directory, but not nested deeper. This code matches +# `docs/index.md` but not `docs/projects/index.md`: +/docs/* @root-docs + +# Include `/**` to specify Code Owners for all subdirectories +# in a directory. This rule matches `docs/projects/index.md` or +# `docs/development/index.md` +/docs/**/*.md @root-docs + +# This code makes matches a `lib` directory nested anywhere in the repository: +lib/ @lib-owner + +# This code match only a `config` directory in the root of the repository: +/config/ @config-owner + +# If the path contains spaces, escape them like this: +path\ with\ spaces/ @space-owner + +# Code Owners section: +[Documentation] +ee/docs @docs +docs @docs + +# Use of default owners for a section. In this case, all files (*) are owned by +the dev team except the README.md and data-models which are owned by other teams. +[Development] @dev-team +* +README.md @docs-team +data-models/ @data-science-team + +# This section is combined with the previously defined [Documentation] section: +[DOCUMENTATION] +README.md @docs +``` diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 5adc678e942..09a443614d0 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: howto, reference --- diff --git a/doc/user/project/deploy_keys/index.md b/doc/user/project/deploy_keys/index.md index 362d5826124..92fdc59dde3 100644 --- a/doc/user/project/deploy_keys/index.md +++ b/doc/user/project/deploy_keys/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -86,6 +86,7 @@ Prerequisites: 1. Complete the fields. 1. Optional. To grant `read-write` permission, select the **Grant write permissions to this key** checkbox. +1. Optional. Update the **Expiration date**. A project deploy key is enabled when it is created. You can modify only a project deploy key's name and permissions. diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index 23cfa7cc500..a81ccd043bd 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index eeaf5d799d2..7dee92f0c29 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -16,7 +16,7 @@ You might want to use these templates: - For different stages of your workflow, for example, feature proposal, feature improvement, or a bug report. - For every issue or merge request for a specific project, so the layout is consistent. -- For a [Service Desk email template](service_desk.md#new-service-desk-issues). +- For a [Service Desk email template](service_desk.md#use-a-custom-template-for-service-desk-issues). For description templates to work, they must be: @@ -156,7 +156,7 @@ To set a default description template for merge requests, either: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Merge requests**. - 1. In the **Merge commit message template** section, fill in **Default description template for merge requests**. + 1. In the **Default description template for merge requests** section, fill in the text area. 1. Select **Save changes**. To set a default description template for issues, either: @@ -167,9 +167,9 @@ To set a default description template for issues, either: - Users on GitLab Premium and higher: set the default template in project settings: 1. On the top bar, select **Main menu > Projects** and find your project. - 1. On the left sidebar, select **Settings**. - 1. Expand **Default issue template**. - 1. Fill in the **Default description template for issues** text area. + 1. On the left sidebar, select **Settings > General**. + 1. Expand **Default description template for issues**. + 1. Fill in the text area. 1. Select **Save changes**. Because GitLab merge request and issues support [Markdown](../markdown.md), you can use it to format diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index b83feda0c96..e4486938edf 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -58,7 +58,7 @@ git-lfs --version ``` If it doesn't recognize this command, you must install it. There are -several [installation methods](https://git-lfs.github.com/) that you can +several [installation methods](https://git-lfs.com/) that you can choose according to your OS. To install it with Homebrew: ```shell diff --git a/doc/user/project/import/bitbucket_server.md b/doc/user/project/import/bitbucket_server.md index 9937ea956c4..28ad1f22b8d 100644 --- a/doc/user/project/import/bitbucket_server.md +++ b/doc/user/project/import/bitbucket_server.md @@ -132,5 +132,4 @@ Consider memory constraints on the system before deciding to increase `SIDEKIQ_M ## Related topics -For information on automating user, group, and project import API calls, see -[Automate group and project import](index.md#automate-group-and-project-import). +- [Automate group and project import](index.md#automate-group-and-project-import) diff --git a/doc/user/project/import/cvs.md b/doc/user/project/import/cvs.md index 00aebb75a50..99221daf750 100644 --- a/doc/user/project/import/cvs.md +++ b/doc/user/project/import/cvs.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Migrating from CVS **(FREE)** [CVS](https://savannah.nongnu.org/projects/cvs) is an old centralized version -control system similar to [SVN](svn.md). +control system similar to [SVN](https://subversion.apache.org/). ## CVS vs Git diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index a1d94d81e69..657343261d9 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -7,7 +7,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Import your project from GitHub to GitLab **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381902) in GitLab 15.8, GitLab no longer automatically creates namespaces or groups that don't exist. GitLab also no longer falls back to using the user's personal namespace if the namespace or group name is taken. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381902) in GitLab 15.8, GitLab no longer automatically creates namespaces or groups that don't exist. GitLab also no longer falls back to using the user's personal namespace if the namespace or group name is taken. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388716) in GitLab 15.10, you no longer need to add any users to the parent group in GitLab to successfully import the **Require a pull request before merging - Allow specified actors to bypass required pull requests** branch protection rule. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378267) in GitLab 15.11, GitLab instances behind proxies no longer require `github.com` and `api.github.com` entries in the [allowlist for local requests](../../../security/webhooks.md#allow-outbound-requests-to-certain-ip-addresses-and-domains). You can import your GitHub projects from either GitHub.com or GitHub Enterprise. Importing projects does not migrate or import any types of groups or organizations from GitHub to GitLab. @@ -52,17 +54,16 @@ To import projects from GitHub: are properly mapped to the same user in GitLab. GitHub Enterprise does not require this field to be populated so you may have to add it on existing accounts. -See also [Branch protection rules and project settings](#branch-protection-rules-and-project-settings) for additional -prerequisites for those imports. +Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/383047), if you are using GitHub as an OmniAuth provider, ensure that the URL +perimeter is specified in the [OmniAuth configuration](../../../integration/github.md#enable-github-oauth-in-gitlab). ### Importing from GitHub Enterprise to self-managed GitLab If you are importing from GitHub Enterprise to a self-managed GitLab instance: - You must first enable the [GitHub integration](../../../integration/github.md). -- If GitLab is behind an HTTP/HTTPS proxy, you must populate the [allowlist for local requests](../../../security/webhooks.md#allow-outbound-requests-to-certain-ip-addresses-and-domains) - with `github.com` and `api.github.com` to solve the hostname. For more information, read the issue - [Importing a GitHub project requires DNS resolution even when behind a proxy](https://gitlab.com/gitlab-org/gitlab/-/issues/37941). +- For GitLab 15.10 and earlier, you must add `github.com` and `api.github.com` entries in the + [allowlist for local requests](../../../security/webhooks.md#allow-outbound-requests-to-certain-ip-addresses-and-domains). ### Importing from GitHub.com to self-managed GitLab @@ -80,10 +81,9 @@ Before you begin, ensure that any GitHub user you want to map to a GitLab user h [publicly visible email address](https://docs.github.com/en/rest/users#get-a-user) on GitHub. If you are importing to GitLab.com, you can alternatively import GitHub repositories using a [personal access token](#use-a-github-token). -We do not recommend this method, as it does not associate all user activity (such as issues and pull requests) with matching GitLab users. -User-matching attempts occur in that order, and if a user is not identified either way, the activity is associated with -the user account that is performing the import. +If a GitHub user's public email address doesn't match any GitLab user email address, the user's activity is associated with the user account that is +performing the import. NOTE: If you are using a self-managed GitLab instance or if you are importing from GitHub Enterprise, this process requires that you have configured @@ -101,9 +101,7 @@ Prerequisite: - Authentication token with administrator access. -Using a personal access token to import projects is not recommended. If you are a GitLab.com user, -you can use a personal access token to import your project from GitHub, but this method cannot -associate all user activity (such as issues and pull requests) with matching GitLab users. +If you are a GitLab.com user, you can use a personal access token to import your project from GitHub. If you are an administrator of a self-managed GitLab instance or if you are importing from GitHub Enterprise, you cannot use a personal access token. The [GitHub integration method (above)](#use-the-github-integration) is recommended for all users. @@ -255,10 +253,7 @@ When they are imported, supported GitHub branch protection rules are mapped to e | **Require signed commits** for the project's default branch | **Reject unsigned commits** GitLab [push rule](../repository/push_rules.md#prevent-unintended-consequences) **(PREMIUM)** | [GitLab 15.5](https://gitlab.com/gitlab-org/gitlab/-/issues/370949) | | **Allow force pushes - Everyone** | **Allowed to force push** [branch protection setting](../protected_branches.md#allow-force-push-on-a-protected-branch) | [GitLab 15.6](https://gitlab.com/gitlab-org/gitlab/-/issues/370943) | | **Require a pull request before merging - Require review from Code Owners** | **Require approval from code owners** [branch protection setting](../protected_branches.md#require-code-owner-approval-on-a-protected-branch) **(PREMIUM)** | [GitLab 15.6](https://gitlab.com/gitlab-org/gitlab/-/issues/376683) | -| **Require a pull request before merging - Allow specified actors to bypass required pull requests** (1) | List of users in the **Allowed to push** list of [branch protection settings](../protected_branches.md#configure-a-protected-branch) **(PREMIUM)**. Without a Premium license, the list of users that are allowed to push is limited to roles. | [GitLab 15.8](https://gitlab.com/gitlab-org/gitlab/-/issues/384939) | - -1. To successfully import the **Require a pull request before merging - Allow specified actors to bypass required pull requests** rule, you must add to the parent group in GitLab - the users that are allowed to bypass required pull requests before you begin importing. +| **Require a pull request before merging - Allow specified actors to bypass required pull requests** | List of users in the **Allowed to push** list of [branch protection settings](../protected_branches.md#configure-a-protected-branch) **(PREMIUM)**. Without a **Premium** subscription, the list of users that are allowed to push is limited to roles. | [GitLab 15.8](https://gitlab.com/gitlab-org/gitlab/-/issues/384939) | Mapping GitHub rule **Require status checks to pass before merging** to [external status checks](../merge_requests/status_checks.md) was considered in issue @@ -282,7 +277,7 @@ These GitHub collaborator roles are mapped to these GitLab [member roles](../../ GitHub Enterprise Cloud has [custom repository roles](https://docs.github.com/en/enterprise-cloud@latest/organizations/managing-peoples-access-to-your-organization-with-roles/about-custom-repository-roles). -These roles aren't supported and cause partial imports. +These roles aren't supported and cause partially completed imports. To import GitHub collaborators, you must have at least the Write role on the GitHub project. Otherwise collaborators import is skipped. diff --git a/doc/user/project/import/gitlab_com.md b/doc/user/project/import/gitlab_com.md index c00709d9697..9cb3ff75d5d 100644 --- a/doc/user/project/import/gitlab_com.md +++ b/doc/user/project/import/gitlab_com.md @@ -33,7 +33,5 @@ When the importer is done, a new GitLab project is created with your imported da ## Related topics -- To automate user, group, and project import API calls, see - [Automate group and project import](index.md#automate-group-and-project-import). -- To import Wiki and merge request data to your new instance, - see [exporting a project](../settings/import_export.md#export-a-project-and-its-data). +- [Automate group and project import](index.md#automate-group-and-project-import) +- [Export a project](../settings/import_export.md#export-a-project-and-its-data) diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index 8c95e68e1f2..ebfe421fd02 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -24,7 +24,7 @@ For any type of source and target, you can migrate GitLab projects: - When [migrating groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended), which allows you to migrate all projects in a group simultaneously. Migrating projects by direct transfer is in - [Beta](../../../policy/alpha-beta-support.md#beta-features). The feature is not ready for production use. + [Beta](../../../policy/alpha-beta-support.md#beta). The feature is not ready for production use. - Using [file exports](../settings/import_export.md). With this method you can migrate projects one by one. No network connection between instances is required. @@ -32,13 +32,27 @@ If you only need to migrate Git repositories, you can [import each project by UR import issues and merge requests this way. To retain metadata like issues and merge requests, either: - [Migrate projects with groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). - This feature is in [Beta](../../../policy/alpha-beta-support.md#beta-features). It is not ready for production use. + This feature is in [Beta](../../../policy/alpha-beta-support.md#beta). It is not ready for production use. - Use [file exports](../settings/import_export.md) to import projects. Keep in mind the limitations of [migrating using file exports](../settings/import_export.md#items-that-are-exported). When migrating from self-managed to GitLab.com, user associations (such as comment author) are changed to the user who is importing the projects. +## Security + +Only import projects from sources you trust. If you import a project from an untrusted source, +an attacker could steal your sensitive data. For example, an imported project +with a malicious `.gitlab-ci.yml` file could allow an attacker to exfiltrate group CI/CD variables. + +GitLab self-managed administrators can reduce their attack surface by disabling import sources they don't need: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > General**. +1. Expand **Visibility and access controls**. +1. Scroll to **Import sources**. +1. Clear checkboxes for importers that are not required. + ## Available project importers You can import projects from: @@ -65,7 +79,7 @@ You can then [connect your external repository to get CI/CD benefits](../../../c GitLab can not automatically migrate Subversion repositories to Git. Converting Subversion repositories to Git can be difficult, but several tools exist including: -- [`git svn`](https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git), for very small and simple repositories. +- [`git svn`](https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git), for very small and basic repositories. - [`reposurgeon`](http://www.catb.org/~esr/reposurgeon/repository-editing.html), for larger and more complex repositories. ## Migrate using the API @@ -82,10 +96,9 @@ over a series of Docker pulls and pushes. Re-run any CI pipelines to retrieve an ## Migrate between two self-managed GitLab instances -To migrate from an existing self-managed GitLab instance to a new self-managed GitLab instance, it's -best to [back up](../../../raketasks/backup_restore.md) -the existing instance and restore it on the new instance. For example, this is useful when migrating -a self-managed instance from an old server to a new server. +To migrate from an existing self-managed GitLab instance to a new self-managed GitLab instance, +you should [back up](../../../raketasks/backup_restore.md) +the existing instance and restore it on the new instance. For example, you could use this method to migrate a self-managed instance from an old server to a new server. The backups produced don't depend on the operating system running GitLab. You can therefore use the restore method to switch between different operating system distributions or versions, as long @@ -154,8 +167,20 @@ GitLab from: For more information, see: +- Information on paid GitLab [migration services](https://about.gitlab.com/services/migration/). - [Quick Start](https://gitlab.com/gitlab-org/professional-services-automation/tools/migration/congregate/-/blob/master/docs/using-congregate.md#quick-start). - [Frequently Asked Migration Questions](https://gitlab.com/gitlab-org/professional-services-automation/tools/migration/congregate/-/blob/master/customer/famq.md), including settings that need checking afterwards and other limitations. For support, customers must enter into a paid engagement with GitLab Professional Services. + +## Troubleshooting + +### Imported repository is missing branches + +If an imported repository does not contain all branches of the source repository: + +1. Set the [environment variable](../../../administration/logs/index.md#override-default-log-level) `IMPORT_DEBUG=true`. +1. Retry the import with a [different group, subgroup, or project name](https://about.gitlab.com/releases/2023/02/22/gitlab-15-9-released/#re-import-projects-from-external-providers). +1. If some branches are still missing, inspect [`importer.log`](../../../administration/logs/index.md#importerlog) + (for example, with [`jq`](../../../administration/logs/log_parsing.md#parsing-gitlab-railsimporterlog)). diff --git a/doc/user/project/import/phabricator.md b/doc/user/project/import/phabricator.md index 8a2dbba1343..d8784db5e29 100644 --- a/doc/user/project/import/phabricator.md +++ b/doc/user/project/import/phabricator.md @@ -15,7 +15,7 @@ and will be removed in GitLab 16.0. WARNING: The Phabricator task importer is in -[beta](../../../policy/alpha-beta-support.md#beta-features) and is +[Beta](../../../policy/alpha-beta-support.md#beta) and is [**not** complete](https://gitlab.com/gitlab-org/gitlab/-/issues/284406). It imports only an issue's title and description. The GitLab project created during the import process contains only issues, and the associated repository is disabled. diff --git a/doc/user/project/import/svn.md b/doc/user/project/import/svn.md deleted file mode 100644 index c9abc0f459d..00000000000 --- a/doc/user/project/import/svn.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'index.md#import-from-subversion' -remove_date: '2023-03-15' ---- - -This document was moved to [another location](index.md). - -<!-- This redirect file can be deleted after <2023-03-15>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/index.md b/doc/user/project/index.md index da0546b85e6..1512039fb25 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -179,8 +179,6 @@ Your project's visibility is set to **Private** by default. To change project vi ## Related topics -- For a list of words that you cannot use as project names, see - [reserved project and group names](../../user/reserved_names.md). -- For a list of characters that you cannot use in project and group names, see - [limitations on project and group names](../../user/reserved_names.md#limitations-on-project-and-group-names). -- [Manage projects](working_with_projects.md). +- [Reserved project and group names](../../user/reserved_names.md) +- [Limitations on project and group names](../../user/reserved_names.md#limitations-on-project-and-group-names) +- [Manage projects](working_with_projects.md) diff --git a/doc/user/project/integrations/bamboo.md b/doc/user/project/integrations/bamboo.md index 12039a70d0e..419849ae5a0 100644 --- a/doc/user/project/integrations/bamboo.md +++ b/doc/user/project/integrations/bamboo.md @@ -125,7 +125,7 @@ For example: ### Builds not triggered If builds are not triggered, ensure you entered the right GitLab IP address in -Bamboo under **Trigger IP addresses**. Also check [service hook logs](index.md#troubleshooting-integrations) for request failures. +Bamboo under **Trigger IP addresses**. Also check [integration webhook logs](index.md#troubleshooting) for request failures. ### Advanced Atlassian Bamboo features not available in GitLab UI diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index 2933203c593..aeb8871d257 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -57,4 +57,4 @@ internal issue tracker, the internal issue is linked. ## Troubleshooting -To see recent service hook deliveries, check [service hook logs](index.md#troubleshooting-integrations). +For recent integration webhook deliveries, check [integration webhook logs](index.md#troubleshooting). diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index 649b0c51818..e23896347ff 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -18,6 +18,10 @@ integrations on GitLab.com. ## Installation +In GitLab 15.0 and later, the GitLab for Slack app uses +[granular permissions](https://medium.com/slack-developer-blog/more-precision-less-restrictions-a3550006f9c3). +Although functionality has not changed, you should [reinstall the app](#update-the-gitlab-for-slack-app). + ### Through the Slack App Directory To enable the GitLab for Slack app for your workspace, @@ -43,8 +47,8 @@ To enable the GitLab integration for your Slack workspace: 1. Select **Install GitLab for Slack app**. 1. Select **Allow** on Slack's confirmation screen. -You can also select **Reinstall GitLab for Slack app** to update the app in your Slack workspace -to the latest version. See [Version history](#version-history) for details. +To update the app in your Slack workspace to the latest version, +you can also select **Reinstall GitLab for Slack app**. ## Slash commands @@ -63,6 +67,7 @@ Replace `<project>` with the project full path, or create a shorter [project ali | `/gitlab <project> issue comment <id> <shift+return> <comment>` | Adds a new comment with the comment body `<comment>` to the issue with the ID `<id>`. | | `/gitlab <project> deploy <from> to <to>` | [Deploys](#the-deploy-slash-command) from the `<from>` environment to the `<to>` environment. | | `/gitlab <project> run <job name> <arguments>` | Executes the [ChatOps](../../../ci/chatops/index.md) job `<job name>` on the default branch. | +| `/gitlab incident declare` | **Beta** Opens modal to [create a new incident](../../../operations/incident_management/slack.md). | ### The deploy slash command @@ -191,8 +196,3 @@ If you're not receiving notifications to a Slack channel, ensure: ### The App Home does not display properly If the [App Home](https://api.slack.com/start/overview#app_home) does not display properly, ensure your [app is up to date](#update-the-gitlab-for-slack-app). - -## Version history - -In GitLab 15.0 and later, the GitLab for Slack app is updated to [Slack's new granular permissions model](https://medium.com/slack-developer-blog/more-precision-less-restrictions-a3550006f9c3). -Although there is no change in functionality, you should [reinstall the app](#update-the-gitlab-for-slack-app). diff --git a/doc/user/project/integrations/google_play.md b/doc/user/project/integrations/google_play.md index 553e82be382..e83602186f3 100644 --- a/doc/user/project/integrations/google_play.md +++ b/doc/user/project/integrations/google_play.md @@ -6,10 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Google Play **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111621) in GitLab 15.10 [with a flag](../../../administration/feature_flags.md) named `google_play_integration`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `google_play_integration`. On GitLab.com, this feature is not available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111621) in GitLab 15.10 [with a flag](../../../administration/feature_flags.md) named `google_play_integration`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/389611) in GitLab 15.11. Feature flag `google_play_integration` removed. With the Google Play integration, you can configure your CI/CD pipelines to connect to the [Google Play Console](https://play.google.com/console) to build and release apps for Android devices. @@ -27,11 +25,12 @@ To enable the Google Play integration in GitLab: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Google Play**. -1. In **Enable Integration**, select the **Active** checkbox. +1. In **Enable integration**, select the **Active** checkbox. +1. In **Package name**, enter the package name of the app (for example, `com.gitlab.app_name`). 1. In **Service account key (.JSON)**, drag or upload your key file. 1. Select **Save changes**. -After you enable the integration, the global variable `$SUPPLY_JSON_KEY_DATA` is created for CI/CD use. +After you enable the integration, the global variables `$SUPPLY_PACKAGE_NAME` and `$SUPPLY_JSON_KEY_DATA` are created for CI/CD use. ### CI/CD variable security diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index 9ff6ad2a237..e5d70dc5790 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -103,9 +103,16 @@ supported by `push_hooks` and `tag_push_hooks` events aren't executed. You can change the number of supported branches or tags by changing the [`push_event_hooks_limit` application setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). -## Troubleshooting integrations +## Contribute to integrations + +If you're interested in developing a new native integration for GitLab, see: + +- [Integrations development guidelines](../../../development/integrations/index.md) +- [GitLab Developer Portal](https://developer.gitlab.com) + +## Troubleshooting -Some integrations use hooks to integrate with external applications. To confirm which ones use integration hooks, see the [available integrations](#available-integrations). For more information, see [Troubleshooting webhooks](webhooks.md#troubleshoot-webhooks). +Some integrations use hooks to integrate with external applications. To confirm which ones use integration hooks, see the [available integrations](#available-integrations). For more information, see [webhook troubleshooting](webhooks.md#troubleshooting). ### `Test Failed. Save Anyway` error @@ -115,7 +122,3 @@ push data to build the test payload, and there are no push events in the project To resolve this error, initialize the repository by pushing a test file to the project and set up the integration again. - -## Contribute to integrations - -To add a new integration, see the [Integrations development guide](../../../development/integrations/index.md). diff --git a/doc/user/project/integrations/microsoft_teams.md b/doc/user/project/integrations/microsoft_teams.md index fc5d4d3cc80..fc448382cb4 100644 --- a/doc/user/project/integrations/microsoft_teams.md +++ b/doc/user/project/integrations/microsoft_teams.md @@ -58,4 +58,4 @@ GitLab to send the notifications: ## Related topics -- [Setting up an incoming webhook on Microsoft Teams](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/connectors-using#setting-up-a-custom-incoming-webhook). +- [Setting up an incoming webhook on Microsoft Teams](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/connectors-using#setting-up-a-custom-incoming-webhook) diff --git a/doc/user/project/integrations/mlflow_client.md b/doc/user/project/integrations/mlflow_client.md index bd14021ab1c..c5931928626 100644 --- a/doc/user/project/integrations/mlflow_client.md +++ b/doc/user/project/integrations/mlflow_client.md @@ -4,33 +4,32 @@ group: Incubation info: Machine Learning Experiment Tracking is a GitLab Incubation Engineering program. No technical writer assigned to this group. --- -# MLFlow Client Integration **(FREE)** +# MLFlow client integration **(FREE)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8560) in GitLab 15.6 as an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) release [with a flag](../../../administration/feature_flags.md) named `ml_experiment_tracking`. Disabled by default. +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8560) in GitLab 15.11 as an [Experiment](../../../policy/alpha-beta-support.md#experiment) release [with a flag](../../../administration/feature_flags.md) named `ml_experiment_tracking`. Disabled by default. -DISCLAIMER: -MLFlow Client Integration is an experimental feature being developed by the Incubation Engineering Department, -and will receive significant changes over time. +NOTE: +Model experiment tracking is an [experimental feature](../../../policy/alpha-beta-support.md). +Refer to <https://gitlab.com/gitlab-org/gitlab/-/issues/381660> for feedback and feature requests. [MLFlow](https://mlflow.org/) is one of the most popular open source tools for Machine Learning Experiment Tracking. -GitLabs works as a backend to the MLFlow Client, [logging experiments](../ml/experiment_tracking/index.md). +GitLab works as a backend to the MLFlow Client, [logging experiments](../ml/experiment_tracking/index.md). Setting up your integrations requires minimal changes to existing code. GitLab plays the role of proxy server, both for artifact storage and tracking data. It reflects the MLFlow [Scenario 5](https://www.mlflow.org/docs/latest/tracking.html#scenario-5-mlflow-tracking-server-enabled-with-proxied-artifact-storage-access). -## Enable MLFlow Client Integration - -Complete this task to enable MLFlow Client Integration. +## Enable MLFlow client integration Prerequisites: - A [personal access token](../../../user/profile/personal_access_tokens.md) for the project, with minimum access level of `api`. - The project ID. To find the project ID, on the top bar, select **Main menu > Projects** and find your project. On the left sidebar, select **Settings > General**. -1. Set the tracking URI and token environment variables on the host that runs the code (your local environment, CI pipeline, or remote host). +To enable MLFlow client integration: - For example: +1. Set the tracking URI and token environment variables on the host that runs the code. + This can be your local environment, CI pipeline, or remote host. For example: ```shell export MLFLOW_TRACKING_URI="http://<your gitlab endpoint>/api/v4/projects/<your project id>/ml/mlflow" @@ -39,43 +38,48 @@ Prerequisites: 1. If your training code contains the call to `mlflow.set_tracking_uri()`, remove it. -When running the training code, MLFlow will create experiments, runs, log parameters, metrics, +When running the training code, MLFlow creates experiments, runs, log parameters, metrics, metadata and artifacts on GitLab. -After experiments are logged, they are listed under `/<your project>/-/ml/experiments`. Runs are registered as Model Candidates, -that can be explored by selecting an experiment. +After experiments are logged, they are listed under `/<your project>/-/ml/experiments`. +Runs are registered as: -## Limitations +- Model Candidates, which can be explored by selecting an experiment. +- Tags, which are registered as metadata. -- The API GitLab supports is the one defined at MLFlow version 1.28.0. -- API endpoints not listed above are not supported. -- During creation of experiments and runs, tags are ExperimentTags and RunTags are stored, even though they are not displayed. -- MLFlow Model Registry is not supported. +## Supported MlFlow client methods and caveats -## Supported methods and caveats - -This is a list of methods we support from the MLFlow client. Other methods might be supported but were not +GitLab supports these methods from the MLFlow client. Other methods might be supported but were not tested. More information can be found in the [MLFlow Documentation](https://www.mlflow.org/docs/1.28.0/python_api/mlflow.html). -### `set_experiment()` - -Accepts both `experiment_name` and `experiment_id` - -### `start_run()` - -- Nested runs have not been tested. -- `run_name` is not supported +| Method | Supported | Version Added | Comments | +|--------------------------|------------------|----------------|----------| +| `get_experiment` | Yes | 15.11 | | +| `get_experiment_by_name` | Yes | 15.11 | | +| `set_experiment` | Yes | 15.11 | | +| `get_run` | Yes | 15.11 | | +| `start_run` | Yes | 15.11 | | +| `log_artifact` | Yes with caveat | 15.11 | (15.11) `artifact_path` must be empty string. Does not support directories. +| `log_artifacts` | Yes with caveat | 15.11 | (15.11) `artifact_path` must be empty string. Does not support directories. +| `log_batch` | Yes | 15.11 | | +| `log_metric` | Yes | 15.11 | | +| `log_metrics` | Yes | 15.11 | | +| `log_param` | Yes | 15.11 | | +| `log_params` | Yes | 15.11 | | +| `log_figure` | Yes | 15.11 | | +| `log_image` | Yes | 15.11 | | +| `log_text` | Yes with caveat | 15.11 | (15.11) Does not support directories. +| `log_dict` | Yes with caveat | 15.11 | (15.11) Does not support directories. +| `set_tag` | Yes | 15.11 | | +| `set_tags` | Yes | 15.11 | | +| `set_terminated` | Yes | 15.11 | | +| `end_run` | Yes | 15.11 | | +| `update_run` | Yes | 15.11 | | +| `log_model` | Partial | 15.11 | (15.11) Saves the artifacts, but not the model data. `artifact_path` must be empty. -### `log_param()`, `log_params()`, `log_metric()`, `log_metrics()` - -Work as defined by the documentation - -### `log_artifact()`, `log_artifacts()` - -`artifact_path` must be empty string. - -### `log_model()` +## Limitations -This is an experimental method in MLFlow, and partial support is offered. It stores the model artifacts, but does -not log the model information. The `artifact_path` parameter must be set to `''`, because Generic Packages do not support folder -structure. +- The API GitLab supports is the one defined at MLFlow version 1.28.0. +- API endpoints not listed above are not supported. +- During creation of experiments and runs, ExperimentTags are stored, even though they are not displayed. +- MLFlow Model Registry is not supported. diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md index 09e7189d4f5..6806b724ac2 100644 --- a/doc/user/project/integrations/slack.md +++ b/doc/user/project/integrations/slack.md @@ -3,14 +3,16 @@ stage: Manage group: Integrations info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- +<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> -# Slack notifications **(FREE)** +# Slack notifications (deprecated) **(FREE)** WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/372411) on GitLab.com -in GitLab 15.9 and is [planned for removal](https://gitlab.com/groups/gitlab-org/-/epics/8673). -For GitLab.com, use the [GitLab for Slack app](gitlab_slack_application.md) instead. -For self-managed GitLab instances, you can continue to use this feature. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/372411) in GitLab 15.9 +and is planned for removal in 17.0. Use the [GitLab for Slack app](gitlab_slack_application.md) instead. +This change is a breaking change. +For the planned support of the GitLab for Slack app for self-managed instances, +see [epic 1211](https://gitlab.com/groups/gitlab-org/-/epics/1211). The Slack notifications integration enables your GitLab project to send events (such as issue creation) to your existing Slack team as notifications. Setting up @@ -150,3 +152,5 @@ p.each do |project| project.slack_integration.update!(:active, false) end ``` + +<!--- end_remove --> diff --git a/doc/user/project/integrations/squash_tm.md b/doc/user/project/integrations/squash_tm.md index 0f63b4a48db..9625387ad1e 100644 --- a/doc/user/project/integrations/squash_tm.md +++ b/doc/user/project/integrations/squash_tm.md @@ -4,7 +4,7 @@ group: Integrations info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Squash TM integration **(FREE SELF)** +# Squash TM **(FREE SELF)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/337855) in GitLab 15.10. diff --git a/doc/user/project/integrations/webhook_events.md b/doc/user/project/integrations/webhook_events.md index 53177004888..f0213e9bb48 100644 --- a/doc/user/project/integrations/webhook_events.md +++ b/doc/user/project/integrations/webhook_events.md @@ -861,6 +861,7 @@ Payload example: "visibility_level":20, "path_with_namespace":"gitlabhq/gitlab-test", "default_branch":"master", + "ci_config_path":"", "homepage":"http://example.com/gitlabhq/gitlab-test", "url":"http://example.com/gitlabhq/gitlab-test.git", "ssh_url":"git@example.com:gitlabhq/gitlab-test.git", @@ -885,7 +886,10 @@ Payload example: "title": "MS-Viewport", "created_at": "2013-12-03T17:23:34Z", "updated_at": "2013-12-03T17:23:34Z", + "last_edited_at": "2013-12-03T17:23:34Z", + "last_edited_by_id": 1, "milestone_id": null, + "state_id": 1, "state": "opened", "blocking_discussions_resolved": true, "work_in_progress": false, @@ -893,6 +897,11 @@ Payload example: "merge_status": "unchecked", "target_project_id": 14, "description": "", + "total_time_spent": 1800, + "time_change": 30, + "human_total_time_spent": "30m", + "human_time_change": "30s", + "human_time_estimate": "30m", "url": "http://example.com/diaspora/merge_requests/1", "source": { "name":"Awesome Project", @@ -929,6 +938,7 @@ Payload example: "last_commit": { "id": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7", "message": "fixed readme", + "title": "Update file README.md", "timestamp": "2012-01-03T23:36:29+02:00", "url": "http://example.com/awesome_space/awesome_project/commits/da1560886d4f094c3e6c9ef40349f7d38b5d27d7", "author": { @@ -997,7 +1007,15 @@ Payload example: "type": "ProjectLabel", "group_id": 41 }] - } + }, + "last_edited_at": { + "previous": null, + "current": "2023-03-15 00:00:10 UTC" + }, + "last_edited_by_id": { + "previous": null, + "current": 3278533 + }, }, "assignees": [ { @@ -1074,7 +1092,8 @@ Payload example: "message": "adding an awesome page to the wiki", "slug": "awesome", "url": "http://example.com/root/awesome-project/-/wikis/awesome", - "action": "create" + "action": "create", + "diff_url": "http://example.com/root/awesome-project/-/wikis/home/diff?version_id=78ee4a6705abfbff4f4132c6646dbaae9c8fb6ec" } } ``` @@ -1392,6 +1411,7 @@ Payload example: "build_started_at": null, "build_finished_at": null, "build_duration": null, + "build_queued_duration": 1095.588715, // duration in seconds "build_allow_failure": false, "build_failure_reason": "script_failure", "retries_count": 2, // the second retry of this job diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 1511e37e31e..1c65ab78ee0 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -99,7 +99,7 @@ You can define URL variables directly using the REST API. ## Configure your webhook receiver endpoint Webhook receiver endpoints should be fast and stable. -Slow and unstable receivers can be [disabled automatically](#failing-webhooks) to ensure system reliability. Webhooks that fail can lead to retries, [which cause duplicate events](#webhook-fails-or-multiple-webhook-requests-are-triggered). +Slow and unstable receivers can be [disabled automatically](#failing-webhooks) to ensure system reliability. Webhooks that fail might lead to [duplicate events](#webhook-fails-or-multiple-webhook-requests-are-triggered). Endpoints should follow these best practices: @@ -129,18 +129,17 @@ FLAG: On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `auto_disabling_web_hooks`. On GitLab.com, this feature is available. -If a webhook fails repeatedly, it may be disabled automatically. +Project or group webhooks that fail four consecutive times are automatically disabled. -Webhooks that return response codes in the `5xx` range are understood to be failing +Project or group webhooks that return response codes in the `5xx` range are understood to be failing intermittently and are temporarily disabled. These webhooks are initially disabled -for 1 minute, which is extended on each retry up to a maximum of 24 hours. +for one minute, which is extended on each subsequent failure up to a maximum of 24 hours. -Webhooks that return response codes in the `4xx` range are understood to be +Project or group webhooks that return response codes in the `4xx` range are understood to be misconfigured and are permanently disabled until you manually re-enable them yourself. -See [Troubleshooting](#troubleshoot-webhooks) for more information on -disabled webhooks and how to re-enable them. +For more information about disabled webhooks, see [troubleshooting](#troubleshooting). ## Test a webhook @@ -151,7 +150,7 @@ For example, to test `push events`, your project should have at least one commit NOTE: Testing is not supported for some types of events for project and groups webhooks. -Read more in [issue 379201](https://gitlab.com/gitlab-org/gitlab/-/issues/379201). +For more information, see [issue 379201](https://gitlab.com/gitlab-org/gitlab/-/issues/379201). Prerequisites: @@ -259,7 +258,7 @@ Image URLs are not rewritten if: ## Events -For more information about supported events for Webhooks, go to [Webhook events](webhook_events.md). +For more information about supported events for webhooks, see [webhook events](webhook_events.md). ## Delivery headers @@ -275,7 +274,7 @@ Webhook requests to your endpoint include the following headers: | `X-Gitlab-Event` | Name of the webhook type. Corresponds to [event types](webhook_events.md) but in the format `"<EVENT> Hook"`. | `"Push Hook"` | | `X-Gitlab-Event-UUID` | Unique ID per webhook that is not recursive. A hook is recursive if triggered by an earlier webhook that hit the GitLab instance. Recursive webhooks have the same value for this header. | `"13792a34-cac6-4fda-95a8-c58e00a3954e"` | -## Troubleshoot webhooks +## Troubleshooting > **Recent events** for group webhooks [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325642) in GitLab 15.3. @@ -293,8 +292,9 @@ To view the table: 1. Scroll down to the webhooks. 1. Each [failing webhook](#failing-webhooks) has a badge listing it as: - - **Failed to connect** if it is misconfigured, and needs manual intervention to re-enable it. - - **Fails to connect** if it is temporarily disabled and is retrying later. + - **Failed to connect** if it's misconfigured and must be manually re-enabled. + - **Fails to connect** if it's temporarily disabled and is automatically + re-enabled after the timeout limit has elapsed.  @@ -330,8 +330,7 @@ Missing intermediate certificates are common causes of verification failure. ### Webhook fails or multiple webhook requests are triggered -If you are receiving multiple webhook requests, the webhook might have timed out and -been retried. +If you're receiving multiple webhook requests, the webhook might have timed out. GitLab expects a response in [10 seconds](../../../user/gitlab_com/index.md#other-limits). On self-managed GitLab instances, you can [change the webhook timeout limit](../../../administration/instance_limits.md#webhook-timeout). diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 98017fb5542..ea775119beb 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -53,7 +53,7 @@ the issue board feature. > - Multiple issue boards per project [moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53811) to GitLab Free in 12.1. > - Multiple issue boards per group are available in GitLab Premium. -Multiple issue boards allow for more than one issue board for a given project in GitLab Free or group in GitLab Premium and higher tiers. +Multiple issue boards allow for more than one issue board for a given project in the Free tier or group in the Premium and Ultimate tier. This is great for large projects with more than one team or when a repository hosts the code of multiple products. Using the search box at the top of the menu, you can filter the listed boards. @@ -97,9 +97,9 @@ Here are some common use cases for issue boards. For examples of using issue boards along with [epics](../group/epics/index.md), [issue health status](issues/managing_issues.md#health-status), and -[scoped labels](labels.md#scoped-labels) for various Agile frameworks, check: +[scoped labels](labels.md#scoped-labels) for various Agile frameworks, see: -- The [How to use GitLab for Agile portfolio planning and project management](https://about.gitlab.com/blog/2020/11/11/gitlab-for-agile-portfolio-planning-project-management/) blog post (November 2020) +- [How to use GitLab for Agile portfolio planning and project management](https://about.gitlab.com/blog/2020/11/11/gitlab-for-agile-portfolio-planning-project-management/) blog post (November 2020) - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Cross-project Agile work management with GitLab](https://www.youtube.com/watch?v=5J0bonGoECs) (15 min, July 2020) @@ -132,10 +132,17 @@ If you have the labels **Backend**, **Frontend**, **Staging**, and With [multiple issue boards](#multiple-issue-boards), each team can have their own board to organize their workflow individually. +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=2IaMXteKSD4">Portfolio Planning - Portfolio Management</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/2IaMXteKSD4" frameborder="0" allowfullscreen> </iframe> +</figure> + #### Scrum team With multiple issue boards, each team has one board. Now you can move issues through each -part of the process. For instance: **To Do**, **Doing**, and **Done**. +part of the process. For example: **To Do**, **Doing**, and **Done**. #### Organization of topics @@ -207,7 +214,7 @@ Prerequisites: - You must have at least the Reporter role for the project. When an issue is created, the system assigns a relative order value that is greater than the maximum value -of that issue's project or root group. This means the issue will be at the bottom of any issue list that +of that issue's project or root group. This means the issue is at the bottom of any issue list that it appears in. When you visit a board, issues appear ordered in any list. You're able to change @@ -226,6 +233,18 @@ This ordering also affects [issue lists](issues/sorting_issue_lists.md). Changing the order in an issue board changes the ordering in an issue list, and vice versa. +## Focus mode + +To enable or disable focus mode, in the upper-right corner, select **Toggle focus mode** (**{maximize}**). +In focus mode, the navigation UI is hidden, allowing you to focus on issues in the board. + +## Group issue boards + +Accessible at the group navigation level, a group issue board offers the same features as a project-level board. +It can display issues from all projects that fall under the group and its descendant subgroups. + +Users on GitLab Free can use a single group issue board. + ## GitLab Enterprise features for issue boards GitLab issue boards are available on the GitLab Free tier, but some @@ -245,7 +264,7 @@ This allows you to create unique boards according to your team's need. You can define the scope of your board when creating it or by selecting the **Edit board** button. After a milestone, iteration, assignee, or weight is assigned to an issue board, you can no longer -filter through these in the search bar. In order to do that, you need to remove the desired scope +filter through these in the search bar. To do that, you need to remove the desired scope (for example, milestone, assignee, or weight) from the issue board. If you don't have editing permission in a board, you're still able to see the configuration by @@ -255,14 +274,6 @@ selecting **View scope**. Watch a [video presentation](https://youtu.be/m5UTNCSqaDk) of the configurable issue board feature. -### Focus mode - -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28597) to GitLab Free SaaS in 12.10. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212331) to GitLab Free self-managed in 13.0. - -To enable or disable focus mode, select the **Toggle focus mode** button (**{maximize}**) at the top -right. In focus mode, the navigation UI is hidden, allowing you to focus on issues in the board. - ### Sum of issue weights **(PREMIUM)** > Moved to GitLab Premium in 13.9. @@ -273,13 +284,6 @@ especially in combination with [assignee lists](#assignee-lists).  -### Group issue boards - -Accessible at the group navigation level, a group issue board offers the same features as a project-level board. -It can display issues from all projects that fall under the group and its descendant subgroups. - -Users on GitLab Free can use a single group issue board. - ### Assignee lists **(PREMIUM)** As in a regular list showing all issues with a chosen label, you can add @@ -601,7 +605,7 @@ eventually pick it up. When they're done, they move it to **Done**, to close the issue. This process can be seen clearly when visiting an issue. With every move -to another list, the label changes and a system note is recorded. +to another list, the label changes and a [system note](system_notes.md) is recorded.  diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 2b302a60d63..79278d1b3ad 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -85,18 +85,21 @@ Learn how to create [merge requests for confidential issues](../merge_requests/c There are two kinds of level access for confidential issues. The general rule is that confidential issues are visible only to members of a project with at -least the Reporter role. However, a guest user can also create +least the **Reporter role**. + +However, a user with the **Guest role** can create confidential issues, but can only view the ones that they created themselves. -Users with the Guest role or non-members can also read the confidential issue if they are assigned to the issue. + +Users with the Guest role or non-members can read the confidential issue if they are assigned to the issue. When a Guest user or non-member is unassigned from a confidential issue, they can no longer view it. -Confidential issues are also hidden in search results for unprivileged users. +Confidential issues are hidden in search results for unprivileged users. For example, here's what a user with the Maintainer role and the Guest role -sees in the project's search results respectively. +sees in the project's search results: -| Maintainer role | Guest role | -|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------| +| Maintainer role | Guest role | +|:----------------|:-----------| |  |  | ## Related topics diff --git a/doc/user/project/issues/issue_weight.md b/doc/user/project/issues/issue_weight.md index d852ad3262b..05c348acf58 100644 --- a/doc/user/project/issues/issue_weight.md +++ b/doc/user/project/issues/issue_weight.md @@ -1,5 +1,4 @@ --- -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/issue_weight.html' stage: Plan group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index e0966909b1e..069bc4582c6 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -103,7 +103,7 @@ When bulk editing issues in a group, you can edit the following attributes: ## Move an issue When you move an issue, it's closed and copied to the target project. -The original issue is not deleted. A system note, which indicates +The original issue is not deleted. A [system note](../system_notes.md), which indicates where it came from and went to, is added to both issues. Be careful when moving an issue to a project with different access rules. Before moving the issue, make sure it does not contain sensitive data. @@ -298,7 +298,7 @@ To disable automatic issue closing: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. -1. Expand **Default branch**. +1. Expand **Branch defaults**. 1. Clear the **Auto-close referenced issues on default branch** checkbox. 1. Select **Save changes**. diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index 356e4e1b194..02a563f59b5 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -28,12 +28,14 @@ To invite a group to a project, you must be at least one of the following: In addition: -- The group you're inviting must have a more restrictive - [visibility level](../../public_access.md#project-and-group-visibility) - than the project. For example, you can invite: - - A private group to a public project. - - An internal group to a public project. - - A private group to an internal project. +- The [visibility level](../../public_access.md#project-and-group-visibility) of the group you're inviting +must be at least as restrictive as that of the project. For example, you can invite: + - A _private_ group to a _private_ project + - A _private_ group to an _internal_ project. + - A _private_ group to a _public_ project. + - An _internal_ group to an _internal_ project. + - An _internal_ group to a _public_ project. + - A _public_ group to a _public_ project. - The group or subgroup must be in the project's [namespace](../../namespace/index.md). For example, a project in the namespace `group/subgroup01/project`: diff --git a/doc/user/project/merge_requests/approvals/index.md b/doc/user/project/merge_requests/approvals/index.md index 21e2055cb61..9988cdcddd0 100644 --- a/doc/user/project/merge_requests/approvals/index.md +++ b/doc/user/project/merge_requests/approvals/index.md @@ -3,7 +3,6 @@ stage: Create group: Source Code info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" type: reference, concepts -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/approvals/index.html' --- # Merge request approvals **(FREE)** @@ -17,7 +16,7 @@ approve merge requests, these approvals are [optional](#optional-approvals). flexibility: - Create required [rules](rules.md) about the number and type of approvers before work can merge. -- Specify a list of users who act as [code owners](../../code_owners.md) for specific files, +- Specify a list of users who act as [code owners](../../codeowners/index.md) for specific files, and require their approval before work can merge. You can configure merge request approvals on a per-project basis, and some approvals can be configured @@ -36,7 +35,7 @@ required approvals before work can merge into your project. You can also extend rules to define what types of users can approve work. Some examples of rules you can create include: - Users with specific permissions can always approve work. -- [Code owners](../../code_owners.md) can approve work for files they own. +- [Code owners](../../codeowners/index.md) can approve work for files they own. - Users with specific permissions can approve work, [even if they don't have merge rights](rules.md#merge-request-approval-segregation-of-duties) to the repository. @@ -102,20 +101,30 @@ Without the approvals, the work cannot merge. Required approvals enable multiple database, for all proposed code changes. - Use the [code owners of changed files](rules.md#code-owners-as-eligible-approvers), to determine who should review the work. -- Require an [approval before merging code that causes test coverage to decline](../../../../ci/pipelines/settings.md#coverage-check-approval-rule) +- Require an [approval before merging code that causes test coverage to decline](../../../../ci/testing/code_coverage.md#coverage-check-approval-rule) - Users on GitLab Ultimate can also [require approval from a security team](../../../application_security/index.md#security-approvals-in-merge-requests) before merging code that could introduce a vulnerability. ## Invalid rules -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334698) in GitLab 15.1. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334698) in GitLab 15.1. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/389905) in GitLab 15.11 [with a flag](../../../../administration/feature_flags.md) named `invalid_scan_result_policy_prevents_merge`. Disabled by default. -Whenever an approval rule cannot be satisfied, the rule will be displayed as `Invalid`. This applies to the following conditions: +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, +ask an administrator to [enable the feature flag](../../../../administration/feature_flags.md) named `invalid_scan_result_policy_prevents_merge`. +On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. + +Whenever an approval rule cannot be satisfied, the rule is displayed as `(!) Auto approved`. This applies to the following conditions: - The only eligible approver is the author of the merge request. - No eligible approvers (either groups or users) have been assigned to the approval rule. -These rules will be automatically approved to unblock their respective merge requests. +These rules will be automatically approved (fail-open state) to unblock their respective merge requests, +unless they were created through a security policy. + +Invalid approval rules created through a security policy are presented with `(!) Action Required` +and are not automatically approved (fail-closed state), blocking their respective merge requests. ## Related topics diff --git a/doc/user/project/merge_requests/approvals/rules.md b/doc/user/project/merge_requests/approvals/rules.md index 0a5e7c29860..5c0282dd9e0 100644 --- a/doc/user/project/merge_requests/approvals/rules.md +++ b/doc/user/project/merge_requests/approvals/rules.md @@ -106,6 +106,9 @@ reduces the number of approvals left (the **Approvals** column) for all rules th  +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Multiple Approvers](https://www.youtube.com/watch?v=8JQJ5821FrA). + ## Eligible approvers > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10294) in GitLab 13.3, when an eligible approver comments on a merge request, it appears in the **Commented by** column of the Approvals widget. @@ -158,7 +161,7 @@ approve in these ways: > Moved to GitLab Premium in 13.9. -If you add [code owners](../../code_owners.md) to your repository, the owners of files +If you add [code owners](../../codeowners/index.md) to your repository, the owners of files become eligible approvers in the project. To enable this merge request approval rule: 1. Go to your project and select **Settings > Merge requests**. @@ -241,6 +244,28 @@ approval rule for certain branches: 1. To enable this configuration, read [Code Owner's approvals for protected branches](../../protected_branches.md#require-code-owner-approval-on-a-protected-branch). +## Code coverage check approval + +You can require specific approvals in the case that a merge request would reduce code test +coverage. + +For more information, see [Coverage check approval rule](../../../../ci/testing/code_coverage.md#coverage-check-approval-rule). + +## Security Approvals **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357021) in GitLab 15.0. + +You can use [scan result policies](../../../application_security/policies/scan-result-policies.md#scan-result-policy-editor) to define security approvals based on the status of vulnerabilities in the merge request and the default branch. +Details for each security policy is shown in the Security Approvals section of your Merge Request configuration. + +The security approval rules are applied to all merge requests until the pipeline is complete. The application of the +security approval rules prevents users from merging in code before the security scans run. After the pipeline is +complete, the security approval rules are checked to determine if the security approvals are still required. + + + +These policies are both created and edited in the [security policy editor](../../../application_security/policies/index.md#policy-editor). + ## Troubleshooting ### Approval rule name can't be blank @@ -262,18 +287,3 @@ isn't recognized as a valid Code Owner for the project, nor does it display in t project's **Approvals** list. To fix this problem, add the approval group as a shared group high enough in the shared hierarchy so the project requiring review inherits this group of users. - -## Security Approvals **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357021) in GitLab 15.0. - -You can use [scan result policies](../../../application_security/policies/scan-result-policies.md#scan-result-policy-editor) to define security approvals based on the status of vulnerabilities in the merge request and the default branch. -Details for each security policy is shown in the Security Approvals section of your Merge Request configuration. - -The security approval rules are applied to all merge requests until the pipeline is complete. The application of the -security approval rules prevents users from merging in code before the security scans run. Once the pipeline is -complete, the security approval rules are checked to determine if the security approvals are still required. - - - -These policies are both created and edited in the [security policy editor](../../../application_security/policies/index.md#policy-editor). diff --git a/doc/user/project/merge_requests/approvals/settings.md b/doc/user/project/merge_requests/approvals/settings.md index 1ab564c108b..154be8a83bb 100644 --- a/doc/user/project/merge_requests/approvals/settings.md +++ b/doc/user/project/merge_requests/approvals/settings.md @@ -78,13 +78,13 @@ their own. To do this: it can't be changed at the project level. 1. Select **Save changes**. -Depending on your version of GitLab, [code owners](../../code_owners.md) who commit +Depending on your version of GitLab, [code owners](../../codeowners/index.md) who commit to a merge request may or may not be able to approve the work: -- In GitLab 13.10 and earlier, [code owners](../../code_owners.md) who commit +- In GitLab 13.10 and earlier, code owners who commit to a merge request can approve it, even if the merge request affects files they own. - In [GitLab 13.11 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/331548), - [code owners](../../code_owners.md) who commit + code owners who commit to a merge request cannot approve it, when the merge request affects files they own. For more information, see the [official Git documentation](https://git-scm.com/book/en/v2/Git-Basics-Viewing-the-Commit-History). @@ -140,7 +140,7 @@ However, approvals are reset if the target branch is changed. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90578) in GitLab 15.3. -If you only want to remove approvals by Code Owners whose files have been changed: +If you only want to remove approvals by Code Owners whose files have been changed when a commit is added: Prerequisite: @@ -153,13 +153,6 @@ To do this: select **Remove approvals by Code Owners if their files changed**. 1. Select **Save changes**. -## Code coverage check approvals - -You can require specific approvals if a merge request would result in a decline in code test -coverage. - -For more information, see [Coverage check approval rule](../../../../ci/pipelines/settings.md#coverage-check-approval-rule). - ## Settings cascading > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285410) in GitLab 14.4. [Deployed behind the `group_merge_request_approval_settings_feature_flag` flag](../../../../administration/feature_flags.md), disabled by default. diff --git a/doc/user/project/merge_requests/cherry_pick_changes.md b/doc/user/project/merge_requests/cherry_pick_changes.md index d2676b1bdf0..004d24778b7 100644 --- a/doc/user/project/merge_requests/cherry_pick_changes.md +++ b/doc/user/project/merge_requests/cherry_pick_changes.md @@ -125,7 +125,7 @@ project, from the GitLab user interface: ## View system notes for cherry-picked commits -When you cherry-pick a merge commit in the GitLab UI or API, GitLab adds a system note +When you cherry-pick a merge commit in the GitLab UI or API, GitLab adds a [system note](../system_notes.md) to the related merge request thread in the format **{cherry-pick-commit}** `[USER]` **picked the changes into the branch** `[BRANCHNAME]` with commit** `[SHA]` `[DATE]`: diff --git a/doc/user/project/merge_requests/confidential.md b/doc/user/project/merge_requests/confidential.md index 307998f697d..edcc5711b98 100644 --- a/doc/user/project/merge_requests/confidential.md +++ b/doc/user/project/merge_requests/confidential.md @@ -65,7 +65,7 @@ Your branch, merge requests, and commits remain in your private fork. This preve prematurely revealing confidential information. Open a merge request -[from your fork to the upstream repository](../repository/forking_workflow.md#merging-upstream) when: +[from your fork to the upstream repository](../repository/forking_workflow.md#merge-changes-back-upstream) when: - You are satisfied the problem is resolved in your private fork. - You are ready to make the confidential commits public. diff --git a/doc/user/project/merge_requests/conflicts.md b/doc/user/project/merge_requests/conflicts.md index 932eec5e631..cc8f8cb2fe6 100644 --- a/doc/user/project/merge_requests/conflicts.md +++ b/doc/user/project/merge_requests/conflicts.md @@ -162,9 +162,8 @@ merge commit. Verify it contains no unintended changes and doesn't break your bu ## Related topics -- [Introduction to Git rebase and force-push](../../../topics/git/git_rebase.md). -- [Git GUI apps](https://git-scm.com/downloads/guis) to help you visualize the - differences between branches and resolve them. +- [Introduction to Git rebase and force-push](../../../topics/git/git_rebase.md) +- [Git applications for visualizing the Git workflow](https://git-scm.com/downloads/guis) <!-- ## Troubleshooting diff --git a/doc/user/project/merge_requests/creating_merge_requests.md b/doc/user/project/merge_requests/creating_merge_requests.md index 4ea549e5986..8a4a61bb80d 100644 --- a/doc/user/project/merge_requests/creating_merge_requests.md +++ b/doc/user/project/merge_requests/creating_merge_requests.md @@ -3,7 +3,6 @@ stage: Create group: Code Review info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: "How to create merge requests in GitLab." -disqus_identifier: 'https://docs.gitlab.com/ee/gitlab-basics/add-merge-request.html' --- # Creating merge requests **(FREE)** @@ -34,7 +33,8 @@ be associated with a given target branch at a time. If your development workflow requires an issue for every merge request, you can create a branch directly from the issue to speed the process up. The new branch, and later its merge request, are marked as related to this issue. -After merging the merge request, the issue is closed automatically, unless [automatic issue closing is disabled](../issues/managing_issues.md#disable-automatic-issue-closing). +After merging the merge request, the issue is closed automatically, unless +[automatic issue closing is disabled](../issues/managing_issues.md#disable-automatic-issue-closing). You can see a **Create merge request** dropdown list below the issue description. NOTE: @@ -50,7 +50,7 @@ instead of immediately creating the merge request. - Your project is private and the issue is confidential. To make this button appear, one possible workaround is to -[remove your project's fork relationship](../settings/index.md#remove-a-fork-relationship). +[remove your project's fork relationship](../repository/forking_workflow.md#unlink-a-fork). After removal, the fork relationship cannot be restored. This project can no longer be able to receive or send merge requests to the source project, or other forks. @@ -58,8 +58,8 @@ The dropdown list contains the options **Create merge request and branch** and * After selecting one of these options, a new branch or branch and merge request is created based on your project's [default branch](../repository/branches/default.md). -The branch name is based on your project's branch name template. The default template -is `%{id}-%{title}`. Supported variables for branch name templates are `%{id}` and `%{title}`. +The branch name is based on your project's [branch name template](../repository/branches/index.md), +but this value can be changed. When you select **Create branch** in an empty repository project, GitLab performs these actions: @@ -151,9 +151,8 @@ You can create a merge request from your fork to contribute back to the main pro 1. Select **Create merge request**. After your work is merged, if you don't intend to -make any other contributions to the upstream project, you can unlink your -fork from its upstream project. Go to **Settings > Advanced Settings** and -[remove the forking relationship](../settings/index.md#remove-a-fork-relationship). +make any other contributions to the upstream project, you can +[unlink your fork](../repository/forking_workflow.md#unlink-a-fork) from its upstream project. For more information, [see the forking workflow documentation](../repository/forking_workflow.md). diff --git a/doc/user/project/merge_requests/drafts.md b/doc/user/project/merge_requests/drafts.md index 62820988701..839f3d17a43 100644 --- a/doc/user/project/merge_requests/drafts.md +++ b/doc/user/project/merge_requests/drafts.md @@ -3,7 +3,6 @@ stage: Create group: Code Review info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference, concepts -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/work_in_progress_merge_requests.html' --- # Draft merge requests **(FREE)** diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/project/merge_requests/getting_started.md deleted file mode 100644 index 4125d8e8fdb..00000000000 --- a/doc/user/project/merge_requests/getting_started.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'index.md' -remove_date: '2023-03-31' ---- - -This document was moved to [another location](index.md). - -<!-- This redirect file can be deleted after <2023-03-31>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index f760b1b730a..ee7f4e5dfed 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -30,7 +30,7 @@ Learn the various ways to [create a merge request](creating_merge_requests.md). You can view merge requests for your project, group, or yourself. -### View merge requests for a project +### For a project To view all merge requests for a project: @@ -39,7 +39,7 @@ To view all merge requests for a project: Or, to use a [keyboard shortcut](../../shortcuts.md), press <kbd>g</kbd> + <kbd>m</kbd>. -### View merge requests for all projects in a group +### For all projects in a group To view merge requests for all projects in a group: @@ -48,7 +48,7 @@ To view merge requests for all projects in a group: If your group contains subgroups, this view also displays merge requests from the subgroup projects. -### View all merge requests assigned to you +### Assigned to you To view all merge requests assigned to you: @@ -79,12 +79,12 @@ To filter the list of merge requests: 1. Above the list of merge requests, select **Search or filter results...**. 1. From the dropdown list, select the attribute you wish to filter by. Some examples: - - [**By environment or deployment date**](#filter-merge-requests-by-environment-or-deployment-date). + - [**By environment or deployment date**](#by-environment-or-deployment-date). - **ID**: Enter filter `#30` to return only merge request 30. - User filters: Type (or select from the dropdown list) any of these filters to display a list of users: - **Approved-By**, for merge requests already approved by a user. **(PREMIUM)**. - **Approver**, for merge requests that this user is eligible to approve. - (For more information, read about [Code owners](../code_owners.md)). **(PREMIUM)** + (For more information, read about [Code owners](../codeowners/index.md)). **(PREMIUM)** - **Reviewer**, for merge requests reviewed by this user. 1. Select or type the operator to use for filtering the attribute. The following operators are available: @@ -100,7 +100,7 @@ To filter the list of merge requests: GitLab displays the results on-screen, but you can also [retrieve them as an RSS feed](../../search/index.md#retrieve-search-results-as-feed). -### Filter merge requests by environment or deployment date +### By environment or deployment date > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/44041) in GitLab 13.6. diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index f0359446b06..102a2ad1c14 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -147,3 +147,11 @@ merge-request-pipeline-job: Instead, use branch (`push`) pipelines or merge request pipelines, when possible. For details on avoiding two pipelines for a single merge request, read the [`rules` documentation](../../../ci/jobs/job_control.md#avoid-duplicate-pipelines). + +### Merged results pipeline allows merge, despite a failed branch pipeline + +When [the **Pipelines must succeed** setting](#require-a-successful-pipeline-for-merge) +is combined with +[the **Merged results pipelines** feature](../../../ci/pipelines/merged_results_pipelines.md), +failed branch pipeline may be ignored. +[Issue 385841](https://gitlab.com/gitlab-org/gitlab/-/issues/385841) is open to track this. diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md index 5878873f209..77fd78ee0d0 100644 --- a/doc/user/project/merge_requests/revert_changes.md +++ b/doc/user/project/merge_requests/revert_changes.md @@ -58,7 +58,8 @@ To do this: 1. On the top bar, select **Main menu > Projects** and find your project. 1. If you know the merge request that contains the commit: 1. On the left sidebar, select **Merge requests** and identify your merge request. - 1. Select **Commits**, then select the title of the commit you want to revert. GitLab displays the contents of the commit. + 1. Select **Commits**, then select the title of the commit you want to revert. This displays the commit in the **Changes** tab of your merge request. + 1. Select the commit hash you want to revert. GitLab displays the contents of the commit. 1. If you don't know the merge request the commit originated from: 1. On the left sidebar, select **Repository > Commits**. 1. Select the title of the commit to display full information about the commit. diff --git a/doc/user/project/merge_requests/reviews/img/add_another_suggestion_to_batch_v13_1.jpg b/doc/user/project/merge_requests/reviews/img/add_another_suggestion_to_batch_v13_1.jpg Binary files differdeleted file mode 100644 index e8aa4b7c730..00000000000 --- a/doc/user/project/merge_requests/reviews/img/add_another_suggestion_to_batch_v13_1.jpg +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/add_first_suggestion_to_batch_v13_1.jpg b/doc/user/project/merge_requests/reviews/img/add_first_suggestion_to_batch_v13_1.jpg Binary files differdeleted file mode 100644 index d15f5d53e55..00000000000 --- a/doc/user/project/merge_requests/reviews/img/add_first_suggestion_to_batch_v13_1.jpg +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/apply_batch_of_suggestions_v13_1.jpg b/doc/user/project/merge_requests/reviews/img/apply_batch_of_suggestions_v13_1.jpg Binary files differdeleted file mode 100644 index 3e1e9c20af9..00000000000 --- a/doc/user/project/merge_requests/reviews/img/apply_batch_of_suggestions_v13_1.jpg +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/custom_commit_v13_9.png b/doc/user/project/merge_requests/reviews/img/custom_commit_v13_9.png Binary files differdeleted file mode 100644 index 170c04542dd..00000000000 --- a/doc/user/project/merge_requests/reviews/img/custom_commit_v13_9.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/multi-line-suggestion-syntax.png b/doc/user/project/merge_requests/reviews/img/multi-line-suggestion-syntax.png Binary files differdeleted file mode 100644 index 80424d1f056..00000000000 --- a/doc/user/project/merge_requests/reviews/img/multi-line-suggestion-syntax.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/remove_suggestion_from_batch_v13_1.jpg b/doc/user/project/merge_requests/reviews/img/remove_suggestion_from_batch_v13_1.jpg Binary files differdeleted file mode 100644 index fa8331effdb..00000000000 --- a/doc/user/project/merge_requests/reviews/img/remove_suggestion_from_batch_v13_1.jpg +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md index 4bd77c7ebdd..22e95c7e639 100644 --- a/doc/user/project/merge_requests/reviews/index.md +++ b/doc/user/project/merge_requests/reviews/index.md @@ -23,14 +23,14 @@ review merge requests in Visual Studio Code. ## Suggested reviewers **(ULTIMATE SAAS)** -> [Introduced](https://gitlab.com/groups/gitlab-org/modelops/applied-ml/review-recommender/-/epics/3) in GitLab 15.4. +> - [Introduced](https://gitlab.com/groups/gitlab-org/modelops/applied-ml/review-recommender/-/epics/3) in GitLab 15.4 as a [Beta](../../../../policy/alpha-beta-support.md#beta) feature [with a flag](../../../../administration/feature_flags.md) named `suggested_reviewers_control`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/368356) in GitLab 15.6. +> - Beta designation [removed from the UI](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113058) in GitLab 15.10. GitLab can suggest reviewers. Using the changes in a merge request and a project's contribution graph, machine learning suggestions appear in the reviewer section of the right sidebar.  -This feature is currently in [Open Beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#open-beta) behind a [feature flag](https://gitlab.com/gitlab-org/gitlab/-/issues/368356). - For more information, see [Data usage in Suggested Reviewers](data_usage.md). ### Enable suggested reviewers @@ -107,7 +107,7 @@ To download the changes included in a merge request as a patch file: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests**. 1. Select your merge request. -1. In the upper-right corner, select **Code > Email patches**. +1. In the upper-right corner, select **Code > Patches**. If you know the URL of the merge request, you can also download the patch from the command line by appending `.patch` to the URL. This example downloads the patch @@ -172,7 +172,7 @@ If you have a review in progress, you can also add a comment from the **Overview When editing the **Reviewers** field in a new or existing merge request, GitLab displays the name of the matching [approval rule](../approvals/rules.md) -below the name of each suggested reviewer. [Code Owners](../../code_owners.md) are displayed as `Codeowner` without group detail. +below the name of each suggested reviewer. [Code Owners](../../codeowners/index.md) are displayed as `Codeowner` without group detail. This example shows reviewers and approval rules when creating a new merge request: diff --git a/doc/user/project/merge_requests/reviews/suggestions.md b/doc/user/project/merge_requests/reviews/suggestions.md index 6976d4052e1..9187c5fad44 100644 --- a/doc/user/project/merge_requests/reviews/suggestions.md +++ b/doc/user/project/merge_requests/reviews/suggestions.md @@ -45,28 +45,40 @@ merge request, authored by the user who suggested the changes. > [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/232339) in GitLab 13.11: suggestions in multi-line comments also become multi-line. -Reviewers can also suggest changes to multiple lines with a single suggestion -within merge request diff threads by selecting and dragging selection to all -relevant line numbers or by adjusting the range offsets. The -offsets are relative to the position of the diff thread, and specify the -range to be replaced by the suggestion when it is applied. - - +When you review a merge request diff, you can propose changes to multiple lines (up to 200) +in a single suggestion, by either: + +- Selecting and dragging, as described in [Create suggestions](#create-suggestions). + GitLab creates a suggestion block for you. +- Selecting a single line, then manually adjusting the range offsets. + +The range offsets in the first line of the suggestion describe line numbers relative +to the line you selected. The offsets specify the lines your suggestion intends to replace. +For example, this suggestion covers 3 lines above and 4 lines below the +commented line: + +````markdown +```suggestion:-3+4 + "--talk-name=ca.desrt.dconf", + "--socket=wayland", +``` +```` -In the previous example, the suggestion covers three lines above and four lines -below the commented line. When applied, it would replace from 3 lines _above_ -to 4 lines _below_ the commented line, with the suggested change. +When applied, the suggestion replaces from 3 lines above to 4 lines below the commented line:  -NOTE: Suggestions for multiple lines are limited to 100 lines _above_ and 100 lines _below_ the commented diff line. This allows for up to 200 changed lines per suggestion. ## Apply suggestions -The merge request author can apply suggested changes directly from the merge request: +Prerequisites: + +- You must be the author of the merge request, or have at least the Developer role in the project. + +To apply suggested changes directly from the merge request: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests** and find your merge request. @@ -143,35 +155,29 @@ For example, to customize the commit message to output ## Batch suggestions -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25486) in GitLab 13.1 as an [alpha feature](../../../../policy/alpha-beta-support.md#alpha-features) with a flag named `batch_suggestions`, disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25486) in GitLab 13.1 as an [Experiment](../../../../policy/alpha-beta-support.md#experiment) with a flag named `batch_suggestions`, disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/227799) in GitLab 13.2. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/320755) in GitLab 13.11. [Feature flag `batch_suggestions`](https://gitlab.com/gitlab-org/gitlab/-/issues/320755) removed. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326168) custom commit messages for batch suggestions in GitLab 14.4. -You can apply multiple suggestions at once to reduce the number of commits added -to your branch to address your reviewers' requests. - -1. To start a batch of suggestions to apply with a single commit, select **Add suggestion to batch**: - -  - -1. Add as many additional suggestions to the batch as you wish: +To reduce the number of commits added to your branch, you can apply multiple +suggestions in a single commit. -  - -1. To remove suggestions, select **Remove from batch**: - -  - -1. Having added all the suggestions to your liking, when ready, select **Apply suggestions**. You - can optionally specify a custom commit message for [batch suggestions](#batch-suggestions) - (GitLab 14.4 and later) to describe your change. If you don't specify it, the default commit - message is used. - -  - -WARNING: -Suggestions applied from multiple authors creates a commit authored by the user applying the suggestions. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. For each suggestion you want to apply, and select **Add suggestion to batch**. +1. Optional. To remove a suggestion, select **Remove from batch**. +1. After you add your desired suggestions, select **Apply suggestions**. + + WARNING: + If you apply a batch of suggestions containing changes from multiple authors, + you are credited as the resulting commit's author. If your project is configured + to [prevent approvals from users who add commits](../approvals/settings.md#prevent-approvals-by-users-who-add-commits), you are no longer an eligible + approver for this merge request. + +1. Optional. Provide a custom commit message for [batch suggestions](#batch-suggestions) + (GitLab 14.4 and later) to describe your change. If you don't specify one, + the default commit message is used. ## Related topics diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index 9f87f1e2e0d..075716e90c8 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -33,7 +33,14 @@ squash commits and merge commits. ## Set default squash options for a merge request Users with permission to create or edit a merge request can set the default squash options -for a merge request. To do this: +for a merge request. + +Prerequisites: + +- Your project must be [configured](#configure-squash-options-for-a-project) to allow or + encourage squashing. + +To do this: 1. Go to the merge request and select **Edit**. 1. Select or clear the **Squash commits when merge request is accepted** checkbox. @@ -58,6 +65,10 @@ squash the commits as part of the merge process: > - [Enabled on GitLab.com and self-managed by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39382) in GitLab 13.3. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/232536) in GitLab 13.8. Feature flag `squash_options` removed. +Prerequisites: + +- You must have at least the Maintainer role for this project. + To configure the default squashing behavior for all merge requests in your project: 1. On the top bar, select **Main menu > Projects** and find your project. diff --git a/doc/user/project/merge_requests/status_checks.md b/doc/user/project/merge_requests/status_checks.md index 6a791a17057..4c806086fab 100644 --- a/doc/user/project/merge_requests/status_checks.md +++ b/doc/user/project/merge_requests/status_checks.md @@ -3,7 +3,6 @@ stage: Govern group: Compliance info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" type: reference, concepts -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/status_checks.html' --- # External status checks **(ULTIMATE)** @@ -12,6 +11,8 @@ disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/statu > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/320783) in GitLab 14.1. > - `failed` status [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/329636) in GitLab 14.9. +Status checks are API calls to external systems that request the status of an external requirement. + You can create a status check that sends merge request data to third-party tools. When users create, change, or close merge requests, GitLab sends a notification. The users or automated workflows can then update the status of merge requests from outside of GitLab. diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index e9de780655a..4641af262ca 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -142,7 +142,7 @@ To edit a milestone: 1. On the top bar, select **Main menu > Projects** and find your project or **Main menu > Groups** and find your group. 1. Select a milestone's title. -1. Select **Edit**. +1. In the top right corner, select **Milestone actions** (**{ellipsis_v}**) and then select **Edit**. 1. Edit the title, start date, due date, or description. 1. Select **Save changes**. @@ -158,7 +158,7 @@ To edit a milestone: 1. On the top bar, select **Main menu > Projects** and find your project or **Main menu > Groups** and find your group. 1. Select a milestone's title. -1. Select **Delete**. +1. In the top right corner, select **Milestone actions** (**{ellipsis_v}**) and then select **Delete**. 1. Select **Delete milestone**. ## Promote a project milestone to a group milestone @@ -185,7 +185,7 @@ To promote a project milestone: 1. On the left sidebar, select **Issues > Milestones**. 1. Either: - Select **Promote to Group Milestone** (**{level-up}**) next to the milestone you want to promote. - - Select the milestone title, and then select **Promote**. + - Select the milestone title, and then select **Milestone actions** (**{ellipsis_v}**) > **Promote**. 1. Select **Promote Milestone**. ## Assign a milestone to an issue or merge request diff --git a/doc/user/project/ml/experiment_tracking/img/candidate_v15_11.png b/doc/user/project/ml/experiment_tracking/img/candidate_v15_11.png Binary files differnew file mode 100644 index 00000000000..50bcd1e8479 --- /dev/null +++ b/doc/user/project/ml/experiment_tracking/img/candidate_v15_11.png diff --git a/doc/user/project/ml/experiment_tracking/img/candidates_v15_11.png b/doc/user/project/ml/experiment_tracking/img/candidates_v15_11.png Binary files differnew file mode 100644 index 00000000000..3a2944c92ae --- /dev/null +++ b/doc/user/project/ml/experiment_tracking/img/candidates_v15_11.png diff --git a/doc/user/project/ml/experiment_tracking/img/experiments_v15_11.png b/doc/user/project/ml/experiment_tracking/img/experiments_v15_11.png Binary files differnew file mode 100644 index 00000000000..f7e25c1a0f1 --- /dev/null +++ b/doc/user/project/ml/experiment_tracking/img/experiments_v15_11.png diff --git a/doc/user/project/ml/experiment_tracking/index.md b/doc/user/project/ml/experiment_tracking/index.md index a7096d633a0..114ccea7c09 100644 --- a/doc/user/project/ml/experiment_tracking/index.md +++ b/doc/user/project/ml/experiment_tracking/index.md @@ -4,78 +4,86 @@ group: Incubation info: Machine Learning Experiment Tracking is a GitLab Incubation Engineering program. No technical writer assigned to this group. --- -# Machine Learning Experiment Tracking **(FREE)** +# Machine learning model experiments **(FREE)** -DISCLAIMER: -Machine Learning Experiment Tracking is an experimental feature being developed by the Incubation Engineering Department, -and will receive significant changes over time. This feature is being release with the aim of getting user feedback, but -is not stable and can lead to performance degradation. See below on how to disable this feature. +FLAG: +On self-managed GitLab, model experiment tracking is disabled by default. +To enable the feature, ask an administrator to [enable the feature flag](../../../../administration/feature_flags.md) named `ml_experiment_tracking`. +On GitLab.com, this feature is in private testing only. -When creating machine learning models, data scientists often experiment with different parameters, configurations, feature -engineering, and so on, to improve the performance of the model. Keeping track of all this metadata and the associated +NOTE: +Model experiment tracking is an [experimental feature](../../../../policy/alpha-beta-support.md). Refer to <https://gitlab.com/gitlab-org/gitlab/-/issues/381660> for feedback and feature requests. + +When creating machine learning models, data scientists often experiment with different parameters, configurations, and feature +engineering to improve the performance of the model. Keeping track of all this metadata and the associated artifacts so that the data scientist can later replicate the experiment is not trivial. Machine learning experiment tracking enables them to log parameters, metrics, and artifacts directly into GitLab, giving easy access later on. - +These features have been proposed: - +- Searching experiments. +- Visual comparison of candidates. +- Creating, deleting, and updating experiments through the GitLab UI. +- Creating, deleting, and updating candidates through the GitLab UI. - +For feature requests, see [epic 9341](https://gitlab.com/groups/gitlab-org/-/epics/9341). ## What is an experiment? -An experiment is a collection of comparable model candidates. Experiments can be long lived (for example, when they represent -a use case), or short lived (results from hyperparameter tuning triggered by a merge request), but usually hold model candidates -that have a similar set of parameters and metrics. +In a project, an experiment is a collection of comparable model candidates. +Experiments can be long-lived (for example, when they represent a use case), or +short-lived (results from hyperparameter tuning triggered by a merge request), +but usually hold model candidates that have a similar set of parameters measured +by the same metrics. + + ## Model candidate A model candidate is a variation of the training of a machine learning model, that can be eventually promoted to a version -of the model. The goal of a data scientist is to find the model candidate whose parameter values lead to the best model +of the model. + + + +The goal of a data scientist is to find the model candidate whose parameter values lead to the best model performance, as indicated by the given metrics. -Example parameters: + + +Some example parameters: -- Algorithm (linear regression, decision tree, and so on). +- Algorithm (such as linear regression or decision tree). - Hyperparameters for the algorithm (learning rate, tree depth, number of epochs). - Features included. -## Usage +## Track new experiments and candidates -### User access management +Experiment and trials can only be tracked through the +[MLFlow](https://www.mlflow.org/docs/latest/tracking.html) client integration. +See [MLFlow client integration](../../integrations/mlflow_client.md) for more information +on how to use GitLab as a backend for the MLFlow Client. -An experiment is always associated to a project. Only users with access to the project an experiment is associated with -can view that experiment data. +## Explore model candidates -### Tracking new experiments and trials +Prerequisites: -Experiment and trials can only be tracked through the [MLFlow](https://www.mlflow.org/docs/latest/tracking.html) client -integration. More information on how to use GitLab as a backend for MLFlow Client can be found [at the documentation page](../../integrations/mlflow_client.md). +- You must have at least the Developer role to view experiment data. -### Exploring model candidates +To list the current active experiments, either go to `https/-/ml/experiments` or: -To list the current active experiments, navigate to `https/-/ml/experiments`. To display all trials -that have been logged, along with their metrics and parameters, select an experiment. To display details for a candidate, -select **Details**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Packages & registries > Model experiments**. +1. To display all candidates that have been logged, along with their metrics, parameters, and metadata, select an experiment. +1. To display details for a candidate, select **Details**. -### Logging artifacts +## View log artifacts Trial artifacts are saved as [generic packages](../../../packages/generic_packages/index.md), and follow all their -conventions. After an artifact is logged for a candidate, all artifacts logged for the candidate are listed in the -package registry. The package name for a candidate is `ml_candidate_<candidate_id>`, with version `-`. The link to the -artifacts can also be accessed from the **Experiment Candidates** list or **Candidate detail**. - -### Limitations and future - -- Searching experiments, searching trials, visual comparison of trials, and creating, deleting and updating experiments and trials through GitLab UI is under development. - -## Disabling or enabling the Feature - -On self-managed GitLab, ML Experiment Tracking is disabled by default. To enable the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `ml_experiment_tracking`. -On GitLab.com, this feature is currently on private testing. - -## Feedback, roadmap and reports +limitations. After an artifact is logged for a candidate, all artifacts logged for the candidate are listed in the +package registry. The package name for a candidate is `ml_experiment_<experiment_id>`, where the version is the candidate +IID. The link to the artifacts can also be accessed from the **Experiment Candidates** list or **Candidate detail**. -For updates on the development, refer to the [development epic](https://gitlab.com/groups/gitlab-org/-/epics/8560). +## Related topics -For feedback, bug reports and feature requests, refer to the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/381660). +- Development details in [epic 8560](https://gitlab.com/groups/gitlab-org/-/epics/8560). +- Add feedback in [issue 381660](https://gitlab.com/gitlab-org/gitlab/-/issues/381660). diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md index a92f65b064e..5cdf493fe6f 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md @@ -1,7 +1,7 @@ --- type: concepts -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index 3bb62921979..e35be518fc6 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -1,7 +1,6 @@ --- -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pages/getting_started_part_three.html' -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -197,38 +196,6 @@ from the GitLab project. in place. Your domain is periodically reverified, and may be disabled if the record is removed. -##### Troubleshoot domain verification - -To manually verify that you have properly configured the domain verification -`TXT` DNS entry, you can run the following command in your terminal: - -```shell -dig _gitlab-pages-verification-code.<YOUR-PAGES-DOMAIN> TXT -``` - -Expect the output: - -```plaintext -;; ANSWER SECTION: -_gitlab-pages-verification-code.<YOUR-PAGES-DOMAIN>. 300 IN TXT "gitlab-pages-verification-code=<YOUR-VERIFICATION-CODE>" -``` - -In some cases it can help to add the verification code with the same domain name as you are trying to register. - -For a root domain: - -| From | DNS Record | To | -| ------------------------------------------------- | ---------- | ---------------------- | -| `example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | -| `_gitlab-pages-verification-code.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | - -For a subdomain: - -| From | DNS Record | To | -| ------------------------------------------------- | ---------- | ---------------------- | -| `www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | -| `_gitlab-pages-verification-code.www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | - ### Add more domain aliases You can add more than one alias (custom domains and subdomains) to the same project. @@ -352,14 +319,36 @@ To enable this setting: If you use Cloudflare CDN in front of GitLab Pages, make sure to set the SSL connection setting to `full` instead of `flexible`. For more details, see the [Cloudflare CDN directions](https://developers.cloudflare.com/ssl/origin-configuration/ssl-modes#h_4e0d1a7c-eb71-4204-9e22-9d3ef9ef7fef). -<!-- ## Troubleshooting +## Troubleshooting + +### Domain verification + +To manually verify that you have properly configured the domain verification +`TXT` DNS entry, you can run the following command in your terminal: + +```shell +dig _gitlab-pages-verification-code.<YOUR-PAGES-DOMAIN> TXT +``` + +Expect the output: + +```plaintext +;; ANSWER SECTION: +_gitlab-pages-verification-code.<YOUR-PAGES-DOMAIN>. 300 IN TXT "gitlab-pages-verification-code=<YOUR-VERIFICATION-CODE>" +``` + +In some cases it can help to add the verification code with the same domain name as you are trying to register. -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +For a root domain: -Each scenario can be a third-level heading, for example, `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +| From | DNS Record | To | +| ------------------------------------------------- | ---------- | ---------------------- | +| `example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | +| `_gitlab-pages-verification-code.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | + +For a subdomain: + +| From | DNS Record | To | +| ------------------------------------------------- | ---------- | ---------------------- | +| `www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | +| `_gitlab-pages-verification-code.www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md index 34a2f221327..130709de3a5 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md @@ -1,8 +1,8 @@ --- type: reference description: "Automatic Let's Encrypt SSL certificates for GitLab Pages." -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md index 0ba00177659..484dc784fdb 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md @@ -1,7 +1,7 @@ --- type: concepts -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/pages/getting_started/pages_ci_cd_template.md b/doc/user/project/pages/getting_started/pages_ci_cd_template.md index 37f74d18d4c..17618146350 100644 --- a/doc/user/project/pages/getting_started/pages_ci_cd_template.md +++ b/doc/user/project/pages/getting_started/pages_ci_cd_template.md @@ -1,7 +1,7 @@ --- type: reference, howto -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -37,4 +37,4 @@ For every change pushed to your repository, GitLab CI/CD runs a new pipeline that immediately publishes your changes to the Pages site. To view the HTML and other assets that were created for the site, -[download the job artifacts](../../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +[download the job artifacts](../../../../ci/jobs/job_artifacts.md#download-job-artifacts). diff --git a/doc/user/project/pages/getting_started/pages_forked_sample_project.md b/doc/user/project/pages/getting_started/pages_forked_sample_project.md index a864c114b3e..e8de80ac137 100644 --- a/doc/user/project/pages/getting_started/pages_forked_sample_project.md +++ b/doc/user/project/pages/getting_started/pages_forked_sample_project.md @@ -1,7 +1,7 @@ --- type: reference, howto -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -18,7 +18,7 @@ configured to generate a Pages site. To fork a sample project and create a Pages website: 1. View the sample projects by navigating to the [GitLab Pages examples](https://gitlab.com/pages) group. -1. Select the name of the project you want to [fork](../../repository/forking_workflow.md#creating-a-fork). +1. Select the name of the project you want to [fork](../../repository/forking_workflow.md#create-a-fork). 1. In the upper-right corner, select **Fork**, then choose a namespace to fork to. 1. For your project, on the left sidebar, select **CI/CD > Pipelines** and then **Run pipeline**. GitLab CI/CD builds and deploys your site. @@ -65,4 +65,4 @@ you can rename it to `<namespace>.gitlab.io`, where `<namespace>` is your GitLab ## Related topics -- [Download the job artifacts](../../../../ci/pipelines/job_artifacts.md#download-job-artifacts) +- [Download the job artifacts](../../../../ci/jobs/job_artifacts.md#download-job-artifacts) diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md index a3d6c8f75f9..c62ead69216 100644 --- a/doc/user/project/pages/getting_started/pages_from_scratch.md +++ b/doc/user/project/pages/getting_started/pages_from_scratch.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -423,16 +423,11 @@ Now GitLab CI/CD not only builds the website, but also: - **Continuously deploys** every push to the `main` branch. To view the HTML and other assets that were created for the site, -[download the job artifacts](../../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +[download the job artifacts](../../../../ci/jobs/job_artifacts.md#download-job-artifacts). ## Related topics -For more information, see the following blog posts. - -- Use GitLab CI/CD `environments` to - [deploy your web app to staging and production](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/). -- Learn how to run jobs - [sequentially, in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2020/12/10/basics-of-gitlab-ci-updated/). -- Learn [how to pull specific directories from different projects](https://about.gitlab.com/blog/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) - to deploy this website, <https://docs.gitlab.com>. -- Learn [how to use GitLab Pages to produce a code coverage report](https://about.gitlab.com/blog/2016/11/03/publish-code-coverage-report-with-gitlab-pages/). +- [Deploy your web app to staging and production](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/) +- [Run jobs sequentially, in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2020/12/10/basics-of-gitlab-ci-updated/) +- [Pull specific directories from different projects](https://about.gitlab.com/blog/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) +- [Use GitLab Pages to produce a code coverage report](https://about.gitlab.com/blog/2016/11/03/publish-code-coverage-report-with-gitlab-pages/) diff --git a/doc/user/project/pages/getting_started/pages_new_project_template.md b/doc/user/project/pages/getting_started/pages_new_project_template.md index 7b6efaa7af4..cb1da3fb21f 100644 --- a/doc/user/project/pages/getting_started/pages_new_project_template.md +++ b/doc/user/project/pages/getting_started/pages_new_project_template.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -32,4 +32,4 @@ For every change pushed to your repository, GitLab CI/CD runs a new pipeline that immediately publishes your changes to the Pages site. To view the HTML and other assets that were created for the site, -[download the job artifacts](../../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +[download the job artifacts](../../../../ci/jobs/job_artifacts.md#download-job-artifacts). diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index b286c59916a..6eb996db210 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 691757e3eca..f3b94bbf666 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -1,7 +1,7 @@ --- description: 'Learn how to use GitLab Pages to deploy a static website at no additional cost.' -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 0a8348f468e..05d0b461fea 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -271,7 +271,7 @@ This problem most likely results from a missing `index.html` file in the public a 404 is encountered, confirm that the public directory contains an `index.html` file. If the file contains a different name such as `test.html`, the Pages site can still be accessed, but the full path would be needed. For example: `https//group-name.pages.example.com/project-name/test.html`. -The contents of the public directory can be confirmed by [browsing the artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts) from the latest pipeline. +The contents of the public directory can be confirmed by [browsing the artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts) from the latest pipeline. Files listed under the public directory can be accessed through the Pages URL for the project. diff --git a/doc/user/project/pages/pages_access_control.md b/doc/user/project/pages/pages_access_control.md index 248a74a8abc..1b046d03f59 100644 --- a/doc/user/project/pages/pages_access_control.md +++ b/doc/user/project/pages/pages_access_control.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/pages/public_folder.md b/doc/user/project/pages/public_folder.md index 3007d1a10d1..22e1f242cf8 100644 --- a/doc/user/project/pages/public_folder.md +++ b/doc/user/project/pages/public_folder.md @@ -1,8 +1,8 @@ --- description: 'Learn how to configure the build output folder for the most common static site generators' -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -146,7 +146,7 @@ module.exports = { ## Should you commit the `public` folder? Not necessarily. However, when the GitLab Pages deploy pipeline runs, it looks -for an [artifact](../../../ci/pipelines/job_artifacts.md) of that name. +for an [artifact](../../../ci/jobs/job_artifacts.md) of that name. If you set up a job that creates the `public` folder before deploy, such as by running `npm run build`, committing the folder isn't required. diff --git a/doc/user/project/pages/redirects.md b/doc/user/project/pages/redirects.md index 75cb1474dc2..bd8206b3bda 100644 --- a/doc/user/project/pages/redirects.md +++ b/doc/user/project/pages/redirects.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 725818a7d25..fa736e79d93 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -79,7 +79,7 @@ Administrators can set a default branch protection level in the Configure protected branches for all projects in a group, or just for a project. -### For all projects in a group +### For all projects in a group **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106532) in GitLab 15.9 behind a feature flag, disabled by default. @@ -100,8 +100,8 @@ To protect a branch for all the projects in a group: 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. In the **Branch** text box, type the branch name or a wildcard. -1. From the **Allowed to merge** list, select a role, group, or user that can merge into this branch. -1. From the **Allowed to push** list, select a role, group, or user that can push to this branch. +1. From the **Allowed to merge** list, select a role that can merge into this branch. +1. From the **Allowed to push** list, select a role that can push to this branch. 1. Select **Protect**. The protected branch is added to the list of protected branches. @@ -270,7 +270,7 @@ Members who can push to this branch can now also force push. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in GitLab 13.5, users and groups who can push to protected branches do not have to use a merge request to merge their feature branches. This means they can skip merge request approval rules. -For a protected branch, you can require at least one approval by a [Code Owner](code_owners.md). +For a protected branch, you can require at least one approval by a [Code Owner](codeowners/index.md). To protect a new branch and enable Code Owner's approval: diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 3a00260770e..38b6bb44b48 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -49,113 +49,117 @@ The following quick actions are applicable to descriptions, discussions, and threads. Some quick actions might not be available to all subscription tiers. <!-- Keep this table sorted alphabetically --> - -| Command | Issue | Merge request | Epic | Action | -|:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `/add_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add one or more active [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). | -| `/approve` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Approve the merge request. | -| `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users. | -| `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself. | -| `/assign_reviewer @user1 @user2` or `/reviewer @user1 @user2` or `/request_review @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users as reviewers. | -| `/assign_reviewer me` or `/reviewer me` or `/request_review me` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself as a reviewer. | -| `/award :emoji:` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Toggle emoji award. | +<!-- Don't prettify this table; it will expand out the last field into illegibility --> + +| Command | Issue | Merge request | Epic | Action | +|:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:-------| +| `/add_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add one or more active [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). +| `/approve` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Approve the merge request. +| `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users. +| `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself. +| `/assign_reviewer @user1 @user2` or `/reviewer @user1 @user2` or `/request_review @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users as reviewers. +| `/assign_reviewer me` or `/reviewer me` or `/request_review me` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself as a reviewer. +| `/award :emoji:` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Toggle emoji award. | `/cc @user` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mention a user. In GitLab 15.0 and later, this command performs no action. You can instead type `CC @user` or only `@user`. [In GitLab 14.9 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/31200), mentioning a user at the start of a line created a specific type of to-do item notification. | -| `/child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Add child epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7330) in GitLab 12.0). | -| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear [health status](issues/managing_issues.md#health-status) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). | -| `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. | -| `/clone <path/to/project> [--with_notes]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clone the issue to given project, or the current one if no arguments are given ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9421) in GitLab 13.7). Copies as much data as possible as long as the target project contains equivalent labels, milestones, and so on. Does not copy comments or system notes unless `--with_notes` is provided as an argument. | -| `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. | +| `/child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Add child epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. +| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear [health status](issues/managing_issues.md#health-status) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). +| `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. +| `/clone <path/to/project> [--with_notes]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clone the issue to given project, or the current one if no arguments are given ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9421) in GitLab 13.7). Copies as much data as possible as long as the target project contains equivalent labels, milestones, and so on. Does not copy comments or system notes unless `--with_notes` is provided as an argument. +| `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. | `/confidential` | **{check-circle}** Yes | **{dotted-circle}** No | **{check-circle}** Yes | Mark issue or epic as confidential. Support for epics [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213741) in GitLab 15.6. | -| `/copy_metadata <!merge_request>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another merge request in the project. | -| `/copy_metadata <#issue>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another issue in the project. | -| `/create_merge_request <branch name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Create a new merge request starting from the current issue. | -| `/done` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mark to-do item as done. | +| `/copy_metadata <!merge_request>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another merge request in the project. +| `/copy_metadata <#issue>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another issue in the project. +| `/create_merge_request <branch name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Create a new merge request starting from the current issue. +| `/done` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mark to-do item as done. | `/draft` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set the [draft status](merge_requests/drafts.md). Use for toggling the draft status ([deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92654) in GitLab 15.4.) | -| `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. See [Chronic](https://gitlab.com/gitlab-org/ruby/gems/gitlab-chronic#examples) for more examples. | -| `/duplicate <#issue>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Close this issue. Mark as a duplicate of, and related to, issue `<#issue>`. | -| `/epic <epic>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. | -| `/estimate <time>` or `/estimate_time <time>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set time estimate. For example, `/estimate 1mo 2w 3d 4h 5m`. For more information, see [Time tracking](time_tracking.md). Alias `/estimate_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. | -| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). | -| `/invite_email email1 email2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add up to six email participants. This action is behind feature flag `issue_email_participants` and is not yet supported in issue templates. | -| `/iteration *iteration:"iteration name"` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set iteration. For example, to set the `Late in July` iteration: `/iteration *iteration:"Late in July"` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). | -| `/label ~label1 ~label2` or `/labels ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. | -| `/lock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Lock the discussions. | +| `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. See [Chronic](https://gitlab.com/gitlab-org/ruby/gems/gitlab-chronic#examples) for more examples. +| `/duplicate <#issue>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Close this issue. Mark as a duplicate of, and related to, issue `<#issue>`. +| `/epic <epic>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. +| `/estimate <time>` or `/estimate_time <time>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set time estimate. For example, `/estimate 1mo 2w 3d 4h 5m`. For more information, see [Time tracking](time_tracking.md). Alias `/estimate_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. +| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). +| `/invite_email email1 email2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add up to six email participants. This action is behind feature flag `issue_email_participants` and is not yet supported in issue templates. +| `/iteration *iteration:"iteration name"` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set iteration. For example, to set the `Late in July` iteration: `/iteration *iteration:"Late in July"` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). +| `/label ~label1 ~label2` or `/labels ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. +| `/lock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Lock the discussions. | `/link` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a link and description to [linked resources](../../operations/incident_management/linked_resources.md) in an incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/374964) in GitLab 15.5). | -| `/merge` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), or adding to a [Merge Train](../../ci/pipelines/merge_trains.md). | -| `/milestone %milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set milestone. | -| `/move <path/to/project>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Move this issue to another project. Be careful when moving an issue to a project with different access rules. Before moving the issue, make sure it does not contain sensitive data. | -| `/parent_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Set parent epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10556) in GitLab 12.1). | -| `/promote` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to epic. | -| `/promote_to_incident` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296787) in GitLab 14.5). In [GitLab 15.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/376760), you can also use the quick action when creating a new issue. | -| `/page <policy name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Start escalations for the incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79977) in GitLab 14.9). | -| `/publish` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Publish issue to an associated [Status Page](../../operations/incident_management/status_page.md) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30906) in GitLab 13.0) | -| `/ready` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set the [ready status](merge_requests/drafts.md#mark-merge-requests-as-ready) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90361) in GitLab 15.1). | -| `/reassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Replace current assignees with those specified. | -| `/reassign_reviewer @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Replace current reviewers with those specified. | +| `/merge` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), or adding to a [Merge Train](../../ci/pipelines/merge_trains.md). +| `/milestone %milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set milestone. +| `/move <path/to/project>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Move this issue to another project. Be careful when moving an issue to a project with different access rules. Before moving the issue, make sure it does not contain sensitive data. +| `/parent_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Set parent epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. +| `/promote` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to epic. +| `/promote_to_incident` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296787) in GitLab 14.5). In [GitLab 15.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/376760), you can also use the quick action when creating a new issue. +| `/page <policy name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Start escalations for the incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79977) in GitLab 14.9). +| `/publish` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Publish issue to an associated [Status Page](../../operations/incident_management/status_page.md) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30906) in GitLab 13.0) +| `/ready` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set the [ready status](merge_requests/drafts.md#mark-merge-requests-as-ready) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90361) in GitLab 15.1). +| `/reassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Replace current assignees with those specified. +| `/reassign_reviewer @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Replace current reviewers with those specified. | `/rebase` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Rebase source branch. This schedules a background task that attempts to rebase the changes in the source branch on the latest commit of the target branch. If `/rebase` is used, `/merge` is ignored to avoid a race condition where the source branch is merged or deleted before it is rebased. If there are merge conflicts, GitLab displays a message that a rebase cannot be scheduled. Rebase failures are displayed with the merge request status. | -| `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. | -| `/relate #issue1 #issue2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Mark issues as related. | -| `/remove_child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove child epic from `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7330) in GitLab 12.0). | -| `/remove_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove one or more [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). | -| `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove due date. | -| `/remove_epic` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove from epic. | -| `/remove_estimate` or `/remove_time_estimate` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time estimate. Alias `/remove_time_estimate` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. | -| `/remove_iteration` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove iteration ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). | -| `/remove_milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove milestone. | -| `/remove_parent_epic` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove parent epic from epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10556) in GitLab 12.1). | -| `/remove_time_spent` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time spent. | -| `/remove_zoom` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove Zoom meeting from this issue. | -| `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. | -| `/severity <severity>` | **{check-circle}** Yes | **{check-circle}** No | **{check-circle}** No | Set the severity. Issue type must be `Incident`. Options for `<severity>` are `S1` ... `S4`, `critical`, `high`, `medium`, `low`, `unknown`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334045) in GitLab 14.2. | -| `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\＿(ツ)＿/¯`. | -| `/spend <time> [<date>]` or `/spend_time <time> [<date>]` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Add or subtract spent time. Optionally, specify the date that time was spent on. For example, `/spend 1mo 2w 3d 4h 5m 2018-08-26` or `/spend -1h 30m`. For more information, see [Time tracking](time_tracking.md). Alias `/spend_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. | -| `/submit_review` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Submit a pending review ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8041) in GitLab 12.7). | -| `/subscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Subscribe to notifications. | -| `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. | -| `/target_branch <local branch name>` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set target branch. | -| `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. | +| `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. +| `/relate #issue1 #issue2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Mark issues as related. +| `/remove_child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove child epic from `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. +| `/remove_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove one or more [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). +| `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove due date. +| `/remove_epic` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove from epic. +| `/remove_estimate` or `/remove_time_estimate` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time estimate. Alias `/remove_time_estimate` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. +| `/remove_iteration` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove iteration ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). +| `/remove_milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove milestone. +| `/remove_parent_epic` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove parent epic from epic. +| `/remove_time_spent` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time spent. +| `/remove_zoom` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove Zoom meeting from this issue. +| `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. +| `/severity <severity>` | **{check-circle}** Yes | **{check-circle}** No | **{check-circle}** No | Set the severity. Issue type must be `Incident`. Options for `<severity>` are `S1` ... `S4`, `critical`, `high`, `medium`, `low`, `unknown`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334045) in GitLab 14.2. +| `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\＿(ツ)＿/¯`. +| `/spend <time> [<date>]` or `/spend_time <time> [<date>]` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Add or subtract spent time. Optionally, specify the date that time was spent on. For example, `/spend 1mo 2w 3d 4h 5m 2018-08-26` or `/spend -1h 30m`. For more information, see [Time tracking](time_tracking.md). Alias `/spend_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. +| `/submit_review` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Submit a pending review. +| `/subscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Subscribe to notifications. +| `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. +| `/target_branch <local branch name>` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set target branch. +| `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. | `/timeline <timeline comment> \| <date(YYYY-MM-DD)> <time(HH:MM)>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a timeline event to this incident. For example, `/timeline DB load spiked \| 2022-09-07 09:30`. ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368721) in GitLab 15.4). | -| `/todo` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add a to-do item. | -| `/unapprove` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Unapprove the merge request. ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8103) in GitLab 14.3 | -| `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove specific assignees. | -| `/unassign` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove all assignees. | +| `/todo` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add a to-do item. +| `/unapprove` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Unapprove the merge request. ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8103) in GitLab 14.3. +| `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove specific assignees. +| `/unassign` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove all assignees. | `/unassign_reviewer @user1 @user2` or `/remove_reviewer @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove specific reviewers. -| `/unassign_reviewer me` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove yourself as a reviewer. | -| `/unassign_reviewer` or `/remove_reviewer` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove all reviewers. | -| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove specified labels. | -| `/unlabel` or `/remove_label` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove all labels. | -| `/unlock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Unlock the discussions. | -| `/unsubscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Unsubscribe from notifications. | -| `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, `2`, and so on. | -| `/zoom <Zoom URL>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a Zoom meeting to this issue or incident. In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/230853) users on GitLab Premium can add a short description when [adding a Zoom link to an incident](../../operations/incident_management/linked_resources.md#link-zoom-meetings-from-an-incident).| +| `/unassign_reviewer me` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove yourself as a reviewer. +| `/unassign_reviewer` or `/remove_reviewer` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove all reviewers. +| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove specified labels. +| `/unlabel` or `/remove_label` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove all labels. +| `/unlock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Unlock the discussions. +| `/unsubscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Unsubscribe from notifications. +| `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, `2`, and so on. +| `/zoom <Zoom URL>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a Zoom meeting to this issue or incident. In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/230853) users on GitLab Premium can add a short description when [adding a Zoom link to an incident](../../operations/incident_management/linked_resources.md#link-zoom-meetings-from-an-incident). ## Work items -The following quick actions can be applied through the description field when editing work items. - -| Command | Task | Objective | Key Result | Action | -|:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. | -| `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. | -| `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. | -| `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\＿(ツ)＿/¯`. | -| `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. | -| `/cc @user` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mention a user. In GitLab 15.0 and later, this command performs no action. You can instead type `CC @user` or only `@user`. [In GitLab 14.9 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/31200), mentioning a user at the start of a line creates a specific type of to-do item notification. | -| `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Assign one or more users. | -| `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Assign yourself. | -| `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Remove specific assignees. | -| `/unassign` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** Yes | Remove all assignees. | -| `/reassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Replace current assignees with those specified. | -| `/label ~label1 ~label2` or `/labels ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. | -| `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. | -| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove specified labels. | -| `/unlabel` or `/remove_label` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove all labels. | -| `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** Yes | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. | -| `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** Yes | Remove due date. | -| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk`. | -| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Clear [health status](issues/managing_issues.md#health-status). | -| `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, and `2`. | -| `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. | +> Executing quick actions from comments [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/391282) in GitLab 15.10. + +Work items in GitLab include [tasks](../tasks.md) and [OKRs](../okrs.md). +The following quick actions can be applied through the description field when editing or commenting on work items. + +| Command | Task | Objective | Key Result | Action +|:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:-------| +| `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. +| `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. +| `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. +| `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\＿(ツ)＿/¯`. +| `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. +| `/cc @user` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mention a user. In GitLab 15.0 and later, this command performs no action. You can instead type `CC @user` or only `@user`. [In GitLab 14.9 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/31200), mentioning a user at the start of a line creates a specific type of to-do item notification. +| `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Assign one or more users. +| `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Assign yourself. +| `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Remove specific assignees. +| `/unassign` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** Yes | Remove all assignees. +| `/reassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Replace current assignees with those specified. +| `/label ~label1 ~label2` or `/labels ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. +| `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. +| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove specified labels. +| `/unlabel` or `/remove_label` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove all labels. +| `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** Yes | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. +| `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** Yes | Remove due date. +| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk`. +| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Clear [health status](issues/managing_issues.md#health-status). +| `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, and `2`. +| `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. ## Commit messages diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index be2c923adcc..f200b2b396c 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -80,10 +80,11 @@ To create a release in the Releases page: - Select an existing Git tag. Selecting an existing tag that is already associated with a release results in a validation error. - Enter a new Git tag name. - 1. From the **Create from** dropdown list, select a branch or commit SHA to use when + 1. From the **Create tag** popover, select a branch or commit SHA to use when creating the new tag. 1. Optional. In the **Set tag message** text box, enter a message to create an [annotated tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging#_annotated_tags). + 1. Select **Save**. 1. Optional. Enter additional information about the release, including: - [Title](release_fields.md#title). - [Milestones](#associate-milestones-with-a-release). diff --git a/doc/user/project/releases/release_cicd_examples.md b/doc/user/project/releases/release_cicd_examples.md index 36e36412cf1..7b5e808641c 100644 --- a/doc/user/project/releases/release_cicd_examples.md +++ b/doc/user/project/releases/release_cicd_examples.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/releases/release_cli.md b/doc/user/project/releases/release_cli.md index 953e1d4b082..bd3cfd26f1b 100644 --- a/doc/user/project/releases/release_cli.md +++ b/doc/user/project/releases/release_cli.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/releases/release_fields.md b/doc/user/project/releases/release_fields.md index 93822be89b6..3269c75d362 100644 --- a/doc/user/project/releases/release_fields.md +++ b/doc/user/project/releases/release_fields.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -245,7 +245,7 @@ release: ``` NOTE: -Directly attaching [job artifacts](../../../ci/pipelines/job_artifacts.md) +Directly attaching [job artifacts](../../../ci/jobs/job_artifacts.md) links to a release is not recommended, because artifacts are ephemeral and are used to pass data in the same pipeline. This means there's a risk that they could either expire or someone might manually delete them. diff --git a/doc/user/project/remote_development/connect_machine.md b/doc/user/project/remote_development/connect_machine.md new file mode 100644 index 00000000000..f8fdd626852 --- /dev/null +++ b/doc/user/project/remote_development/connect_machine.md @@ -0,0 +1,107 @@ +--- +stage: Create +group: IDE +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Tutorial: Connect a remote machine to the Web IDE **(FREE)** + +This tutorial shows you how to: + +- Create a development environment outside of GitLab. +- Connect a remote machine to the [Web IDE](../web_ide_beta/index.md). + +To connect a remote machine to the Web IDE, you must: + +1. [Generate Let's Encrypt certificates](#generate-lets-encrypt-certificates). +1. [Connect a development environment to the Web IDE](#connect-a-development-environment-to-the-web-ide). + +## Prerequisites + +- A remote virtual machine with root access +- A domain address resolving to that machine +- Docker installation + +## Generate Let's Encrypt certificates + +To generate Let's Encrypt certificates: + +1. Create an `A` record to point a domain to your remote machine (for example, from `example.remote.gitlab.dev` to `1.2.3.4`). +1. Install [Certbot](https://certbot.eff.org/) to enable HTTPS: + + ```shell + sudo apt-get update + sudo apt-get install certbot + ``` + +1. Generate the certificates: + + ```shell + export EMAIL="YOUR_EMAIL@example.com" + export DOMAIN="example.remote.gitlab.dev" + + certbot -d "${DOMAIN}" \ + -m "${EMAIL}" \ + --config-dir ~/.certbot/config \ + --logs-dir ~/.certbot/logs \ + --work-dir ~/.certbot/work \ + --manual \ + --preferred-challenges dns certonly + ``` + +## Connect a development environment to the Web IDE + +To connect a development environment to the Web IDE: + +1. Create a development environment: + + ```shell + export CERTS_DIR="/home/ubuntu/.certbot/config/live/${DOMAIN}" + export PROJECTS_DIR="/home/ubuntu" + + docker run -d \ + --name my-environment \ + -p 3443:3443 \ + -v "${CERTS_DIR}/fullchain.pem:/gitlab-rd-web-ide/certs/fullchain.pem" \ + -v "${CERTS_DIR}/privkey.pem:/gitlab-rd-web-ide/certs/privkey.pem" \ + -v "${PROJECTS_DIR}:/projects" \ + registry.gitlab.com/gitlab-org/remote-development/gitlab-rd-web-ide-docker:0.2-alpha \ + --log-level warn --domain "${DOMAIN}" --ignore-version-mismatch + ``` + + The new development environment starts automatically. + +1. Fetch a token: + + ```shell + docker exec my-environment cat TOKEN + ``` + +1. [Configure a remote connection](#configure-a-remote-connection). + +### Configure a remote connection + +To configure a remote connection from the Web IDE: + +1. Open the Web IDE. +1. On the menu bar, select **View > Terminal** or press <kbd>Control</kbd>+<kbd>`</kbd>. +1. In the terminal panel, select **Configure a remote connection**. +1. Enter the URL for the remote host including the port (for example, `yourdomain.com:3443`). +1. Enter the project path. +1. Enter the token you've fetched. + +Alternatively, you can pass the parameters from a URL and connect directly to the Web IDE: + +1. Run this command: + + ```shell + echo "https://gitlab-org.gitlab.io/gitlab-web-ide?remoteHost=${DOMAIN}:3443&hostPath=/projects" + ``` + +1. Go to that URL and enter the token you've fetched. + +You've done it! Your development environment now runs as a remote host that's connected to the Web IDE. + +## Related topics + +- [Manage a development environment](index.md#manage-a-development-environment) diff --git a/doc/user/project/remote_development/index.md b/doc/user/project/remote_development/index.md index 58d43f64812..e3c2b6ccfb2 100644 --- a/doc/user/project/remote_development/index.md +++ b/doc/user/project/remote_development/index.md @@ -1,18 +1,20 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Remote Development **(FREE)** +# Remote development **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.4 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.4 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/371084) in GitLab 15.7. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115741) in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The feature is not ready for production use. WARNING: -This feature is in [Alpha](../../../policy/alpha-beta-support.md#alpha-features) and subject to change without notice. +This feature is an [Experiment](../../../policy/alpha-beta-support.md#experiment) and subject to change without notice. DISCLAIMER: This page contains information related to upcoming products, features, and functionality. @@ -22,70 +24,40 @@ As with all projects, the items mentioned on this page are subject to change or The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc. -You can use the [Web IDE](../web_ide/index.md) to commit changes to a project directly from your web browser without installing any dependencies or cloning any repositories. The Web IDE, however, lacks a native runtime environment on which you would compile code, run tests, or generate real-time feedback in the IDE. For a more complete IDE experience, you can pair the [Web IDE Beta](../web_ide_beta/index.md) with a Remote Development environment that has been properly configured to run as a host. +## Web IDE as a frontend -## Connect a remote machine to the Web IDE +You can use the [Web IDE](../web_ide_beta/index.md) to make, commit, and push changes to a project directly from your web browser. +This way, you can update any project without having to install any dependencies or clone any repositories locally. -Prerequisites: +The Web IDE, however, lacks a native runtime environment where you could compile code, run tests, or generate real-time feedback. +With remote development, you can use: -- A remote virtual machine with root access -- A domain address resolving to that machine -- Docker installation +- The Web IDE as a frontend +- A separate machine as a backend runtime environment -To connect a remote machine to the Web IDE, you must: +For a complete IDE experience, connect the Web IDE to a [development environment](#workspace) that's configured to run as a remote host. +For more information, see [connect a remote machine to the Web IDE](connect_machine.md). -1. [Generate Let's Encrypt certificates](#generate-lets-encrypt-certificates). -1. [Connect a development environment to the Web IDE](#connect-a-development-environment-to-the-web-ide). +## Workspace -### Generate Let's Encrypt certificates +A workspace is a virtual sandbox environment for your code that includes: -To generate Let's Encrypt certificates: +- A runtime environment +- Dependencies +- Configuration files -1. [Point a domain to your remote machine](#point-a-domain-to-your-remote-machine). -1. [Install Certbot](#install-certbot). -1. [Generate the certificates](#generate-the-certificates). +You can create a workspace from scratch or from a template that you can also customize. -#### Point a domain to your remote machine +When you configure and connect a workspace to the [Web IDE](../web_ide_beta/index.md), you can: -To point a domain to your remote machine, create an `A` record from `example.remote.gitlab.dev` to `1.2.3.4`. +- Edit files directly from the Web IDE and commit and push changes to GitLab. +- Use the Web IDE to run tests, debug code, and view real-time feedback. -#### Install Certbot +## Manage a development environment -[Certbot](https://certbot.eff.org/) is a free and open-source software tool that automatically uses Let's Encrypt certificates on manually administered websites to enable HTTPS. +### Create a development environment -To install Certbot, run the following command: - -```shell -sudo apt-get update -sudo apt-get install certbot -``` - -#### Generate the certificates - -```shell -export EMAIL="YOUR_EMAIL@example.com" -export DOMAIN="example.remote.gitlab.dev" - -certbot -d "${DOMAIN}" \ - -m "${EMAIL}" \ - --config-dir ~/.certbot/config \ - --logs-dir ~/.certbot/logs \ - --work-dir ~/.certbot/work \ - --manual \ - --preferred-challenges dns certonly -``` - -### Connect a development environment to the Web IDE - -To connect a development environment to the Web IDE: - -1. [Create a development environment](#manage-a-development-environment). -1. [Fetch a token](#fetch-a-token). -1. [Configure a remote connection](#configure-a-remote-connection). - -#### Manage a development environment - -**Create a development environment** +To create a development environment, run this command: ```shell export CERTS_DIR="/home/ubuntu/.certbot/config/live/${DOMAIN}" @@ -103,58 +75,31 @@ docker run -d \ The new development environment starts automatically. -**Stop a development environment** +### Stop a development environment + +To stop a running development environment, run this command: ```shell docker container stop my-environment ``` -**Start a development environment** +### Start a development environment + +To start a stopped development environment, run this command: ```shell docker container start my-environment ``` -The token changes every time you restart the development environment. +The token changes every time you start the development environment. -**Remove a development environment** +### Remove a development environment To remove a development environment: -1. Stop the development environment. -1. Run the following command: +1. [Stop the development environment](#stop-a-development-environment). +1. Run this command: ```shell docker container rm my-environment ``` - -#### Fetch a token - -```shell -docker exec my-environment cat TOKEN -``` - -#### Configure a remote connection - -To configure a remote connection from the Web IDE: - -1. Open the Web IDE. -1. In the Menu Bar, select **View > Terminal** or press <kbd>Control</kbd>+<kbd>`</kbd>. -1. In the terminal panel, select **Configure a remote connection**. -1. Enter the URL for the remote host including the port (for example, `yourdomain.com:3443`). -1. Enter the project path. -1. Enter the [token you fetched](#fetch-a-token). - -Alternatively, you can pass the parameters from a URL and connect directly to the Web IDE: - -1. Run the following command: - - ```shell - echo "https://gitlab-org.gitlab.io/gitlab-web-ide?remoteHost=${DOMAIN}:3443&hostPath=/projects" - ``` - -1. Go to that URL and enter the [token you fetched](#fetch-a-token). - -## Related topics - -- [Web IDE Beta](../web_ide_beta/index.md) diff --git a/doc/user/project/repository/branches/img/view_branch_protections_v15_10.png b/doc/user/project/repository/branches/img/view_branch_protections_v15_10.png Binary files differindex 09b30af91d0..8472015c21b 100644 --- a/doc/user/project/repository/branches/img/view_branch_protections_v15_10.png +++ b/doc/user/project/repository/branches/img/view_branch_protections_v15_10.png diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index ef625739956..bb02e5876c9 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -25,6 +25,61 @@ Branches are the foundation of development in a project: 1. Preview changes in a branch with a [review app](../../../../ci/review_apps/index.md). 1. After the contents of your branch are merged, [delete the merged branch](#delete-merged-branches). +## Create branch + +To create a new branch from the GitLab UI: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Branches**. +1. On the top right, select **New branch**. +1. Enter a **Branch name**. +1. In **Create from**, select the base of your branch: an existing branch, an existing + tag, or a commit SHA. +1. Select **Create branch**. + +### In a blank project + +A [blank project](../../index.md#create-a-blank-project) does not contain a branch, but +you can add one. + +Prerequisites: + +- You must have at least the Developer role in the project. +- Unless you have the Maintainer or Owner roles, the + [default branch protection](../../../group/manage.md#change-the-default-branch-protection-of-a-group) + must be set to `Partially protected` or `Not protected` for you to push a commit + to the default branch. + +To add a [default branch](default.md) to an empty project: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. Scroll to **The repository for this project is empty** and select the type of + file you want to add. +1. In the Web IDE, make any desired changes to this file, then select **Create commit**. +1. Enter a commit message, and select **Commit**. + +GitLab creates a default branch and adds your file to it. + +### From an issue + +When viewing an issue, you can create an associated branch directly from that page. + +Prerequisites: + +- You must have the Developer or higher role in the project. + +To create a branch from an issue: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Issues** (**{issues}**) and find your issue. +1. Below the issue description, find the **Create merge request** dropdown list, and select + **{chevron-down}** to display the dropdown list. +1. Select **Create branch**. A default **Branch name** is provided, based on the + [default pattern](#configure-default-pattern-for-branch-names-from-issues) for + this project. If desired, enter a different **Branch name**. +1. Select **Create branch** to create the branch based on your project's + [default branch](default.md). + ## Manage and protect branches GitLab provides you multiple methods to protect individual branches. These methods @@ -63,9 +118,10 @@ On this page, you can: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88279) in GitLab 15.1 with a flag named `branch_rules`. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/363170) in GitLab 15.10. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/363170) in GitLab 15.11 FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../feature_flags.md) named `branch_rules`. +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../feature_flags.md) named `branch_rules`. On GitLab.com, this feature is available. Branches in your repository can be [protected](../../protected_branches.md) in multiple ways. You can: @@ -82,7 +138,7 @@ and their protection methods: Prerequisites: -- You must have at least the Developer role in the project. +- You must have at least the Maintainer role in the project. To view the **Branch rules overview** list: @@ -99,7 +155,35 @@ To view the **Branch rules overview** list: - [Approval rules](../../merge_requests/approvals/rules.md). - [Status checks](../../merge_requests/status_checks.md). -## Prefix branch names with issue numbers +## Name your branch + +If you follow GitLab standards for [naming branches](#prefix-branch-names-with-issue-numbers), +and configure branch names that follow these standards, GitLab can streamline your workflow: + +- Connect a merge request to its parent issue. +- Connect a branch to an issue. +- [Close the issue](../../issues/managing_issues.md#closing-issues-automatically) + when the connected merge request closes, and the connected branch merges. + +### Configure default pattern for branch names from issues + +By default, GitLab uses the pattern `%{id}-%{title}` when creating a branch from +an issue, but you can change this pattern. + +Prerequisites: + +- You must have at least the Maintainer role for the project. + +To change the default pattern for branches created from issues: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository** and expand **Branch defaults**. +1. Scroll to **Branch name template** and enter a value. The field supports these variables: + - `%{id}`: The numeric ID of the issue. + - `%{title}`: The title of the issue, modified to use only characters acceptable in Git branch names. +1. Select **Save changes**. + +### Prefix branch names with issue numbers To streamline the creation of merge requests, start your branch name with an issue number. GitLab uses the issue number to import data into the merge request: @@ -144,10 +228,10 @@ automatically when a merge request was merged. ## Related topics -- [Protected branches](../../protected_branches.md) user documentation. -- [Branches API](../../../../api/branches.md), for information on operating on repository branches using the GitLab API. -- [Protected Branches API](../../../../api/protected_branches.md). -- [Getting started with Git](../../../../topics/git/index.md) and GitLab. +- [Protected branches](../../protected_branches.md) +- [Branches API](../../../../api/branches.md) +- [Protected Branches API](../../../../api/protected_branches.md) +- [Getting started with Git](../../../../topics/git/index.md) ## Troubleshooting diff --git a/doc/user/project/repository/code_suggestions.md b/doc/user/project/repository/code_suggestions.md new file mode 100644 index 00000000000..68b1551eb16 --- /dev/null +++ b/doc/user/project/repository/code_suggestions.md @@ -0,0 +1,142 @@ +--- +stage: ModelOps +group: AI Assisted +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: index, reference +--- + +# Code Suggestions (Beta) **(ULTIMATE SAAS)** + +> - Enabled as opt-in with GitLab 15.11 as [Beta](/ee/policy/alpha-beta-support.md#beta). +> - [Introduced](https://about.gitlab.com/releases/2023/02/22/gitlab-15-9-released/#code-suggestions-available-in-closed-beta) in GitLab 15.9 as [Beta](/ee/policy/alpha-beta-support.md#beta) for early access Ultimate customers. + +WARNING: +This feature is in [Beta](/ee/policy/alpha-beta-support.md#beta). Code Suggestions use generative AI to suggest code while you're developing. Due to high demand, this feature will have unscheduled downtime and code suggestions in VS Code may be delayed. Code Suggestions may produce [low-quality or incomplete suggestions](#model-accuracy-and-quality). Beta users should read about the [known limitations](#known-limitations). We look forward to hearing your feedback. + +Use Code Suggestions to write code more efficiently by viewing code suggestions +as you type. Depending on the cursor position, the extension either: + +- Provides entire code snippets, like generating functions. +- Completes the current line. + +To accept a suggestion, press <kbd>Tab</kbd>. + +Code Suggestions are supported in Visual Studio Code with the GitLab Workflow extension. GitLab plans to support the [new GitLab WebIDE in an upcoming release](../web_ide_beta/index.md) in the future. + +Code Suggestions may produce [low-quality or incomplete suggestions](#model-accuracy-and-quality). Beta users should read about the [known limitations](#known-limitations). The best results from Code Suggestions are expected for these six languages: + +- C +- C++ +- Go +- Java +- JavaScript +- Python + +Suggestions may be mixed for other languages. Using natural language code comments to request completions may also not function as expected. + +GitLab is continuously improving the model and expects to support an additional seven languages soon, as well as natural language code comments. + +Usage of Code Suggestions is governed by the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). Learn about [data usage when using Code Suggestions](#code-suggestions-data-usage). + +## Group level setting + +[Group owners](../../permissions.md#group-members-permissions) can enable Code Suggestions for all projects in a group by using the [group level Code Suggestions setting](../../group/manage.md#group-code-suggestions). + +## Enable Code Suggestions in VS Code + +Prerequisites: + +- Your group owner must enable the [group level code suggestions setting](../../group/manage.md#group-code-suggestions). +- You must have [a personal access token](../../profile/personal_access_tokens.md#create-a-personal-access-token) with the `read_api` and `read_user` scopes. + +To enable Code Suggestions in VS Code: + +1. Download and configure the + [GitLab Workflow extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) + for Visual Studio Code. +1. In **GitLab: Add Account to VS Code on Mac**, add your GitLab work account to the VS Code extension: + - In macOS, press <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>P</kbd>. + - In Windows, press <kbd>Shift</kbd> + <kbd>Control</kbd> + <kbd>P</kbd>. +1. Provide your GitLab instance URL. A default is provided. +1. Provide your personal access token. +1. After your GitLab account connects successfully, in the left sidebar, select **Extensions**. +1. Find the **GitLab workflow** extension, select **Settings** (**{settings}**), and select **Extension Settings**. +1. Enable **GitLab > AI Assisted Code Suggestions**. + +Start typing and receive suggestions for your GitLab projects. + +<div class="video-fallback"> + See an end-to-end demo: <a href="https://www.youtube.com/watch?v=WnxBYxN2-p4">Get started with GitLab Code Suggestions in VS Code</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/WnxBYxN2-p4" frameborder="0" allowfullscreen> </iframe> +</figure> + +## Code Suggestions Data Usage + +### Overview + +Code Suggestions is a generative artificial intelligence (AI) model hosted on GitLab.com that can empower your developers to code more efficiently by suggesting code as they type. + +The personal access token enables a secure API connection to GitLab.com. This API connection securely transmits a context window from VS Code to the Code Suggestions ML model for inference, and the generated suggestion is transmitted back to VS Code. + +#### Progressive enhancement + +This feature is designed as a progressive enhancement to the existing VS Code GitLab Workflow plugin. Code Suggestions offer a completion if the machine learning engine can generate a recommendation. In the event of a connection issue or model inference failure, the feature gracefully degrades. Code Suggestions do not prevent you from writing code in VS Code. + +#### Off by default + +Code Suggestions are off by default and require a group owner to enable the feature with a [group-level setting](#group-level-setting). + +After the group level setting is enabled, Developers using Visual Studio Code with the [GitLab Workflow extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) can connect to GitLab.com via a GitLab [personal access token](../../profile/personal_access_tokens.md#create-a-personal-access-token) with the `read_api` and `read_user` scopes. + +#### Generating suggestions + +Once configured by a developer in VS Code. The personal access token enables a secure API connection to GitLab.com. This API connection securely transmits a context window from VS Code to the Code Suggestions ML model for inference, and the generated suggestion is transmitted back to VS Code. + +Code Suggestions only work when you have internet connectivity and can access GitLab.com. Code Suggestions are not available for self-managed customers, nor customers operating within an air-gapped environment. + +### Stability and performance + +This feature is currently in [Beta](/ee/policy/alpha-beta-support.md#beta). While the Code Suggestions inference API operates completely within GitLab.com's enterprise infrastructure, we expect a high demand for this Beta feature, which may cause degraded performance or unexpected downtime of the feature. We have built this feature to gracefully degrade and have controls in place to allow us to mitigate abuse or misuse. GitLab may disable this feature for any or all customers at any time at our discretion. + +## Data privacy + +Code Suggestions operate completely in the GitLab.com infrastructure, providing the same level of [security](https://about.gitlab.com/security/) as any other feature of GitLab.com, and processing any personal data in accordance with our [Privacy Statement](https://about.gitlab.com/privacy/). + +No new additional data is collected to enable this feature. The content of your GitLab hosted source code is not used as training data. Source code inference against the Code Suggestions model is not used to re-train the model. Your data also never leaves GitLab.com. All training and inference is done in GitLab.com infrastructure. + +[Read more about the security of GitLab.com](https://about.gitlab.com/security/faq/). + +### Model accuracy and quality + +While in Beta, Code Suggestions can generate low-quality, incomplete, and possibly insecure code. We strongly encourage all beta users to leverage GitLab native [Code Quality Scanning](../../../ci/testing/code_quality.md) and [Security Scanning](../../application_security/index.md) capabilities. + +GitLab uses a customized open source dataset to fine-tune the model to support multiple languages. Based on the languages you code in, GitLab routes the request to a targeted inference and prompt engine to get relevant suggestions. + +GitLab is actively refining these models to improve the quality of recommendations, add support for more languages, and add protections to limit personal data, insecure code, and other unwanted behavior that the model may have learned from training data. + +### Training data + +Code Suggestions uses open source pre-trained base models from the [CodeGen family](https://openreview.net/forum?id=iaYcJKpY2B_) including CodeGen-MULTI and CodeGen-NL. We then re-train and fine-tune these base models with a customized open source dataset to enable multi-language support and additional use cases. This customized dataset contains non-preprocessed open source code in 13 programming languages from [The Pile](https://pile.eleuther.ai/) and [Google's BigQuery source code dataset](https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code). We then process this raw dataset against heuristics that aim to increase the quality of the dataset. + +The Code Suggestions model is not trained on GitLab customer data. + +## Known limitations + +While in Beta, we are working on improving the accuracy of overall generated content. However, Code Suggestions may generate suggestions that are: + +- Low-quality +- Incomplete +- Produce failed pipelines +- Insecure code +- Offensive or insensitive + +We are also aware of specific situations that can produce unexpected or incoherent results including: + +- Suggestions based on natural language code comments. +- Suggestions that mixed programming languages in unexpected ways. + +## Feedback + +Report issues in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/405152). diff --git a/doc/user/project/repository/file_finder.md b/doc/user/project/repository/file_finder.md index 3fcc19db344..e22dc549a4a 100644 --- a/doc/user/project/repository/file_finder.md +++ b/doc/user/project/repository/file_finder.md @@ -1,42 +1,28 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/file_finder.html' --- # File finder **(FREE)** -The file finder feature allows you to search for a file in a repository using the -GitLab UI. To use it: +With file finder, you can search for a file in a repository directly from the GitLab UI. -1. Go to your project's **Repository > Files**. -1. In the upper-right corner, select **Find File**. +File finder is powered by the [`fuzzaldrin-plus`](https://github.com/jeancroy/fuzz-aldrin-plus) library, which uses fuzzy search and highlights results as you type. -If you prefer to keep your fingers on the keyboard, use the -[shortcut button](../../shortcuts.md), which you can invoke from anywhere -in a project. +## Search for a file -Press `t` to launch the File search function when in **Issues**, -**Merge requests**, **Milestones**, even the project's settings. +To search for a file: -Start typing what you are searching for and watch the magic happen. With the -up/down arrows, you go up and down the results, with `Esc` you close the search -and go back to **Files** +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Files**. +1. In the upper right, select **Find file**. +1. In the search box, start typing the filename. +1. From the dropdown list, select the file. -## How it works +To go to file finder, you can also press <kbd>t</kbd> anywhere in your project. +To go back to **Files**, press <kbd>Esc</kbd>. -The File finder feature is powered by the [Fuzzy filter](https://github.com/jeancroy/fuzz-aldrin-plus) library. - -It implements a fuzzy search with the highlight and tries to provide intuitive -results by recognizing patterns that people use while searching. - -For example, consider the [GitLab FOSS repository](https://gitlab.com/gitlab-org/gitlab-foss/tree/master) and that we want to open -the `app/controllers/admin/deploy_keys_controller.rb` file. - -Using a fuzzy search, we start by typing letters that get us closer to the file. - -NOTE: -To narrow down your search, include `/` in your search terms. +To narrow down your results, include `/` in your search.  diff --git a/doc/user/project/repository/forking_workflow.md b/doc/user/project/repository/forking_workflow.md index 8c7c94613a7..423bdd178c6 100644 --- a/doc/user/project/repository/forking_workflow.md +++ b/doc/user/project/repository/forking_workflow.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/forking_workflow.html' --- # Project forking workflow **(FREE)** @@ -16,7 +15,7 @@ A fork is a personal copy of the repository and all its branches, which you crea in a namespace of your choice. Make changes in your own fork and submit them through a merge request to the repository you don't have access to. -## Creating a fork +## Create a fork > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15013) a new form in GitLab 13.11 [with a flag](../../../user/feature_flags.md) named `fork_project_form`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77181) in GitLab 14.8. Feature flag `fork_project_form` removed. @@ -40,7 +39,7 @@ GitLab creates your fork, and redirects you to the new fork's page. ## Update your fork To copy the latest changes from the upstream repository into your fork, update it -[from the command line](#from-the-command-line). GitLab Premium and higher tiers can also +from the command line. GitLab Premium and higher tiers can also [configure forks as pull mirrors](#with-repository-mirroring) of the upstream repository. @@ -102,7 +101,7 @@ an `upstream` remote repository for your fork: A fork can be configured as a mirror of the upstream if all these conditions are met: -1. Your subscription is **(PREMIUM)** or a higher tier. +1. Your subscription is **Premium** or higher. 1. You create all changes in branches (not `main`). 1. You do not work on [merge requests for confidential issues](../merge_requests/confidential.md), which requires changes to `main`. @@ -114,7 +113,7 @@ For instructions, read [Configure pull mirroring](mirror/pull.md#configure-pull- WARNING: With mirroring, before approving a merge request, you are asked to sync. You should automate it. -## Merging upstream +## Merge changes back upstream When you are ready to send your code back to the upstream project, [create a merge request](../merge_requests/creating_merge_requests.md). For **Source branch**, @@ -129,19 +128,55 @@ Then you can add labels, a milestone, and assign the merge request to someone wh your changes. Then select **Submit merge request** to conclude the process. When successfully merged, your changes are added to the repository and branch you're merging into. -## Removing a fork relationship +## Unlink a fork -You can unlink your fork from its upstream project in the [advanced settings](../settings/index.md#remove-a-fork-relationship). +Removing a fork relationship unlinks your fork from its upstream project. +Your fork then becomes an independent project. + +Prerequisites: + +- You must be a project owner to unlink a fork. + +WARNING: +If you remove a fork relationship, you can't send merge requests to the source. +If anyone has forked your project, their fork also loses the relationship. +To restore the fork relationship, [use the API](../../../api/projects.md#create-a-forked-fromto-relation-between-existing-projects). + +To remove a fork relationship: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Advanced**. +1. In the **Remove fork relationship** section, select **Remove fork relationship**. +1. To confirm, enter the project path and select **Confirm**. + +When you unlink a fork that uses a [hashed storage pool](../../../administration/repository_storage_types.md#hashed-object-pools) +to share objects with another repository: + +- All objects are copied from the pool into your fork. +- After the copy process completes, no further updates from the storage pool are propagated to your fork. ## Related topics -- GitLab blog post: [How to keep your fork up to date with its origin](https://about.gitlab.com/blog/2016/12/01/how-to-keep-your-fork-up-to-date-with-its-origin/). -- GitLab community forum: [Refreshing a fork](https://forum.gitlab.com/t/refreshing-a-fork/). +- GitLab blog post: [Keep your fork up to date with its origin](https://about.gitlab.com/blog/2016/12/01/how-to-keep-your-fork-up-to-date-with-its-origin/) +- GitLab community forum: [Refreshing a fork](https://forum.gitlab.com/t/refreshing-a-fork/) ## Troubleshooting -### An error occurred while forking the project. Please try again +### Error: `An error occurred while forking the project. Please try again` This error can be due to a mismatch in shared runner settings between the forked project and the new namespace. See [Forks](../../../ci/runners/configure_runners.md#forks) in the Runner documentation for more information. + +### Removing fork relationship fails + +If removing the fork through the UI or API is not working, you can attempt the +fork relationship removal in a +[Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session): + +```ruby +p = Project.find_by_full_path('<project_path>') +u = User.find_by_username('<username>') +Projects::UnlinkForkService.new(p, u).execute +``` diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 1c64a47985b..9f0c46591b9 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -277,15 +277,15 @@ to fetch configuration from a project that is renamed or moved. ## Related topics -- [GitLab Workflow VS Code extension](vscode.md). -- To lock files and prevent change conflicts, use [file locking](../file_lock.md). -- [Repository API](../../../api/repositories.md). -- [Find files](file_finder.md) in a repository. -- [Branches](branches/index.md). -- [Create a directory](web_editor.md#create-a-directory). -- [Find file history](git_history.md). -- [Identify changes by line (Git blame)](git_blame.md). -- [Use Jupyter notebooks with GitLab](jupyter_notebooks/index.md). +- [GitLab Workflow VS Code extension](vscode.md) +- [Lock files and prevent change conflicts](../file_lock.md) +- [Repository API](../../../api/repositories.md) +- [Find files](file_finder.md) +- [Branches](branches/index.md) +- [Create a directory](web_editor.md#create-a-directory) +- [Find file history](git_history.md) +- [Identify changes by line (Git blame)](git_blame.md) +- [Use Jupyter notebooks with GitLab](jupyter_notebooks/index.md) ## Troubleshooting diff --git a/doc/user/project/repository/jupyter_notebooks/index.md b/doc/user/project/repository/jupyter_notebooks/index.md index 28f0ffb6fbd..1526c7b4dc2 100644 --- a/doc/user/project/repository/jupyter_notebooks/index.md +++ b/doc/user/project/repository/jupyter_notebooks/index.md @@ -25,7 +25,7 @@ GitLab. ## Cleaner diffs and raw diffs -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6589) in GitLab 14.5 as an [Alpha](../../../../policy/alpha-beta-support.md#alpha-features) release [with a flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. Enabled by default. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6589) in GitLab 14.5 as an [Experiment](../../../../policy/alpha-beta-support.md#experiment) release [with a flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. Enabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75500) in GitLab 14.9. Feature flag `jupyter_clean_diffs` removed. > - [Reintroduced toggle](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85079) in GitLab 15.0 [with a flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`. Enabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95373) in GitLab 15.6. Feature flag `ipynb_semantic_diff` removed. diff --git a/doc/user/project/repository/mirror/bidirectional.md b/doc/user/project/repository/mirror/bidirectional.md index 9942469dd5c..0563f747e8d 100644 --- a/doc/user/project/repository/mirror/bidirectional.md +++ b/doc/user/project/repository/mirror/bidirectional.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html' --- # Bidirectional mirroring **(PREMIUM)** @@ -168,4 +167,4 @@ Read about [Git Fusion settings on Perforce.com](https://www.perforce.com/manual ## Related topics -- [Configure server hooks](../../../../administration/server_hooks.md) on a GitLab server. +- [Configure server hooks](../../../../administration/server_hooks.md) diff --git a/doc/user/project/repository/mirror/index.md b/doc/user/project/repository/mirror/index.md index 7a50c294dfc..61aa2ae3300 100644 --- a/doc/user/project/repository/mirror/index.md +++ b/doc/user/project/repository/mirror/index.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html' --- # Repository mirroring **(FREE)** @@ -331,6 +330,29 @@ and requires the full version including the protocol (`ssh://git@gitlab.com/gitl Make sure that host and project path are separated using `/` instead of `:`. +### Host key verification failed + +This error is returned when the target host public SSH key changes. +Public SSH keys rarely, if ever, change. If host key verification fails, +but you suspect the key is still valid, you can refresh the key's information. + +Prerequisites: + +- You must have at least the Maintainer role for a project. + +To resolve the issue: + +1. [Verify the host key](#verify-a-host-key). +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Mirroring repositories**. +1. To refresh the keys, either: + + - Select **Detect host keys** for GitLab to fetch the host keys from the server, and display the fingerprints. + - Select **Input host keys manually**, and enter the host key into the **SSH host key** field. + +- Select **Mirror repository**. + ### Transfer mirror users and tokens to a single service account in Rails console This requires access to the [GitLab Rails console](../../../../administration/operations/rails_console.md#starting-a-rails-console-session). diff --git a/doc/user/project/repository/mirror/pull.md b/doc/user/project/repository/mirror/pull.md index 0eb51f40208..2b8470b9e3d 100644 --- a/doc/user/project/repository/mirror/pull.md +++ b/doc/user/project/repository/mirror/pull.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html' --- # Pull from a remote repository **(PREMIUM)** @@ -142,6 +141,5 @@ end ## Related topics -- Configure [pull mirroring intervals](../../../../administration/instance_limits.md#pull-mirroring-interval) - on self-managed instances. -- Configure [pull mirroring through the API](../../../../api/projects.md#configure-pull-mirroring-for-a-project). +- [Pull mirroring intervals](../../../../administration/instance_limits.md#pull-mirroring-interval) +- [Pull mirroring API](../../../../api/projects.md#configure-pull-mirroring-for-a-project) diff --git a/doc/user/project/repository/mirror/push.md b/doc/user/project/repository/mirror/push.md index ea58c472f4a..11093958650 100644 --- a/doc/user/project/repository/mirror/push.md +++ b/doc/user/project/repository/mirror/push.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html' --- # Push mirroring **(FREE)** @@ -196,4 +195,4 @@ If it isn't working correctly, a red `error` tag appears, and shows the error me ## Related topics -- [Remote mirrors API](../../../../api/remote_mirrors.md). +- [Remote mirrors API](../../../../api/remote_mirrors.md) diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index cf7d4f44aad..4bebe4c1a97 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -22,16 +22,27 @@ Rewriting repository history is a destructive operation. Make sure to back up yo you begin. The best way to back up a repository is to [export the project](../settings/import_export.md#export-a-project-and-its-data). +## Calculate repository size + +The size of a repository is determined by computing the accumulated size of all files in the repository. +It is similar to executing `du --summarize --bytes` on your repository's +[hashed storage path](../../../administration/repository_storage_types.md). + ## Purge files from repository history -To reduce the size of your repository in GitLab, you must first remove references to large files from branches, tags, *and* -other internal references (refs) that are automatically created by GitLab. These refs include: +GitLab [prunes unreachable objects](../../../administration/housekeeping.md#prune-unreachable-objects) +as part of housekeeping. In GitLab, to reduce the disk size of your repository manually, you must +first remove references to large files from branches, tags, *and* other internal references (refs) +created by GitLab. These refs include: -- `refs/merge-requests/*` for merge requests. -- `refs/pipelines/*` for - [pipelines](../../../ci/troubleshooting.md#fatal-reference-is-not-a-tree-error). -- `refs/environments/*` for environments. -- `refs/keep-around/*` are created as hidden refs to prevent commits referenced in the database from being removed +- `refs/merge-requests/*` +- `refs/pipelines/*` +- `refs/environments/*` +- `refs/keep-around/*` + +NOTE: +For details on each of these references, see +[GitLab-specific references](../../../development/gitaly.md#gitlab-specific-references). These refs are not automatically downloaded and hidden refs are not advertised, but we can remove these refs using a project export. diff --git a/doc/user/project/repository/tags/index.md b/doc/user/project/repository/tags/index.md index 706b50a916d..e252a9a433b 100644 --- a/doc/user/project/repository/tags/index.md +++ b/doc/user/project/repository/tags/index.md @@ -101,8 +101,20 @@ To create a tag from the GitLab UI: To prevent users from removing a tag with `git push`, create a [push rule](../push_rules.md). +## Trigger pipelines from a tag + +GitLab CI/CD provides a [`CI_COMMIT_TAG` variable](../../../../ci/variables/predefined_variables.md) +to identify tags. Use this variable in job rules and workflow rules to test if the pipeline +was triggered by a tag. + +In your `.gitlab-ci.yml` file for the CI/CD pipeline configuration of your project, +you can trigger based on a new tag: + +- At the job level, with the [`only` keyword](../../../../ci/yaml/index.md#only--except). +- At the pipeline level, with the [workflow rules keywords](../../../../ci/yaml/workflow.md). + ## Related topics -- [Tagging](https://git-scm.com/book/en/v2/Git-Basics-Tagging) Git reference page. -- [Protected tags](../../protected_tags.md). -- [Tags API](../../../../api/tags.md). +- [Tagging (Git reference page)](https://git-scm.com/book/en/v2/Git-Basics-Tagging) +- [Protected tags](../../protected_tags.md) +- [Tags API](../../../../api/tags.md) diff --git a/doc/user/project/repository/vscode.md b/doc/user/project/repository/vscode.md index 9260d59bc3a..2a33476b545 100644 --- a/doc/user/project/repository/vscode.md +++ b/doc/user/project/repository/vscode.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -22,6 +22,7 @@ do more day-to-day tasks in Visual Studio Code, such as: and paste snippets to, and from, your editor. - [Browse repositories](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow#browse-a-repository-without-cloning) without cloning them. +- [Receive Code Suggestions](code_suggestions.md) ## Download the extension @@ -34,6 +35,7 @@ you can [configure](https://marketplace.visualstudio.com/items?itemName=GitLab.g - [Features to display or hide](https://gitlab.com/gitlab-org/gitlab-vscode-extension#extension-settings). - [Self-signed certificate](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow#self-signed-certificates) information. +- [Code Suggestions](code_suggestions.md) ## Report issues with the extension @@ -44,5 +46,4 @@ Report any issues, bugs, or feature requests in the - [Download the extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) - [Extension documentation](https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/blob/main/README.md) -- The extension source code is available in the - [`gitlab-vscode-extension`](https://gitlab.com/gitlab-org/gitlab-vscode-extension/) project. +- [View source code](https://gitlab.com/gitlab-org/gitlab-vscode-extension/) diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md index ace7e119469..dc988846676 100644 --- a/doc/user/project/repository/web_editor.md +++ b/doc/user/project/repository/web_editor.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/repository/x509_signed_commits/index.md b/doc/user/project/repository/x509_signed_commits/index.md index 42f7be30822..4d0ffd6a70a 100644 --- a/doc/user/project/repository/x509_signed_commits/index.md +++ b/doc/user/project/repository/x509_signed_commits/index.md @@ -101,7 +101,7 @@ To configure Windows or macOS: - Downloading the installer. - Running `brew install smimesign` on macOS. 1. Get the ID of your certificate by running `smimesign --list-keys`. -1. Set your signing key by running `git config --global user.signingkey ID`. +1. Set your signing key by running `git config --global user.signingkey <ID>`, replacing `<ID>` with the certificate ID. 1. Configure X.509 with this command: ```shell diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md index d385daa4fa1..b623be20b3a 100644 --- a/doc/user/project/requirements/index.md +++ b/doc/user/project/requirements/index.md @@ -24,6 +24,12 @@ If an industry standard *requires* that your application has a certain feature o [create a requirement](#create-a-requirement) to reflect this. When a feature is no longer necessary, you can [archive the related requirement](#archive-a-requirement). +NOTE: +Requirements and [test cases](../../../ci/test_cases/index.md) are being +[migrated to work items](https://gitlab.com/groups/gitlab-org/-/epics/5171). +[Issue 323790](https://gitlab.com/gitlab-org/gitlab/-/issues/323790) proposes to link requirements to test cases. +For more information, see [Product Stage Direction - Plan](https://about.gitlab.com/direction/plan/). + <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [GitLab 12.10 Introduces Requirements Management](https://www.youtube.com/watch?v=uSS7oUNSEoU). diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index c4fcdc5733e..1dd83949716 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -41,44 +41,45 @@ Meanwhile: - Your team saves time by not having to leave GitLab (or set up integrations) to follow up with your customer. -## Configuring Service Desk +## Configure Service Desk -Users with Maintainer and higher access in a project can configure Service Desk. +To start using Service Desk for a project, you must first turn it on. +By default, Service Desk is turned off. -Service Desk issues are [confidential](issues/confidential_issues.md), so they are -only visible to project members. +Prerequisites: -If you have [templates](description_templates.md) in your repository, you can optionally select -one from the selector menu to append it to all Service Desk issues. +- You must have at least the Maintainer role for the project. +- On GitLab self-managed, you must [set up incoming email](../../administration/incoming_email.md#set-it-up) + for the GitLab instance. You should use + [email sub-addressing](../../administration/incoming_email.md#email-sub-addressing), + but you can also use [catch-all mailboxes](../../administration/incoming_email.md#catch-all-mailbox). + To do this, you must have administrator access. To enable Service Desk in your project: -1. (GitLab self-managed only) [Set up incoming email](../../administration/incoming_email.md#set-it-up) for the GitLab instance. - You should use [email sub-addressing](../../administration/incoming_email.md#email-sub-addressing), - but you can also use [catch-all mailboxes](../../administration/incoming_email.md#catch-all-mailbox). -1. In a project, in the left sidebar, go to **Settings > General** and expand the **Service Desk** section. -1. Enable the **Activate Service Desk** toggle. This reveals a unique email address to email issues - to the project. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Service Desk**. +1. Turn on the **Activate Service Desk** toggle. +1. Optional. Complete the fields. + - [Add a suffix](#configure-a-custom-email-address-suffix) to your Service Desk email address. + - If the list below **Template to append to all Service Desk issues** is empty, create a + [description template](description_templates.md) in your repository. +1. Select **Save changes**. -Service Desk is now enabled for this project! To access it in a project, in the left sidebar, select -**Issues > Service Desk**. +Service Desk is now enabled for this project. +If anyone sends an email to the address available below **Email address to use for Support Desk**, +GitLab creates a confidential issue with the email's content. -WARNING: -Anyone in your project can use the Service Desk email address to create an issue in this project, **regardless -of their access level** to your GitLab instance. +### Improve your project's security -To improve your project's security, you should: +To improve your Service Desk project's security, you should: - Put the Service Desk email address behind an alias on your email system so you can change it later. - [Enable Akismet](../../integration/akismet.md) on your GitLab instance to add spam checking to this service. Unblocked email spam can result in many spam issues being created. -The unique internal email address is visible to project members at least -the Reporter role in your GitLab instance. -An external user (issue creator) cannot see the internal email address -displayed in the information note. - -### Using customized email templates +### Create customized email templates > - Moved from GitLab Premium to GitLab Free in 13.2. > - `UNSUBSCRIBE_URL`, `SYSTEM_HEADER`, `SYSTEM_FOOTER`, and `ADDITIONAL_TEXT` placeholders [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285512) in GitLab 15.9. @@ -89,14 +90,6 @@ An email is sent to the author when: - A new note is created on a Service Desk issue. You can customize the body of these email messages with templates. -Save your templates in the `.gitlab/service_desk_templates/` -directory in your repository. - -With Service Desk, you can use templates for: - -- [Thank you emails](#thank-you-email) -- [New note emails](#new-note-email) -- [New Service Desk issues](#new-service-desk-issues) #### Email header and footer **(FREE SELF)** @@ -109,7 +102,9 @@ visible in the email template. For more information, see #### Thank you email When a user submits an issue through Service Desk, GitLab sends a **thank you email**. -You must name the template file `thank_you.md`. + +To create a custom email template, in the `.gitlab/service_desk_templates/` +directory in your repository, create a file named `thank_you.md`. You can use these placeholders to be automatically replaced in each email: @@ -126,7 +121,9 @@ the response email does not contain the issue link. #### New note email When a user-submitted issue receives a new comment, GitLab sends a **new note email**. -You must name the template file `new_note.md`. + +To create a custom email template, in the `.gitlab/service_desk_templates/` +directory in your repository, create a file named `new_note.md`. You can use these placeholders to be automatically replaced in each email: @@ -138,7 +135,7 @@ You can use these placeholders to be automatically replaced in each email: - `%{SYSTEM_FOOTER}`: [system footer message](../admin_area/appearance.md#system-header-and-footer-messages) - `%{ADDITIONAL_TEXT}`: [custom additional text](../admin_area/settings/email.md#custom-additional-text) -#### New Service Desk issues +### Use a custom template for Service Desk issues You can select one [description template](description_templates.md#create-an-issue-template) **per project** to be appended to every new Service Desk issue's description. @@ -151,44 +148,57 @@ You can set description templates at various levels: The templates are inherited. For example, in a project, you can also access templates set for the instance, or the project's parent groups. +Prerequisite: + +- You must have [created a description template](description_templates.md#create-an-issue-template). + To use a custom description template with Service Desk: 1. On the top bar, select **Main menu > Projects** and find your project. -1. [Create a description template](description_templates.md#create-an-issue-template). -1. On the left sidebar, select **Settings > General > Service Desk**. +1. On the left sidebar, select **Settings > General**. +1. Expand **Service Desk**. 1. From the dropdown list **Template to append to all Service Desk issues**, search or select your template. -### Using a custom email display name +### Support Bot user + +Behind the scenes, Service Desk works by the special Support Bot user creating issues. +This user isn't a [billable user](../../subscriptions/self_managed/index.md#billable-users), +so it does not count toward the license limit count. + +#### Change the Support Bot's display name -You can customize the email display name. Emails sent from Service Desk have +You can change the display name of the Support Bot user. Emails sent from Service Desk have this name in the `From` header. The default display name is `GitLab Support Bot`. To edit the custom email display name: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General > Service Desk**. -1. Enter a new name in **Email display name**. -1. Select **Save Changes**. +1. On the left sidebar, select **Settings > General**. +1. Expand **Service Desk**. +1. Below **Email display name**, enter a new name. +1. Select **Save changes**. -### Using a custom email address **(FREE SELF)** +### Use a custom email address **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2201) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/284656) in GitLab 13.8. -It is possible to customize the email address used by Service Desk. To do this, you must configure -a [custom mailbox](#configuring-a-custom-mailbox). If you want you can also configure a -[custom suffix](#configuring-a-custom-email-address-suffix). +You can use a custom email address with Service Desk. -#### Configuring a custom mailbox +To do this, you must configure +a [custom mailbox](#configure-a-custom-mailbox). You can also configure a +[custom suffix](#configure-a-custom-email-address-suffix). + +#### Configure a custom mailbox NOTE: On GitLab.com a custom mailbox is already configured with `contact-project+%{key}@incoming.gitlab.com` as the email address, you can still configure the -[custom suffix](#configuring-a-custom-email-address-suffix) in project settings. +[custom suffix](#configure-a-custom-email-address-suffix) in project settings. Service Desk uses the [incoming email](../../administration/incoming_email.md) configuration by default. However, by using the `service_desk_email` configuration, you can customize the mailbox used by Service Desk. This allows you to have -a separate email address for Service Desk by also configuring a [custom suffix](#configuring-a-custom-email-address-suffix) +a separate email address for Service Desk by also configuring a [custom suffix](#configure-a-custom-email-address-suffix) in project settings. Prerequisites: @@ -255,7 +265,7 @@ The configuration options are the same as for configuring > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108279) in GitLab 15.9. Instead of having the Service Desk email credentials stored in plaintext in the configuration files, you can optionally -use an encrypted file for the Incoming email credentials. +use an encrypted file for the incoming email credentials. Prerequisites: @@ -384,7 +394,8 @@ read about [Helm IMAP secrets](https://docs.gitlab.com/charts/installation/secre > - Alternative Azure deployments [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5978) in GitLab 14.9. Service Desk can be configured to read Microsoft Exchange Online mailboxes with the Microsoft -Graph API instead of IMAP. Follow the [documentation in the incoming email section for setting up an OAuth 2.0 application for Microsoft Graph](../../administration/incoming_email.md#microsoft-graph). +Graph API instead of IMAP. Set up an OAuth 2.0 application for Microsoft Graph +[the same way as for incoming email](../../administration/incoming_email.md#microsoft-graph). - Example for Omnibus GitLab installations: @@ -403,32 +414,44 @@ Graph API instead of IMAP. Follow the [documentation in the incoming email secti } ``` -For Microsoft Cloud for US Government or [other Azure deployments](https://learn.microsoft.com/en-us/graph/deployments), configure the `azure_ad_endpoint` and `graph_endpoint` settings. +For Microsoft Cloud for US Government or [other Azure deployments](https://learn.microsoft.com/en-us/graph/deployments), +configure the `azure_ad_endpoint` and `graph_endpoint` settings. - Example for Microsoft Cloud for US Government: ```ruby - gitlab_rails['service_desk_email_inbox_options'] = { - 'azure_ad_endpoint': 'https://login.microsoftonline.us', - 'graph_endpoint': 'https://graph.microsoft.us', - 'tenant_id': '<YOUR-TENANT-ID>', - 'client_id': '<YOUR-CLIENT-ID>', - 'client_secret': '<YOUR-CLIENT-SECRET>', - 'poll_interval': 60 # Optional - } +gitlab_rails['service_desk_email_inbox_options'] = { + 'azure_ad_endpoint': 'https://login.microsoftonline.us', + 'graph_endpoint': 'https://graph.microsoft.us', + 'tenant_id': '<YOUR-TENANT-ID>', + 'client_id': '<YOUR-CLIENT-ID>', + 'client_secret': '<YOUR-CLIENT-SECRET>', + 'poll_interval': 60 # Optional } ``` -The Microsoft Graph API is not yet supported in source installations. See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/326169) for more details. +The Microsoft Graph API is not yet supported in source installations. +For more information, see [issue 326169](https://gitlab.com/gitlab-org/gitlab/-/issues/326169). -#### Configuring a custom email address suffix +#### Configure a custom email address suffix -You can set a custom suffix in your project's Service Desk settings after you have configured a [custom mailbox](#configuring-a-custom-mailbox). -It can contain only lowercase letters (`a-z`), numbers (`0-9`), or underscores (`_`). +You can set a custom suffix in your project's Service Desk settings. + +A suffix can contain only lowercase letters (`a-z`), numbers (`0-9`), or underscores (`_`). When configured, the custom suffix creates a new Service Desk email address, consisting of the `service_desk_email_address` setting and a key of the format: `<project_full_path>-<custom_suffix>` +Prerequisites: + +- You must have configured a [custom mailbox](#configure-a-custom-mailbox). + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Service Desk**. +1. Below **Email address suffix**, enter the suffix to use. +1. Select **Save changes**. + For example, suppose the `mygroup/myproject` project Service Desk settings has the following configured: - Email address suffix is set to `support`. @@ -437,13 +460,23 @@ For example, suppose the `mygroup/myproject` project Service Desk settings has t The Service Desk email address for this project is: `contact+mygroup-myproject-support@example.com`. The [incoming email](../../administration/incoming_email.md) address still works. -If you don't configure the custom suffix, the default project identification is used for identifying the project. You can see that email address in the project settings. +If you don't configure a custom suffix, the default project identification is used for identifying +the project. -## Using Service Desk +## Use Service Desk You can use Service Desk to [create an issue](#as-an-end-user-issue-creator) or [respond to one](#as-a-responder-to-the-issue). In these issues, you can also see our friendly neighborhood [Support Bot](#support-bot-user). +### View Service Desk email address + +To check what the Service Desk email address is for your project: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Issues > Service Desk**. + +The email address is available at the top of the issue list. + ### As an end user (issue creator) > Support for additional email headers [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346600) in GitLab 14.6. In earlier versions, the Service Desk email address had to be in the "To" field. @@ -483,42 +516,51 @@ You can read and write comments as you usually do in GitLab: - The project's visibility (private, internal, public) does not affect Service Desk. - The path to the project, including its group or namespace, is shown in emails. -#### Receiving files attached to comments as email attachments +#### View Service Desk issues -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11733) in GitLab 15.8 [with a flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. Disabled by default. -> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/386860) in GitLab 15.10. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. -On GitLab.com, this feature is available. +Prerequisites: -If a comment contains any attachments and their total size is less than or equal to 10 MB, these -attachments are sent as part of the email. In other cases, the email contains links to the attachments. +- You must have at least the Reporter role for the project. -In GitLab 15.9 and earlier, uploads to a comment are sent as links in the email. +To view Service Desk issues: -#### Special HTML formatting in HTML emails +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Issues > Service Desk**. -<!-- When the feature flag is removed, delete this topic and add as a line in version history under one of the topics above this one.--> +### Email contents and formatting -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/372301) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `service_desk_html_to_text_email_handler`. Disabled by default. +#### Special HTML formatting in HTML emails -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `service_desk_html_to_text_email_handler`. -On GitLab.com, this feature is not available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/109811) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `service_desk_html_to_text_email_handler`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116809) in GitLab 15.11. Feature flag `service_desk_html_to_text_email_handler` removed. -When this feature is enabled, HTML emails correctly show additional HTML formatting, such as: +HTML emails show HTML formatting, such as: - Tables - Blockquotes - Images - Collapsible sections -#### Privacy considerations +#### Files attached to comments + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11733) in GitLab 15.8 [with a flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/386860) in GitLab 15.10. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. +On GitLab.com, this feature is available. + +If a comment contains any attachments and their total size is less than or equal to 10 MB, these +attachments are sent as part of the email. In other cases, the email contains links to the attachments. + +In GitLab 15.9 and earlier, uploads to a comment are sent as links in the email. + +## Privacy considerations > [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108901) the minimum required role to view the creator's and participant's email in GitLab 15.9. -Service Desk issues are confidential, but the project owner can +Service Desk issues are [confidential](issues/confidential_issues.md), so they are +only visible to project members. The project owner can [make an issue public](issues/confidential_issues.md#modify-issue-confidentiality). When a Service Desk issue becomes public, the issue creator's and participants' email addresses are visible to signed-in users with at least the Reporter role for the project. @@ -526,17 +568,18 @@ visible to signed-in users with at least the Reporter role for the project. In GitLab 15.8 and earlier, when a Service Desk issue becomes public, the issue creator's email address is disclosed to everyone who can view the project. -### Support Bot user +Anyone in your project can use the Service Desk email address to create an issue in this project, **regardless +of their role** in the project. -Behind the scenes, Service Desk works by the special Support Bot user creating issues. This user -does not count toward the license limit count. +The unique internal email address is visible to project members at least +the Reporter role in your GitLab instance. +An external user (issue creator) cannot see the internal email address +displayed in the information note. ### Moving a Service Desk issue > [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/372246) in GitLab 15.7: customers continue receiving notifications when a Service Desk issue is moved. -Service Desk issues can be moved like any other issue in GitLab. - You can move a Service Desk issue the same way you [move a regular issue](issues/managing_issues.md#move-an-issue) in GitLab. diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 230795222a0..85ef3dc8405 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -7,40 +7,64 @@ info: "To determine the technical writer assigned to the Stage/Group associated # Migrating projects using file exports **(FREE)** Existing projects on any self-managed GitLab instance or GitLab.com can be exported to a file and -then imported into a new GitLab instance. You can also: +then imported into another GitLab instance. You can also copy GitLab projects to another location with more automation by +[migrating groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). -- Migrate projects when you [migrate groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). -- [Migrate groups by using file exports](../../group/import/index.md#migrate-groups-by-uploading-an-export-file-deprecated). +## Preserving user contributions + +Preserving user contribution depends on meeting the following requirements: + +### Migrating from GitLab self-managed to GitLab.com + +When migrating projects by using file exports, an administrator's access token is required for user contributions to map correctly. + +Therefore, user contributions never map correctly when importing file exports from a self-managed instance to GitLab.com. Instead, all GitLab user associations (such as +comment author) are changed to the user importing the project. To preserve contribution history, do one of the following: + +- [Migrate by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). +- Consider paid GitLab [migration services](https://about.gitlab.com/services/migration/). + +### Migrating to GitLab self-managed -GitLab maps user contributions correctly when an admin access token is used to perform the import. +To ensure GitLab maps users and their contributions correctly: -Consequently, migrating projects using file exports does not map user contributions correctly when you are importing -projects from a self-managed instance to GitLab.com. +- The owner of the project's top-level group should export the project so that the information of all members (direct and inherited) with access to the project can be included in the exported file. Project maintainers and owners can initiate the project export. However, only direct members of a project are then exported. +- An administrator must perform the import with an administrator access token. +- Required users must exist on the destination GitLab instance. An administrator can create confirmed users either in bulk in a Rails console or one by one in the UI. +- Users must [set a public email in their profiles](../../profile/index.md#set-your-public-email) on the source GitLab instance that matches their primary email + address on the destination GitLab instance. You can also manually add users' public emails by + [editing project export files](#edit-project-export-files). -Instead, all GitLab user associations (such as comment author) are changed to the user importing the project. For more -information, see the prerequisites and important notes in these sections: +When the email of an existing user matches the email of an imported user, that user is added as a [direct member](../members/index.md) to the imported project. -- [Export a project and its data](../settings/import_export.md#export-a-project-and-its-data). -- [Import the project](../settings/import_export.md#import-a-project-and-its-data). +If any of the previous conditions are not met, user contributions are not mapped correctly. Instead, all GitLab user associations are changed to the user who performed the import. +That user becomes an author of merge requests created by other users. Supplementary comments mentioning original authors are: -To preserve contribution history, [migrate using direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). +- Added for comments, merge request approvals, linked tasks, and items. +- Not added for the merge request or issue creator, added or removed labels, and merged-by information. -If you migrate from GitLab.com to self-managed GitLab, an administrator can create users on the self-managed GitLab instance. +## Edit project export files + +You can add or remove data from export files. For example, you can: + +- Manually add users public emails to the `project_members.ndjson` file. +- Trim CI pipelines by removing lines from the `ci_pipelines.ndjson` file. + +To edit a project export file: + +1. Extract the exported `.tar.gz` file. +1. Edit the appropriate file . For example, `tree/project/project_members.ndjson`. +1. Compress the files back to a `.tar.gz` file. + +You can also make sure that all members were exported by checking the `project_members.ndjson` file. ## Compatibility -FLAG: -On self-managed GitLab by default project, file exports are in NDJSON format. To make GitLab produce project file -exports in JSON format, ask an administrator to [disable the feature flags](../../../administration/feature_flags.md) -named `project_export_as_ndjson`. To allow GitLab to import project file exports in JSON format, ask an administrator to -[disable the feature flags](../../../administration/feature_flags.md) named `project_import_ndjson`. On GitLab.com, -project file exports are in NDJSON format only. +> Support for JSON-formatted project file exports [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/389888) in GitLab 15.11. -Project file exports are in NDJSON format. Before version 14.0, GitLab produced project file exports in JSON format. -To support transitions, you can still import JSON-formatted project file exports if you configure the relevant feature -flags. +Project file exports are in NDJSON format. -From GitLab 13.0, GitLab can import project file exports that were exported from a version of GitLab up to two +You can import project file exports that were exported from a version of GitLab up to two [minor](../../../policy/maintenance.md#versioning) versions behind, which is similar to our process for [security releases](../../../policy/maintenance.md#security-releases). @@ -85,7 +109,6 @@ Prerequisites: - Review the list of [items that are exported](#items-that-are-exported). Not all items are exported. - You must have at least the Maintainer role for the project. -- Users must [set a public email](../../profile/index.md#set-your-public-email) in the source GitLab instance that matches one of their verified emails in the target GitLab instance for the user mapping to work correctly. To export a project and its data, follow these steps: @@ -103,64 +126,68 @@ moved to your configured `uploads_directory`. Every 24 hours, a worker deletes t ### Items that are exported -The [`import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) -file for projects lists many of the items exported and imported when migrating projects using file exports. View this file in the branch -for your version of GitLab to see the list of items relevant to you. For example, -[`import_export.yml` on the `14-10-stable-ee` branch](https://gitlab.com/gitlab-org/gitlab/-/blob/14-10-stable-ee/lib/gitlab/import_export/project/import_export.yml). +Exported project items depend on the version of GitLab you use. To determine if a +specific project item is exported: -Migrating projects with file exports uses the same export and import mechanisms as creating projects from templates at the [group](../../group/custom_project_templates.md) and -[instance](../../admin_area/custom_project_templates.md) levels. Therefore, the list of exported items is the same. +1. Check the [`exporters` array](https://gitlab.com/gitlab-org/gitlab/blob/0a60d6dcfa7b809cf4fe0c2e239f406014d92e34/app/services/projects/import_export/export_service.rb#L25-28). +1. Check the [`project/import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) file for projects for your GitLab version (for example, `<https://gitlab.com/gitlab-org/gitlab/-/blob/15-9-stable-ee/lib/gitlab/import_export/project/import_export.yml>` for GitLab 15.9). -Items that are exported include: +For a quick overview, items that are exported include: - Project and wiki repositories - Project uploads - Project configuration, excluding integrations - Issues - Issue comments - - Issue iteration ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96184) in 15.4) - - Issue resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Issue resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Issue resource iteration events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue iterations ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96184) in GitLab 15.4) + - Issue resource state events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue resource milestone events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue resource iteration events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - Merge requests - Merge request diffs - Merge request comments - - Merge request resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Merge request multiple assignees ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) - - Merge request reviewers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) - - Merge request approvers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request resource state events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Merge request multiple assignees ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request reviewers ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request approvers ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) +- Commit comments ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/391601) in GitLab 15.10) - Labels - Milestones - Snippets - Time tracking and other project entities -- Design Management files and data +- Design management files and data - LFS objects - Issue boards - Pipelines history -- Push Rules +- Push rules - Awards -- Group members are exported as project members, as long as the user has the Maintainer role in the - exported project's group, or is an administrator +- Group members as long as the user has the Maintainer role in the + exported project's group or is an administrator + +### Items that are not exported Items that are **not** exported include: - [Child pipeline history](https://gitlab.com/gitlab-org/gitlab/-/issues/221088) +- Pipeline triggers - Build traces and artifacts - Package and container registry images - CI/CD variables -- Pipeline triggers - Webhooks - Any encrypted tokens - [Number of required approvals](https://gitlab.com/gitlab-org/gitlab/-/issues/221087) - Repository size limits - Deploy keys allowed to push to protected branches -- Secure Files +- Secure files - [Activity logs for Git-related events](https://gitlab.com/gitlab-org/gitlab/-/issues/214700) (for example, pushing and creating tags) - Security policies associated with your project +Migrating projects with file exports uses the same export and import mechanisms as creating projects from templates at the [group](../../group/custom_project_templates.md) and +[instance](../../admin_area/custom_project_templates.md) levels. Therefore, the list of exported items is the same. + ## Import a project and its data -> Default maximum import file size [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. +> Default maximum import file size [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. Administrators of self-managed instances can [set maximum import file size](#set-maximum-import-file-size). On GitLab.com, the value is [set to 5 GB](../../gitlab_com/index.md#account-and-limit-settings). WARNING: Only import projects from sources you trust. If you import a project from an untrusted source, it @@ -203,12 +230,7 @@ Deploy keys aren't imported. To use deploy keys, you must enable them in your im If you have a larger project, consider [using a Rake task](../../../administration/raketasks/project_import_export.md#import-large-projects). -## Automate group and project import **(PREMIUM)** - -For information on automating user, group, and project import API calls, see -[Automate group and project import](../import/index.md#automate-group-and-project-import). - -## Maximum import file size +## Set maximum import file size **(FREE SELF)** Administrators can set the maximum import file size one of two ways: @@ -217,34 +239,15 @@ Administrators can set the maximum import file size one of two ways: The default is `0` (unlimited). -For the GitLab.com setting, see the [Account and limit settings](../../gitlab_com/index.md#account-and-limit-settings) -section of the GitLab.com settings page. - -## Map users for import - -Imported users can be mapped by their public email addresses on self-managed instances, if an administrator (not an owner) does the import. - -- The project must be exported by a project or group member with the Owner role. -- Public email addresses are not set by default. Users must [set it in their profiles](../../profile/index.md#set-your-public-email) - for mapping to work correctly. -- For contributions to be mapped correctly, users must be an existing member of the namespace, - or they can be added as a member of the project. Otherwise, a supplementary comment is left to mention that the original - author and the merge requests, notes, or issues that are owned by the importer. -- Imported users are set as [direct members](../members/index.md) - in the imported project. - -For project migration imports performed over GitLab.com groups, preserving author information is -possible through a [professional services engagement](https://about.gitlab.com/services/migration/). - ## Rate limits To help avoid abuse, by default, users are rate limited to: -| Request Type | Limit | -| ---------------- | ----- | -| Export | 6 projects per minute | -| Download export | 1 download per group per minute | -| Import | 6 projects per minute | +| Request type | Limit | +|:----------------|:--------------------------------| +| Export | 6 projects per minute | +| Download export | 1 download per group per minute | +| Import | 6 projects per minute | ## Related topics @@ -252,3 +255,5 @@ To help avoid abuse, by default, users are rate limited to: - [Project import and export administration Rake tasks](../../../administration/raketasks/project_import_export.md) - [Migrating GitLab groups](../../group/import/index.md) - [Group import and export API](../../../api/group_import_export.md) +- [Migrate groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). +- [Migrate groups by using file exports](../../group/import/index.md#migrate-groups-by-uploading-an-export-file-deprecated). diff --git a/doc/user/project/settings/import_export_troubleshooting.md b/doc/user/project/settings/import_export_troubleshooting.md index 82412a1dcbf..530e7497b52 100644 --- a/doc/user/project/settings/import_export_troubleshooting.md +++ b/doc/user/project/settings/import_export_troubleshooting.md @@ -10,10 +10,10 @@ If you have problems with [migrating projects using file exports](import_export. ## Troubleshooting commands -Finds information about the status of the import and further logs using the JID: +Finds information about the status of the import and further logs using the JID, +using the [Rails console](../../../administration/operations/rails_console.md): ```ruby -# Rails console Project.find_by_full_path('group/project').import_state.slice(:jid, :status, :last_error) > {"jid"=>"414dec93f941a593ea1a6894", "status"=>"finished", "last_error"=>nil} ``` @@ -35,6 +35,27 @@ Review [issue 276930](https://gitlab.com/gitlab-org/gitlab/-/issues/276930), and - Ensure shared runners are enabled in both the source and destination projects. - Disable shared runners on the parent group when you import the project. +## Users missing from imported project + +If users aren't imported with imported projects, see the [preserving user contributions](import_export.md#preserving-user-contributions) requirements. + +A common reason for missing users is that the [public email setting](../../profile/index.md#set-your-public-email) isn't configured for users. +To resolve this issue, ask users to configure this setting using the GitLab UI. + +If there are too many users for manual configuration to be feasible, +you can set all user profiles to use a public email address using the +[Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session): + +```ruby +User.where("public_email IS NULL OR public_email = '' ").find_each do |u| + next if u.bot? + + puts "Setting #{u.username}'s currently empty public email to #{u.email}…" + u.public_email = u.email + u.save! +end +``` + ## Import workarounds for large repositories [Maximum import size limitations](import_export.md#import-a-project-and-its-data) @@ -153,12 +174,12 @@ Rather than attempting to push all changes at once, this workaround: You usually export a project through [the web interface](import_export.md#export-a-project-and-its-data) or through [the API](../../../api/project_import_export.md). Exporting using these methods can sometimes fail without giving enough information to troubleshoot. In these cases, -[open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session) and loop through +[open a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session) and loop through [all the defined exporters](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/projects/import_export/export_service.rb). Execute each line individually, rather than pasting the entire block at once, so you can see any errors each command returns. -```shell +```ruby # User needs to have permission to export u = User.find_by_username('someuser') p = Project.find_by_full_path('some/project') diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 7250406144f..18030ab0a81 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -75,28 +75,28 @@ To configure visibility, features, and permissions for a project: Use the toggles to enable or disable features in the project. -| Option | More access limit options | Description | -| :------------------------------- | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Issues** | ✓ | Activates the GitLab issues tracker. | -| **Repository** | ✓ | Enables [repository](../repository/index.md) functionality | -| **Merge requests** | ✓ | Enables [merge request](../merge_requests/index.md) functionality; also see [Merge request settings](#configure-merge-request-settings-for-a-project). | -| **Forks** | ✓ | Enables [forking](../repository/forking_workflow.md) functionality. | -| **Git Large File Storage (LFS)** | | Enables the use of [large files](../../../topics/git/lfs/index.md#git-large-file-storage-lfs). | -| **Packages** | | Supports configuration of a [package registry](../../../administration/packages/index.md#gitlab-package-registry-administration) functionality. | -| **CI/CD** | ✓ | Enables [CI/CD](../../../ci/index.md) functionality. | -| **Container Registry** | | Activates a [registry](../../packages/container_registry/index.md) for your Docker images. | -| **Analytics** | ✓ | Enables [analytics](../../analytics/index.md). | -| **Requirements** | ✓ | Control access to [Requirements Management](../requirements/index.md). | -| **Security and Compliance** | ✓ | Control access to [security features](../../application_security/index.md). | -| **Wiki** | ✓ | Enables a separate system for [documentation](../wiki/index.md). | -| **Snippets** | ✓ | Enables [sharing of code and text](../../snippets.md). | -| **Pages** | ✓ | Allows you to [publish static websites](../pages/index.md). | -| **Metrics Dashboard** | ✓ | Control access to [metrics dashboard](../integrations/prometheus.md). | -| **Releases** | ✓ | Control access to [Releases](../releases/index.md). | -| **Environments** | ✓ | Control access to [Environments and Deployments](../../../ci/environments/index.md). | -| **Feature flags** | ✓ | Control access to [Feature Flags](../../../operations/feature_flags.md). | -| **Monitor** | ✓ | Control access to [Monitor](../../../operations/index.md) features. | -| **Infrastructure** | ✓ | Control access to [Infrastructure](../../infrastructure/index.md) features. | +| Option | More access limit options | Description +| :------------------------------- | :------------------------ | :---------- | +| **Issues** | **{check-circle}** Yes | Activates the GitLab issues tracker. +| **Repository** | **{check-circle}** Yes | Enables [repository](../repository/index.md) functionality. +| **Merge requests** | **{check-circle}** Yes | Enables [merge request](../merge_requests/index.md) functionality; also see [Merge request settings](#configure-merge-request-settings-for-a-project). +| **Forks** | **{check-circle}** Yes | Enables [forking](../repository/forking_workflow.md) functionality. +| **Git Large File Storage (LFS)** | **{dotted-circle}** No | Enables the use of [large files](../../../topics/git/lfs/index.md#git-large-file-storage-lfs). +| **Packages** | **{dotted-circle}** No | Supports configuration of a [package registry](../../../administration/packages/index.md#gitlab-package-registry-administration) functionality. +| **CI/CD** | **{check-circle}** Yes | Enables [CI/CD](../../../ci/index.md) functionality. +| **Container Registry** | **{dotted-circle}** No | Activates a [registry](../../packages/container_registry/index.md) for your Docker images. +| **Analytics** | **{check-circle}** Yes | Enables [analytics](../../analytics/index.md). +| **Requirements** | **{check-circle}** Yes | Control access to [Requirements Management](../requirements/index.md). +| **Security and Compliance** | **{check-circle}** Yes | Control access to [security features](../../application_security/index.md). +| **Wiki** | **{check-circle}** Yes | Enables a separate system for [documentation](../wiki/index.md). +| **Snippets** | **{check-circle}** Yes | Enables [sharing of code and text](../../snippets.md). +| **Pages** | **{check-circle}** Yes | Allows you to [publish static websites](../pages/index.md). +| **Metrics Dashboard** | **{check-circle}** Yes | Control access to [metrics dashboard](../integrations/prometheus.md). +| **Releases** | **{check-circle}** Yes | Control access to [Releases](../releases/index.md). +| **Environments** | **{check-circle}** Yes | Control access to [Environments and Deployments](../../../ci/environments/index.md). +| **Feature flags** | **{check-circle}** Yes | Control access to [Feature flags](../../../operations/feature_flags.md). +| **Monitor** | **{check-circle}** Yes | Control access to [Monitor](../../../operations/index.md) features. +| **Infrastructure** | **{check-circle}** Yes | Control access to [Infrastructure](../../infrastructure/index.md) features. When you disable a feature, the following additional features are also disabled: @@ -162,6 +162,7 @@ Configure your project's merge request settings: - Configure [suggested changes commit messages](../merge_requests/reviews/suggestions.md#configure-the-commit-message-for-applied-suggestions). - Configure [merge and squash commit message templates](../merge_requests/commit_templates.md). - Configure [the default target project](../merge_requests/creating_merge_requests.md#set-the-default-target-project) for merge requests coming from forks. +- Enable [Suggested Reviewers](../merge_requests/reviews/index.md#suggested-reviewers). ## Service Desk @@ -174,7 +175,7 @@ Learn how to [export a project](import_export.md#import-a-project-and-its-data) ## Advanced project settings Use the advanced settings to archive, rename, transfer, -remove a fork relationship, or delete a project. +[remove a fork relationship](../repository/forking_workflow.md#unlink-a-fork), or delete a project. ### Archive a project @@ -347,24 +348,6 @@ To restore a project marked for deletion: 1. Navigate to your project, and select **Settings > General > Advanced**. 1. In the Restore project section, select **Restore project**. -## Remove a fork relationship - -Prerequisites: - -- You must be a project owner to remove a fork relationship. - -WARNING: -If you remove a fork relationship, you can't send merge requests to the source. If anyone has forked your project, their fork also loses the relationship. -To restore the fork relationship, [use the API](../../../api/projects.md#create-a-forked-fromto-relation-between-existing-projects). - -To remove a fork relationship: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Advanced**. -1. In the **Remove fork relationship** section, select **Remove fork relationship**. -1. To confirm, enter the project path and select **Confirm**. - ## Monitor settings ### Alerts @@ -398,16 +381,6 @@ to enable the syncing of public Issues to a [deployed status page](../../../oper When working with project settings, you might encounter the following issues, or require alternate methods to complete specific tasks. -### Remove a fork relationship through console - -If removing the fork through the UI or API is not working, you can attempt the fork relationship removal in a [Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session). - -```ruby -p = Project.find_by_full_path('<project_path>') -u = User.find_by_username('<username>') -Projects::UnlinkForkService.new(p, u).execute -``` - ### Transfer a project through console If transferring a project through the UI or API is not working, you can attempt the transfer in a [Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session). diff --git a/doc/user/project/system_notes.md b/doc/user/project/system_notes.md new file mode 100644 index 00000000000..1eee4364d95 --- /dev/null +++ b/doc/user/project/system_notes.md @@ -0,0 +1,56 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# System notes **(FREE)** + +System notes are short descriptions that help you understand the history of events +that occur during the life cycle of a GitLab object, such as: + +- [Alerts](../../operations/incident_management/alerts.md). +- [Designs](issues/design_management.md). +- [Issues](issues/index.md). +- [Merge requests](merge_requests/index.md). +- [Objectives and key results](../okrs.md) (OKRs). +- [Tasks](../tasks.md). + +GitLab logs information about events triggered by Git or the GitLab application +in system notes. System notes use the format `<Author> <action> <time ago>`. + +## Show or filter system notes + +By default, system notes do not display. When displayed, they are shown oldest first. +If you change the filter or sort options, your selection is remembered across sections. +The filtering options are: + +- **Show all activity** displays both comments and history. +- **Show comments only** hides system notes. +- **Show history only** hides user comments. + +### On an epic + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Epics** (**{epic}**). +1. Identify your desired epic, and select its title. +1. Go to the **Activity** section. +1. For **Sort or filter**, select **Show all activity**. + +### On an issue + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Issues** and find your issue. +1. Go to **Activity**. +1. For **Sort or filter**, select **Show all activity**. + +### On a merge request + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. Go to **Activity**. +1. For **Sort or filter**, select **Show all activity**. + +## Related topics + +- [Notes API](../../api/notes.md) diff --git a/doc/user/project/time_tracking.md b/doc/user/project/time_tracking.md index d5f0b7e6180..bd935db6f02 100644 --- a/doc/user/project/time_tracking.md +++ b/doc/user/project/time_tracking.md @@ -1,6 +1,5 @@ --- type: reference -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/time_tracking.html' stage: Plan group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index e2a74f02008..889ab1442b6 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -296,7 +296,7 @@ An example `package.json`: > [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/211685) from GitLab Ultimate to GitLab Free in 13.1. WARNING: -Interactive Web Terminals for the Web IDE is currently in [**Beta**](../../../policy/alpha-beta-support.md#beta-features). +Interactive Web Terminals for the Web IDE is currently in [Beta](../../../policy/alpha-beta-support.md#beta). GitLab.com shared runners [do not yet support Interactive Web Terminals](https://gitlab.com/gitlab-org/gitlab/-/issues/24674), so you must use your own private runner to make use of this feature. @@ -450,7 +450,7 @@ when: - You select any area outside the file editor after editing a file. - A file or folder is created, deleted, or renamed. -### Known issues +## Known issues The Web IDE has a few limitations: @@ -458,7 +458,9 @@ The Web IDE has a few limitations: active terminal at a time. - LFS files can be rendered and displayed but they cannot be updated and committed using the Web IDE. If an LFS file is modified and pushed to the repository, the LFS pointer in the repository is overwritten with the modified LFS file content. -### Troubleshooting +## Troubleshooting + +### Web terminals - If the terminal's text is gray and unresponsive, then the terminal has stopped and it can no longer be used. A stopped terminal can be restarted by selecting @@ -466,7 +468,3 @@ The Web IDE has a few limitations: - If the terminal displays **Connection Failure**, then the terminal could not connect to the runner. Try to stop and restart the terminal. If the problem persists, double check your runner configuration. - -## Related topics - -- [Web IDE Beta](../web_ide_beta/index.md) diff --git a/doc/user/project/web_ide_beta/index.md b/doc/user/project/web_ide_beta/index.md index 635b5e12575..933634f0c29 100644 --- a/doc/user/project/web_ide_beta/index.md +++ b/doc/user/project/web_ide_beta/index.md @@ -1,15 +1,17 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Web IDE Beta **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.7 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.7 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/371084) in GitLab 15.7. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115741) in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. As announced in [this blog post](https://about.gitlab.com/blog/2022/05/23/the-future-of-the-gitlab-web-ide/), the current implementation of the [Web IDE](../web_ide/index.md) is being replaced @@ -18,15 +20,6 @@ development. For updates, see [this epic](https://gitlab.com/groups/gitlab-org/- To pair the Web IDE Beta with a Remote Development environment, see [Remote Development](../remote_development/index.md). -## Enable the Web IDE Beta - -To use the Web IDE Beta on a self-managed GitLab instance, -ensure that the `vscode_web_ide` feature flag -[is enabled](../../../administration/feature_flags.md). - -On GitLab.com, this feature is available by default. However, you can -[stop using it if you choose](#stop-using-the-web-ide-beta). - ## Use the Web IDE Beta To open the Web IDE Beta from anywhere in the UI: @@ -55,6 +48,10 @@ To open the Web IDE Beta from a merge request: 1. Go to your merge request. 1. In the upper-right corner, select **Code > Open in Web IDE**. +The Web IDE Beta opens new and modified files in separate tabs and displays changes side by side with the original source. To optimize loading time, only the top 10 files (by number of lines changed) are opened automatically. + +In the file tree, any new or modified file in the merge request is indicated by an icon next to the filename. To view changes to a file, right-click the filename and select **Compare with merge request base**. + ## Open a file in the Web IDE Beta To open any file by its name: @@ -99,11 +96,21 @@ Full file search is planned for a later date. ## View a list of changed files To view a list of files you changed in the Web IDE Beta, -in the Activity Bar on the left, select **Source Control**. +on the Activity Bar on the left, select **Source Control**. Your `CHANGES`, `STAGED CHANGES`, and `MERGE CHANGES` are displayed. For details, see the [VS Code documentation](https://code.visualstudio.com/docs/sourcecontrol/overview#_commit). +## Commit changes + +To commit your changes in the Web IDE Beta: + +1. On the Activity Bar on the left, select **Source Control**, +or press <kbd>Control</kbd>+<kbd>Shift</kbd>+<kbd>G</kbd>. +1. Enter your commit message. +1. Select **Commit & Push**. +1. Commit to the current branch, or create a new branch. + ## Open the command palette In the Web IDE Beta, you can access many commands through the command palette. @@ -128,8 +135,3 @@ The [Web Terminal](../web_ide/index.md#interactive-web-terminals-for-the-web-ide and [Live Preview](../web_ide/index.md#live-preview-removed) are not available in the Web IDE Beta. These features might become available at a later date. - -## Related topics - -- [Remote Development](../remote_development/index.md) -- [Web IDE](../web_ide/index.md) diff --git a/doc/user/project/wiki/group.md b/doc/user/project/wiki/group.md index 1513ea18ed2..a3a6cad65e1 100644 --- a/doc/user/project/wiki/group.md +++ b/doc/user/project/wiki/group.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -71,7 +71,7 @@ To open group settings: 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. 1. Scroll to **Wiki** and select one of these options: - - **Enabled**: Everyone who can access the group can access the wiki. + - **Enabled**: For public groups, everyone can access the wiki. For internal groups, only authenticated users can access the wiki. - **Private**: Only group members can access the wiki. - **Disabled**: The wiki isn't accessible, and cannot be downloaded. 1. Select **Save changes**. diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index f01331ac784..eb13814f2ad 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/working_with_projects.md b/doc/user/project/working_with_projects.md index 24ca86fc93b..69087791a3e 100644 --- a/doc/user/project/working_with_projects.md +++ b/doc/user/project/working_with_projects.md @@ -328,7 +328,7 @@ download starts, the `insteadOf` configuration sends the traffic to the secondar - [Import a project](../../user/project/import/index.md). - [Connect an external repository to GitLab CI/CD](../../ci/ci_cd_for_external_repos/index.md). -- [Fork a project](repository/forking_workflow.md#creating-a-fork). +- [Fork a project](repository/forking_workflow.md#create-a-fork). - [Adjust project visibility and access levels](settings/index.md#configure-project-visibility-features-and-permissions). - [Limitations on project and group names](../../user/reserved_names.md#limitations-on-project-and-group-names) diff --git a/doc/user/public_access.md b/doc/user/public_access.md index 87380ec324e..58eebc16d74 100644 --- a/doc/user/public_access.md +++ b/doc/user/public_access.md @@ -95,8 +95,7 @@ For more information, see [Restrict visibility levels](admin_area/settings/visib ## Related topics -- For more granular control of project features, you can - [change the visibility of features](project/working_with_projects.md#change-the-visibility-of-individual-features-in-a-project). +- [Change the visibility of features](project/working_with_projects.md#change-the-visibility-of-individual-features-in-a-project) <!-- ## Troubleshooting diff --git a/doc/user/report_abuse.md b/doc/user/report_abuse.md index a20a699e550..de2b82c28d3 100644 --- a/doc/user/report_abuse.md +++ b/doc/user/report_abuse.md @@ -66,5 +66,4 @@ A URL to the reported user's comment is pre-filled in the abuse report's ## Related topics -- Administrators can view and resolve abuse reports. - For more information, see [abuse reports administration documentation](admin_area/review_abuse_reports.md). +- [Abuse reports administration documentation](admin_area/review_abuse_reports.md) diff --git a/doc/user/search/exact_code_search.md b/doc/user/search/exact_code_search.md index 76a298a2cee..674b2813985 100644 --- a/doc/user/search/exact_code_search.md +++ b/doc/user/search/exact_code_search.md @@ -10,9 +10,8 @@ type: reference > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/105049) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `index_code_with_zoekt` and `search_code_with_zoekt` which enables indexing and searching respectively. Both are disabled by default. WARNING: -Exact Code Search is an MVC. For the Exact Code Search feature roadmap, see [epic 9404](https://gitlab.com/groups/gitlab-org/-/epics/9404). -When this feature reaches the [**Alpha**](../../policy/alpha-beta-support.md#alpha-features) version, GitLab will dogfood it, and roll it out only to specific customers on GitLab.com who request access to it. -On self-managed GitLab, this feature is available and can be enabled. However, GitLab does not provide support or documentation at this development stage. +We are still actively making changes to the Exact Code Search feature. GitLab will dogfood it first, and roll it out only to specific customers on GitLab.com who request access to it. We will make an announcement when it's available for GitLab.com customers to tryout. You can follow our development progress by checking [the Exact Code Search feature roadmap](https://gitlab.com/groups/gitlab-org/-/epics/9404). +On self-managed GitLab, it is technically possible to enable this feature, however, GitLab does not provide support or documentation at this stage of development and it has not been widely tested at scale. There are also many known limitations. ## Usage @@ -27,5 +26,7 @@ expressions, certain special characters will require escaping. Backslash can escape special characters and wrapping in double quotes can be used for phrase searches. +## Syntax + To understand the possible filtering options, see the [Zoekt query syntax](https://github.com/sourcegraph/zoekt/blob/main/doc/query_syntax.md). diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 310d733800a..f6733abd305 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -95,12 +95,10 @@ To filter code search results by one or more languages: ## Search for projects by full path -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108906) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `full_path_project_search`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. -To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `full_path_project_search`. -On GitLab.com, this feature is not available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108906) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `full_path_project_search`. Disabled by default. +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/388473) on GitLab.com in GitLab 15.9. +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111808) on self-managed GitLab 15.10. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114932) in GitLab 15.11. Feature flag `full_path_project_search` removed. You can search for a project by entering its full path (including the namespace it belongs to) in the search box. As you type the project path, [autocomplete suggestions](#autocomplete-suggestions) are displayed. diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index 9c2925ec647..c0d2b74fd84 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -1,9 +1,8 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/shortcuts.html' --- # GitLab keyboard shortcuts **(FREE)** @@ -242,7 +241,7 @@ This shortcut is available when viewing a [wiki page](project/wiki/index.md): ### Content editor These shortcuts are available when editing a file with the -[Content Editor](https://about.gitlab.com/direction/create/editor/content_editor/): +[Content Editor](https://about.gitlab.com/direction/plan/knowledge/content_editor/): | macOS shortcut | Windows shortcut | Description | |----------------|------------------|-------------| diff --git a/doc/user/ssh.md b/doc/user/ssh.md index 7b6fa66b1d7..a74c3fea360 100644 --- a/doc/user/ssh.md +++ b/doc/user/ssh.md @@ -20,8 +20,6 @@ SSH uses two keys, a public key and a private key. - The public key can be distributed. - The private key should be protected. -When you need to copy or upload your SSH public key, make sure you do not accidentally copy or upload your private key instead. - You cannot expose data by uploading your public key. When you need to copy or upload your SSH public key, make sure you do not accidentally copy or upload your private key instead. You can use your private key to [sign commits](project/repository/ssh_signed_commits/index.md), @@ -280,7 +278,7 @@ You can use [1Password](https://1password.com/) and the [1Password browser exten 1. Sign in to GitLab. 1. On the top bar, in the upper-right corner, select your avatar. -1. Select **Preferences**. +1. Select **Edit profile**. 1. On the left sidebar, select **SSH Keys**. 1. Select **Key**, and you should see the 1Password helper appear. 1. Select the 1Password icon and unlock 1Password. @@ -291,7 +289,7 @@ You can use [1Password](https://1password.com/) and the [1Password browser exten 1. Optional. Update **Expiration date** to modify the default expiration date. 1. Select **Add key**. -For more information about using 1Password with SSH keys, see the [1Password documentation](https://developer.1password.com/docs/ssh/get-started). +For more information about using 1Password with SSH keys, see the [1Password documentation](https://developer.1password.com/docs/ssh/get-started/). ## Add an SSH key to your GitLab account @@ -325,7 +323,7 @@ To use SSH with GitLab, copy your public key to your GitLab account: 1. Sign in to GitLab. 1. On the top bar, in the upper-right corner, select your avatar. -1. Select **Preferences**. +1. Select **Edit profile**. 1. On the left sidebar, select **SSH Keys**. 1. In the **Key** box, paste the contents of your public key. If you manually copied the key, make sure you copy the entire key, @@ -398,7 +396,7 @@ on `ssh` command options, see the `man` pages for both `ssh` and `ssh_config`. 1. Sign in to GitLab. 1. On the top bar, in the upper-right corner, select your avatar. -1. Select **Preferences**. +1. Select **Edit profile**. 1. On the left sidebar, select **SSH Keys**. Your existing SSH keys are listed at the bottom of the page. The information includes: diff --git a/doc/user/tasks.md b/doc/user/tasks.md index eb0184da929..48a6b3e6490 100644 --- a/doc/user/tasks.md +++ b/doc/user/tasks.md @@ -298,10 +298,6 @@ To add a task to an iteration: > - Filtering activity [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/389971) in GitLab 15.10. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/334812) in GitLab 15.10. -Prerequisites: - -- You must have at least the Reporter role for the project. - You can view all the system notes related to the task. By default they are sorted by **Oldest first**. You can always change the sorting order to **Newest first**, which is remembered across sessions. You can also filter activity by **Comments only** and **History only** in addition to the default **All activity** which is remembered across sessions. diff --git a/doc/user/todos.md b/doc/user/todos.md index 4951ba024af..29fe3336ea2 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -1,5 +1,4 @@ --- -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/todos.html' stage: Plan group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments @@ -34,12 +33,13 @@ Also, you can sort them by [**Label priority**](project/labels.md#set-label-prio ## Actions that create to-do items Many to-do items are created automatically. -A to-do item is added to your To-Do List when: +Some of the actions that add a to-do item to your To-Do List: - An issue or merge request is assigned to you. +- A [merge request review](project/merge_requests/reviews/index.md) is requested. - You're [mentioned](discussions/index.md#mentions) in the description or comment of an issue, merge request, or epic. -- You are mentioned in a comment on a commit or design. +- You're mentioned in a comment on a commit or design. - The CI/CD pipeline for your merge request fails. - An open merge request cannot be merged due to conflict, and one of the following is true: diff --git a/doc/user/workspace/quick_start/index.md b/doc/user/workspace/quick_start/index.md new file mode 100644 index 00000000000..933dbf1766b --- /dev/null +++ b/doc/user/workspace/quick_start/index.md @@ -0,0 +1,18 @@ +--- +stage: Create +group: IDE +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +> Introduced in GitLab 15.11 [with a flag](../../../administration/feature_flags.md) named `remote_development_feature_flag`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `remote_development_feature_flag`. +On GitLab.com, this feature is not available. +The feature is not ready for production use. + +# Tutorial: Create and run your first GitLab Workspace **(ULTIMATE)** + +This tutorial shows you how to configure and run your first Remote Development Workspace in GitLab. |
