diff options
Diffstat (limited to 'doc')
759 files changed, 14618 insertions, 7613 deletions
diff --git a/doc/.vale/gitlab/British.yml b/doc/.vale/gitlab/British.yml index f724eb19fa9..c63cbd917c7 100644 --- a/doc/.vale/gitlab/British.yml +++ b/doc/.vale/gitlab/British.yml @@ -80,6 +80,9 @@ swap: neighbour: neighbor normalise: normalize offence: offense + optimise: optimize + optimised: optimized + optimising: optimizing organise: organize orientated: oriented paralyse: paralyze diff --git a/doc/.vale/gitlab/InternalLinkExtension.yml b/doc/.vale/gitlab/InternalLinkExtension.yml index 5783c4347a9..52142b50dfc 100644 --- a/doc/.vale/gitlab/InternalLinkExtension.yml +++ b/doc/.vale/gitlab/InternalLinkExtension.yml @@ -5,9 +5,9 @@ # # For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles extends: existence -message: 'Link "%s" must use the .md file extension.' +message: 'Link "%s" must link directly to a file and use the .md file extension.' link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links-to-internal-documentation level: error scope: raw raw: - - '\[.+\]\([\w\/\.-]+\.html[^)]*\)' + - '\[[^\]]+\]\([^:\)]+(\/(#[^\)]+)?\)|\.html(#.+)?\))' diff --git a/doc/.vale/gitlab/Markdown_emoji.yml b/doc/.vale/gitlab/Markdown_emoji.yml new file mode 100644 index 00000000000..ac0dab2d69d --- /dev/null +++ b/doc/.vale/gitlab/Markdown_emoji.yml @@ -0,0 +1,13 @@ +--- +# Warning: gitlab.Markdown_emoji +# +# Check for use of GLFM emoji syntax (https://docs.gitlab.com/ee/user/markdown.html#emojis), which doesn't render correctly in documentation. +# +# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles +extends: existence +message: 'This appears to be GLFM emoji syntax. Replace "%s" with GitLab SVGs or Unicode emojis.' +link: https://docs.gitlab.com/ee/development/documentation/styleguide/#gitlab-svg-icons +level: warning +scope: text +raw: + - '(?:\s+|^):[a-zA-Z0-9\-_\+]+:(?:\s+|$|\.)' diff --git a/doc/administration/application_settings_cache.md b/doc/administration/application_settings_cache.md index 30fd9ab85a8..88e39a50b6c 100644 --- a/doc/administration/application_settings_cache.md +++ b/doc/administration/application_settings_cache.md @@ -18,7 +18,9 @@ extra load for Redis and PostgreSQL. To change the expiry value: -**For Omnibus installations** +::Tabs + +:::TabTitle Omnibus package 1. Edit `/etc/gitlab/gitlab.rb`: @@ -34,7 +36,7 @@ To change the expiry value: gitlab-ctl restart ``` -**For installations from source** +:::TabTitle Source 1. Edit `config/gitlab.yml`: @@ -45,3 +47,5 @@ To change the expiry value: 1. Save the file, and then [restart](restart_gitlab.md#installations-from-source) GitLab for the changes to take effect. + +::EndTabs diff --git a/doc/administration/audit_event_streaming.md b/doc/administration/audit_event_streaming.md index 3bb0ce41861..9ec7b81bfd0 100644 --- a/doc/administration/audit_event_streaming.md +++ b/doc/administration/audit_event_streaming.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -16,7 +16,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Custom HTTP headers API [made generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/366524) in GitLab 15.3. [Feature flag `streaming_audit_event_headers`](https://gitlab.com/gitlab-org/gitlab/-/issues/362941) removed. > - Custom HTTP headers UI [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361630) in GitLab 15.2 [with a flag](feature_flags.md) named `custom_headers_streaming_audit_events_ui`. Disabled by default. > - Custom HTTP headers UI [made generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/365259) in GitLab 15.3. [Feature flag `custom_headers_streaming_audit_events_ui`](https://gitlab.com/gitlab-org/gitlab/-/issues/365259) removed. -> - [Improved user experience](https://gitlab.com/gitlab-org/gitlab/-/issues/367963) in GitLab 15.4. +> - [Improved user experience](https://gitlab.com/gitlab-org/gitlab/-/issues/367963) in GitLab 15.3. +> - User-specified verification token API support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360813) in GitLab 15.4. Users can set a streaming destination for a top-level group to receive all audit events about the group, its subgroups, and projects as structured JSON. @@ -38,7 +39,7 @@ Streaming destinations receive **all** audit event data, which could include sen Users with the Owner role for a group can add streaming destinations for it: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Audit events**. 1. On the main area, select **Streams** tab. 1. Select **Add streaming destination** to show the section for adding destinations. @@ -71,6 +72,25 @@ mutation { } ``` +Group owners can also optionally specify their own verification token (instead of the default GitLab-generated one) using the GraphQL `auditEventsStreamingHeadersCreate` +mutation. Verification token length must be within 16 to 24 characters and trailing whitespace are not trimmed. GitLab recommends setting a cryptographically random and unique value. For example: + +```graphql +mutation { + externalAuditEventDestinationCreate(input: { destinationUrl: "https://mydomain.io/endpoint/ingest", groupPath: "my-group", verificationToken: "unique-random-verification-token-here" } ) { + errors + externalAuditEventDestination { + id + destinationUrl + verificationToken + group { + name + } + } + } +} +``` + Event streaming is enabled if: - The returned `errors` object is empty. @@ -97,7 +117,7 @@ Users with the Owner role for a group can list streaming destinations. To list the streaming destinations: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Audit events**. 1. On the main area, select **Streams** tab. 1. To the right of the item, select **Edit** (**{pencil}**) to see all the custom HTTP headers. @@ -141,7 +161,7 @@ Users with the Owner role for a group can update streaming destinations. To update a streaming destinations custom HTTP headers: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Audit events**. 1. On the main area, select **Streams** tab. 1. To the right of the item, select **Edit** (**{pencil}**). @@ -195,14 +215,14 @@ When the last destination is successfully deleted, streaming is disabled for the To delete a streaming destination: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Audit events**. 1. On the main area, select the **Streams** tab. 1. To the right of the item, select **Delete** (**{remove}**). To delete only the custom HTTP headers for a streaming destination: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Audit events**. 1. On the main area, select the **Streams** tab. 1. To the right of the item, **Edit** (**{pencil}**). @@ -248,9 +268,9 @@ The header is deleted if the returned `errors` object is empty. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345424) in GitLab 14.8. Each streaming destination has a unique verification token (`verificationToken`) that can be used to verify the authenticity of the event. This -token is generated when the event destination is created and cannot be changed. +token is either specified by the Owner or generated automatically when the event destination is created and cannot be changed. -Each streamed event contains a random alphanumeric identifier for the `X-Gitlab-Event-Streaming-Token` HTTP header that can be verified against +Each streamed event contains the verification token in the `X-Gitlab-Event-Streaming-Token` HTTP header that can be verified against the destination's value when [listing streaming destinations](#list-streaming-destinations). ### Use the GitLab UI @@ -259,7 +279,7 @@ the destination's value when [listing streaming destinations](#list-streaming-de Users with the Owner role for a group can list streaming destinations and see the verification tokens: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Audit events**. 1. On the main area, select the **Streams**. 1. View the verification token on the right side of each item. @@ -297,7 +317,7 @@ FLAG: On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](feature_flags.md) named `audit_event_streaming_git_operations`. -Streaming audit events can be sent when signed-in users push or pull a project's remote Git repositories: +Streaming audit events can be sent when signed-in users push, pull, or clone a project's remote Git repositories: - [Using SSH](../user/ssh.md). - Using HTTP or HTTPS. diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index 92504c226fb..b6c267bfd0c 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -152,7 +152,7 @@ From there, you can see the following actions: - Added, removed, or updated protected branches - Release was added to a project - Release was updated -- Release was deleted ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/94793/) in GitLab 13.5) +- Release was deleted ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/94793/) in GitLab 15.3) - Release milestone associations changed - Permission to approve merge requests by committers was updated ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7531) in GitLab 12.9) - Permission to approve merge requests by committers was updated. @@ -209,7 +209,7 @@ Instance events do not include group or project audit events. To view the server-wide audit events: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Audit Events**. The following user actions are recorded: @@ -309,7 +309,7 @@ audit events. To export the audit events to CSV: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Audit Events**. 1. Select the available search [filters](#search). 1. Select **Export as CSV**. diff --git a/doc/administration/audit_reports.md b/doc/administration/audit_reports.md index e33b5153c5b..e363e7862ea 100644 --- a/doc/administration/audit_reports.md +++ b/doc/administration/audit_reports.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments description: 'Learn how to create evidence artifacts typically requested by a 3rd party auditor.' diff --git a/doc/administration/auditor_users.md b/doc/administration/auditor_users.md index 561aa5d7b2e..bd48cfdbc4f 100644 --- a/doc/administration/auditor_users.md +++ b/doc/administration/auditor_users.md @@ -31,7 +31,7 @@ To create a new user account with auditor access (or change an existing user): To create a user account with auditor access: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Create a new user or edit an existing one. Set **Access Level** to **Auditor**. 1. If you created a user, select **Create user**. For an existing user, select **Save changes**. diff --git a/doc/administration/auth/atlassian.md b/doc/administration/auth/atlassian.md index 6be53922a5a..1d20d87bdf4 100644 --- a/doc/administration/auth/atlassian.md +++ b/doc/administration/auth/atlassian.md @@ -71,7 +71,10 @@ To enable the Atlassian OmniAuth provider for passwordless authentication you mu 1. Change `YOUR_CLIENT_ID` and `YOUR_CLIENT_SECRET` to the Client credentials you received in [application registration](#atlassian-application-registration) steps. 1. Save the configuration file. -1. [Reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect if you installed GitLab via Omnibus or from source respectively. + +1. For the changes to take effect: + - If you installed via Omnibus, [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + - If you installed from source, [restart GitLab](../restart_gitlab.md#installations-from-source). On the sign-in page there should now be an Atlassian icon below the regular sign in form. Select the icon to begin the authentication process. diff --git a/doc/administration/auth/index.md b/doc/administration/auth/index.md index a412875501e..2644e3a1f50 100644 --- a/doc/administration/auth/index.md +++ b/doc/administration/auth/index.md @@ -34,31 +34,3 @@ For more information, see the links shown on this page for each external provide | **User Removal** | SCIM (remove user from top-level group) | LDAP (remove user from groups and block from the instance) | 1. Using Just-In-Time (JIT) provisioning, user accounts are created when the user first signs in. - -## Change apps or configuration - -When GitLab doesn't support having multiple providers (such as OAuth), GitLab configuration and user identification must be -updated at the same time if the provider or app is changed. - -These instructions apply to all methods of authentication where GitLab stores an `extern_uid` and it is the only data used -for user authentication. - -When changing apps within a provider, if the user `extern_uid` does not change, only the GitLab configuration must be -updated. - -To swap configurations: - -1. Change provider configuration in your `gitlab.rb` file. -1. Update `extern_uid` for all users that have an identity in GitLab for the previous provider. - -To find the `extern_uid`, look at an existing user's current `extern_uid` for an ID that matches the appropriate field in -your current provider for the same user. - -There are two methods to update the `extern_uid`: - -- Using the [Users API](../../api/users.md#user-modification). Pass the provider name and the new `extern_uid`. -- Using the [Rails console](../operations/rails_console.md): - - ```ruby - Identity.where(extern_uid: 'old-id').update!(extern_uid: 'new-id')` - ``` diff --git a/doc/administration/auth/jwt.md b/doc/administration/auth/jwt.md index 99cba3f220d..71ab084065a 100644 --- a/doc/administration/auth/jwt.md +++ b/doc/administration/auth/jwt.md @@ -70,8 +70,9 @@ JWT provides you with a secret key for you to use. 1. Change `YOUR_APP_SECRET` to the client secret and set `auth_url` to your redirect URL. 1. Save the configuration file. -1. [Reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect if you - installed GitLab via Omnibus or from source respectively. +1. For the changes to take effect: + - If you installed via Omnibus, [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + - If you installed from source, [restart GitLab](../restart_gitlab.md#installations-from-source). On the sign in page there should now be a JWT icon below the regular sign in form. Select the icon to begin the authentication process. JWT asks the user to diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md index 2f0a0db9d6f..19f656d2f14 100644 --- a/doc/administration/auth/ldap/index.md +++ b/doc/administration/auth/ldap/index.md @@ -344,7 +344,7 @@ The `user_filter` DN can contain special characters. For example: OU=GitLab\2C Inc,DC=gitlab,DC=com ``` -- Escape open and close brackets with `\28` and `\29`, respectively. For example: +- Escape open brackets with `\28` and close brackets with `\29`. For example: ```plaintext OU=Gitlab \28Inc\29,DC=gitlab,DC=com diff --git a/doc/administration/auth/ldap/ldap-troubleshooting.md b/doc/administration/auth/ldap/ldap-troubleshooting.md index 64ef27cbf51..a68e6ae2649 100644 --- a/doc/administration/auth/ldap/ldap-troubleshooting.md +++ b/doc/administration/auth/ldap/ldap-troubleshooting.md @@ -5,7 +5,9 @@ group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# LDAP Troubleshooting for Administrators **(FREE SELF)** +# Troubleshooting LDAP **(FREE SELF)** + +If you are an administrator, use the following information to troubleshoot LDAP. ## Common Problems & Workflows @@ -165,7 +167,7 @@ may see the following message: `Access denied for your LDAP account`. We have a workaround, based on toggling the access level of affected users: -1. As an administrator, on the top bar, select **Menu > Admin**. +1. As an administrator, on the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the name of the affected user. 1. In the user's administrative page, press **Edit** on the top right of the page. @@ -210,7 +212,7 @@ This shows you which user has this email address. One of two steps must be taken remove this email as a secondary email and make it a primary one so GitLab associates this profile to the LDAP identity. -The user can do either of these steps +The user can do either of these steps [in their profile](../../../user/profile/index.md#access-your-user-profile) or an administrator can do it. #### Projects limit errors @@ -223,7 +225,7 @@ field contains no data: To resolve this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, go to **Settings > General**. 1. Expand both of the following: - **Account and limit**. @@ -366,7 +368,7 @@ things to debug the situation. - Ensure the correct [LDAP group link is added to the GitLab group](ldap_synchronization.md#add-group-links). - Check that the user has an LDAP identity: 1. Sign in to GitLab as an administrator user. - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Search for the user. 1. Open the user by selecting their name. Do not select **Edit**. @@ -411,6 +413,63 @@ If all the above are true and the users are still not getting access, [look through the output](#example-console-output-after-a-group-sync) to see what happens when GitLab syncs the `admin_group`. +#### Sync now button stuck in the UI + +The **Sync now** button on the **Group > Members** page of a group can become stuck. The button becomes stuck after it is pressed and the page is reloaded. The button then +cannot be selected again. + +The **Sync now** button can become stuck for many reasons and requires debugging for specific cases. The following are two possible causes and possible solutions to the problem. + +##### Invalid memberships + +The **Sync now** button becomes stuck if some of the group's members or requesting members are invalid. You can track progress on improving the visibility of this problem in +a [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/348226). You can use a [Rails console](#rails-console) to confirm if this problem is causing the **Sync now** +button to be stuck: + +```ruby +# Find the group in question +group = Group.find_by(name: 'my_gitlab_group') + +# Look for errors on the Group itself +group.valid? +group.errors.map(&:full_messages) + +# Look for errors among the group's members and requesters +group.requesters.map(&:valid?) +group.requesters.map(&:errors).map(&:full_messages) +group.members.map(&:valid?) +group.members.map(&:errors).map(&:full_messages) +``` + +A displayed error can identify the problem and point to a solution. For example, the Support Team has seen the following error: + +```ruby +irb(main):018:0> group.members.map(&:errors).map(&:full_messages) +=> [["The member's email address is not allowed for this group. Go to the group’s 'Settings > General' page, and check 'Restrict membership by email domain'."]] +``` + +This error showed that an Administrator chose to [restrict group membership by email domain](../../../user/group/access_and_permissions.md#restrict-group-access-by-domain), +but there was a typo in the domain. After the domain setting was fixed, the **Sync now** button functioned again. + +##### Missing LDAP configuration on Sidekiq nodes + +The **Sync now** button becomes stuck when GitLab is scaled over multiple nodes and the LDAP configuration is missing from +[the `/etc/gitlab/gitlab.rb` on the nodes running Sidekiq](../../sidekiq/index.md#configure-ldap-and-user-or-group-synchronization). +In this case, the Sidekiq jobs seem to disappear. + +LDAP is required on the Sidekiq nodes because LDAP has multiple jobs that are +run asynchronously that require a local LDAP configuration: + +- [User sync](ldap_synchronization.md#user-sync). +- [Group sync](ldap_synchronization.md#group-sync). + +You can test whether missing LDAP configuration is the problem by running [the Rake task to check LDAP](#ldap-check) +on each node that is running Sidekiq. If LDAP is set up correctly on this node, it connects to the LDAP server and returns users. + +To solve this issue, [configure LDAP](../../sidekiq/index.md#configure-ldap-and-user-or-group-synchronization) on the Sidekiq nodes. +When configured, run [the Rake task to check LDAP](#ldap-check) to confirm +that the GitLab node can connect to LDAP. + #### Sync all groups NOTE: @@ -430,7 +489,7 @@ Next, [learn how to read the output](#example-console-output-after-a-group-sync) ##### Example console output after a group sync -Like the output from the user sync, the output from the +Like the output from the user sync, the output from the [manual group sync](#sync-all-groups) is also very verbose. However, it contains lots of helpful information. @@ -600,6 +659,66 @@ end You can then [run a UserSync](#sync-all-users) **(PREMIUM SELF)** to sync the latest DN for each of these users. +## Could not authenticate you from ldapmain because "Unknown provider" + +You can receive the following error when authenticating with an LDAP server: + +```plaintext +Could not authenticate you from Ldapmain because "Unknown provider (ldapsecondary). available providers: ["ldapmain"]". +``` + +This error is caused when using an account that previously authenticated with an LDAP server that is renamed or removed from your GitLab configuration. For example: + +- Initially, `main` and `secondary` are set in `ldap_servers` in GitLab configuration. +- The `secondary` setting is removed or renamed to `main`. +- A user attempting to sign in has an `identify` record for `secondary`, but that is no longer configured. + +Use the [Rails console](../../operations/rails_console.md) to list affected users and check what LDAP servers they have identities for: + +```ruby +ldap_identities = Identity.where(provider: "ldapsecondary") +ldap_identities.each do |identity| + u=User.find_by_id(identity.user_id) + ui=Identity.where(user_id: identity.user_id) + puts "user: #{u.username}\n #{u.email}\n last activity: #{u.last_activity_on}\n #{identity.provider} ID: #{identity.id} external: #{identity.extern_uid}" + puts " all identities:" + ui.each do |alli| + puts " - #{alli.provider} ID: #{alli.id} external: #{alli.extern_uid}" + end +end;nil +``` + +You can solve this error in two ways. + +### Rename references to the LDAP server + +This solution is suitable when the LDAP servers are replicas of each other, and the affected users should be able to sign in using a configured LDAP server. For example, if a +load balancer is now used to manage LDAP high availability and a separate secondary sign-in option is no longer needed. + +NOTE: +If the LDAP servers aren't replicas of each other, this solution stops affected users from being able to sign in. + +To [rename references to the LDAP server](../../raketasks/ldap.md#other-options) that is no longer configured, run: + +```shell +sudo gitlab-rake gitlab:ldap:rename_provider[ldapsecondary,ldapmain] +``` + +### Remove the `identity` records that relate to the removed LDAP server + +With this solution, affected users can sign in with the configured LDAP servers and a new `identity` record is created by GitLab. In a +[Rails console](../../operations/rails_console.md), delete the `ldapsecondary` identities: + +```ruby +ldap_identities = Identity.where(provider: "ldapsecondary") +ldap_identities.each do |identity| + puts "Destroying identity: #{identity.id} #{identity.provider}: #{identity.extern_uid}" + identity.destroy! +rescue => e + puts 'Error generated when destroying identity:\n ' + e.to_s +end; nil +``` + ## Expired license causes errors with multiple LDAP servers Using [multiple LDAP servers](index.md#use-multiple-ldap-servers) requires a valid license. An expired license can diff --git a/doc/administration/auth/ldap/ldap_synchronization.md b/doc/administration/auth/ldap/ldap_synchronization.md index 62706a9e3b9..37a27fc058e 100644 --- a/doc/administration/auth/ldap/ldap_synchronization.md +++ b/doc/administration/auth/ldap/ldap_synchronization.md @@ -38,9 +38,11 @@ fail. This means the user cannot sign in or push or pull code. The process also updates the following user information: -- Email address -- SSH public keys (if `sync_ssh_keys` is set) -- Kerberos identity (if Kerberos is enabled) +- Name. Because of a [sync issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342598), `name` is not synchronized if + [**Prevent users from changing their profile name**](../../../user/admin_area/settings/account_and_limit_settings.md#disable-user-profile-name-changes) is enabled. +- Email address. +- SSH public keys if `sync_ssh_keys` is set. +- Kerberos identity if Kerberos is enabled. ### Adjust LDAP user sync schedule @@ -192,7 +194,7 @@ When enabled, the following applies: To enable it, you must: 1. [Configure LDAP](index.md#configure-ldap). -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Ensure the **Lock memberships to LDAP synchronization** checkbox is selected. diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md index 8c5bf96e99e..9f3c96902f8 100644 --- a/doc/administration/auth/oidc.md +++ b/doc/administration/auth/oidc.md @@ -250,7 +250,7 @@ but `LocalAccounts` works for authenticating against local, Active Directory acc <OutputClaim ClaimTypeReferenceId="signInNames.emailAddress" PartnerClaimType="email" /> ``` -1. For OIDC discovery to work with B2C, the policy must be configured with an issuer compatible with the +1. For OIDC discovery to work with B2C, the policy must be configured with an issuer compatible with the [OIDC specification](https://openid.net/specs/openid-connect-discovery-1_0.html#rfc.section.4.3). See the [token compatibility settings](https://docs.microsoft.com/en-us/azure/active-directory-b2c/configure-tokens?pivots=b2c-custom-policy#token-compatibility-settings). In `TrustFrameworkBase.xml` under `JwtIssuer`, set `IssuanceClaimPattern` to `AuthorityWithTfp`: diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index 8c7f8bf766d..573ffbf4686 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/environment_variables.md b/doc/administration/environment_variables.md index 3adb057daa4..e70758e2774 100644 --- a/doc/administration/environment_variables.md +++ b/doc/administration/environment_variables.md @@ -36,6 +36,7 @@ You can use the following environment variables to override certain values: | `GITLAB_SHARED_RUNNERS_REGISTRATION_TOKEN` | string | Sets the initial registration token used for runners. | | `RAILS_ENV` | string | The Rails environment; can be one of `production`, `development`, `staging`, or `test`. | | `UNSTRUCTURED_RAILS_LOG` | string | Enables the unstructured log in addition to JSON logs (defaults to `true`). | +| `GITLAB_RAILS_CACHE_DEFAULT_TTL_SECONDS` | integer | The default TTL used for entries stored in the Rails-cache. Default is `28800`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95042) in 15.3. | ## Adding more variables diff --git a/doc/administration/external_pipeline_validation.md b/doc/administration/external_pipeline_validation.md index 99876cdf503..b9bf0cfdc50 100644 --- a/doc/administration/external_pipeline_validation.md +++ b/doc/administration/external_pipeline_validation.md @@ -44,6 +44,7 @@ required number of seconds. "required" : [ "project", "user", + "credit_card", "pipeline", "builds", "total_builds_count", @@ -85,6 +86,17 @@ required number of seconds. "sign_in_count": { "type": "integer" } } }, + "credit_card": { + "type": "object", + "required": [ + "similar_cards_count", + "similar_holder_names_count" + ], + "properties": { + "similar_cards_count": { "type": "integer" }, + "similar_holder_names_count": { "type": "integer" } + } + }, "pipeline": { "type": "object", "required": [ diff --git a/doc/administration/feature_flags.md b/doc/administration/feature_flags.md index 77f7f621a07..75e65c2a9d6 100644 --- a/doc/administration/feature_flags.md +++ b/doc/administration/feature_flags.md @@ -117,7 +117,7 @@ Some feature flags can be enabled or disabled on a per project basis: Feature.enable(:<feature flag>, Project.find(<project id>)) ``` -For example, to enable the [`:product_analytics`](../operations/product_analytics.md#enable-or-disable-product-analytics) feature flag for project `1234`: +For example, to enable the [`:product_analytics`](../operations/product_analytics.md) feature flag for project `1234`: ```ruby Feature.enable(:product_analytics, Project.find(1234)) @@ -155,6 +155,9 @@ You can view all GitLab administrator set feature flags: ```ruby Feature.all => [#<Flipper::Feature:198220 name="my_awesome_feature", state=:on, enabled_gate_names=[:boolean], adapter=:memoizable>] + +# Nice output +Feature.all.map {|f| [f.name, f.state]} ``` ### Unset feature flag diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md index 97c9a6c5576..d3aa2c97833 100644 --- a/doc/administration/geo/disaster_recovery/background_verification.md +++ b/doc/administration/geo/disaster_recovery/background_verification.md @@ -55,7 +55,7 @@ Feature.enable('geo_repository_verification') On the **primary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. 1. Expand **Verification information** tab for that site to view automatic checksumming status for repositories and wikis. Successes are shown in green, pending work @@ -65,7 +65,7 @@ On the **primary** site: On the **secondary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. 1. Expand **Verification information** tab for that site to view automatic checksumming status for repositories and wikis. Successes are shown in green, pending work @@ -93,7 +93,7 @@ increase load and vice versa. On the **primary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. 1. Select **Edit** for the **primary** site to customize the minimum re-verification interval: @@ -141,7 +141,7 @@ sudo gitlab-rake geo:verification:wiki:reset If the **primary** and **secondary** sites have a checksum verification mismatch, the cause may not be apparent. To find the cause of a checksum mismatch: 1. On the **primary** site: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Projects**. 1. Find the project that you want to check the checksum differences and select its name. diff --git a/doc/administration/geo/disaster_recovery/bring_primary_back.md b/doc/administration/geo/disaster_recovery/bring_primary_back.md index a2d4f35a7c3..1991b747af0 100644 --- a/doc/administration/geo/disaster_recovery/bring_primary_back.md +++ b/doc/administration/geo/disaster_recovery/bring_primary_back.md @@ -41,7 +41,7 @@ To bring the former **primary** site up to date: NOTE: If you [changed the DNS records](index.md#step-4-optional-updating-the-primary-domain-dns-record) - for this site during disaster recovery procedure you may need to + for this site during disaster recovery procedure you may need to [block all the writes to this site](planned_failover.md#prevent-updates-to-the-primary-site) during this procedure. diff --git a/doc/administration/geo/disaster_recovery/index.md b/doc/administration/geo/disaster_recovery/index.md index 0cae94fcec1..e82c130ba01 100644 --- a/doc/administration/geo/disaster_recovery/index.md +++ b/doc/administration/geo/disaster_recovery/index.md @@ -93,7 +93,7 @@ Note the following when promoting a secondary: the **secondary** to the **primary**. - If you encounter an `ActiveRecord::RecordInvalid: Validation failed: Name has already been taken` error message during this process, for more information, see this - [troubleshooting advice](../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node). + [troubleshooting advice](../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-site). - If you run into errors when using `--force` or `--skip-preflight-checks` before 13.5 during this process, for more information, see this [troubleshooting advice](../replication/troubleshooting.md#errors-when-using---skip-preflight-checks-or---force). @@ -774,4 +774,4 @@ If you are running GitLab 14.4 and earlier: ## Troubleshooting -This section was moved to [another location](../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node). +This section was moved to [another location](../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-site). diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md index 6c3353e7d7e..d0dbecce43a 100644 --- a/doc/administration/geo/disaster_recovery/planned_failover.md +++ b/doc/administration/geo/disaster_recovery/planned_failover.md @@ -149,7 +149,7 @@ ensure these processes are close to 100% as possible during active use. On the **secondary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. Replicated objects (shown in green) should be close to 100%, and there should be no failures (shown in red). If a large proportion of @@ -177,7 +177,7 @@ This [content was moved to another location](background_verification.md). On the **primary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Messages**. 1. Add a message notifying users on the maintenance window. You can check under **Geo > Sites** to estimate how long it @@ -190,7 +190,7 @@ To ensure that all data is replicated to a secondary site, updates (write reques be disabled on the **primary** site: 1. Enable [maintenance mode](../../maintenance_mode/index.md) on the **primary** site. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Cron**. 1. Select `Disable All` to disable non-Geo periodic background jobs. @@ -206,7 +206,7 @@ GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary 1. If you are manually replicating any data not managed by Geo, trigger the final replication process now. 1. On the **primary** site: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all queues except those with `geo` in the name to drop to 0. @@ -221,7 +221,7 @@ GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary - The Geo log cursor is up to date (0 events behind). 1. On the **secondary** site: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all the `geo` queues to drop to 0 queued and 0 running jobs. diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md index 46b2ccddefd..11baf383c67 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md @@ -68,7 +68,7 @@ GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary On the **secondary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites** to see its status. Replicated objects (shown in green) should be close to 100%, and there should be no failures (shown in red). If a large proportion of @@ -133,7 +133,7 @@ follow these steps to avoid unnecessary data loss: connection. 1. On the **primary** site: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Cron**. 1. Select `Disable All` to disable any non-Geo periodic background jobs. @@ -151,7 +151,7 @@ follow these steps to avoid unnecessary data loss: [data not managed by Geo](../../replication/datatypes.md#limitations-on-replicationverification), trigger the final replication process now. 1. On the **primary** site: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all queues except those with `geo` in the name to drop to 0. @@ -166,7 +166,7 @@ follow these steps to avoid unnecessary data loss: - The Geo log cursor is up to date (0 events behind). 1. On the **secondary** site: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all the `geo` queues to drop to 0 queued and 0 running jobs. @@ -274,7 +274,7 @@ the **secondary** to the **primary**. WARNING: If you encounter an `ActiveRecord::RecordInvalid: Validation failed: Name has already been taken` error during this process, read -[the troubleshooting advice](../../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node). +[the troubleshooting advice](../../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-site). The `gitlab-ctl promote-to-primary-node` command cannot be used yet in conjunction with multiple servers, as it can only diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md index da26023e4f9..2958c119c20 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md @@ -118,7 +118,7 @@ follow these steps to avoid unnecessary data loss: connection. 1. On the **primary** site: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Cron**. 1. Select `Disable All` to disable any non-Geo periodic background jobs. @@ -136,7 +136,7 @@ follow these steps to avoid unnecessary data loss: [data not managed by Geo](../../replication/datatypes.md#limitations-on-replicationverification), trigger the final replication process now. 1. On the **primary** site: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all queues except those with `geo` in the name to drop to 0. @@ -151,7 +151,7 @@ follow these steps to avoid unnecessary data loss: - The Geo log cursor is up to date (0 events behind). 1. On the **secondary** site: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all the `geo` queues to drop to 0 queued and 0 running jobs. @@ -220,7 +220,7 @@ Note the following when promoting a secondary: the **secondary** to the **primary**. - If you encounter an `ActiveRecord::RecordInvalid: Validation failed: Name has already been taken` error during this process, read - [the troubleshooting advice](../../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node). + [the troubleshooting advice](../../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-site). To promote the secondary site running GitLab 14.5 and later: diff --git a/doc/administration/geo/index.md b/doc/administration/geo/index.md index e3bf5ccdfe7..db298d4fdfc 100644 --- a/doc/administration/geo/index.md +++ b/doc/administration/geo/index.md @@ -47,7 +47,7 @@ In addition, it: Geo provides: - Read-only **secondary** sites: Maintain one **primary** GitLab site while still enabling read-only **secondary** sites for each of your distributed teams. -- Authentication system hooks: **Secondary** sites receives all authentication data (like user accounts and logins) from the **primary** instance. +- Authentication system hooks: **Secondary** sites receive all authentication data (like user accounts and logins) from the **primary** instance. - An intuitive UI: **Secondary** sites use the same web interface your team has grown accustomed to. In addition, there are visual notifications that block write operations and make it clear that a user is on a **secondary** sites. ### Gitaly Cluster @@ -124,6 +124,7 @@ The following are required to run Geo: - Git 2.9 or later - Git-lfs 2.4.2 or later on the user side when using LFS - All sites must run [the same GitLab and PostgreSQL versions](setup/database.md#postgresql-replication). +- If using different operating system versions between Geo sites, [check OS locale data compatibility](replication/troubleshooting.md#check-os-locale-data-compatibility) across Geo sites. Additionally, check the GitLab [minimum requirements](../../install/requirements.md), and we recommend you use the latest version of GitLab for a better experience. @@ -158,7 +159,7 @@ public URL of the primary site is used. To update the internal URL of the primary Geo site: -1. On the top bar, go to **Menu > Admin > Geo > Sites**. +1. On the top bar, select **Main menu > Admin > Geo > Sites**. 1. Select **Edit** on the primary site. 1. Change the **Internal URL**, then select **Save changes**. @@ -204,6 +205,7 @@ This list of limitations only reflects the latest version of GitLab. If you are - GitLab Runners cannot register with a **secondary** site. Support for this is [planned for the future](https://gitlab.com/gitlab-org/gitlab/-/issues/3294). - [Selective synchronization](replication/configuration.md#selective-synchronization) only limits what repositories and files are replicated. The entire PostgreSQL data is still replicated. Selective synchronization is not built to accommodate compliance / export control use cases. - [Pages access control](../../user/project/pages/pages_access_control.md) doesn't work on secondaries. See [GitLab issue #9336](https://gitlab.com/gitlab-org/gitlab/-/issues/9336) for details. +- [GitLab chart with Geo](https://docs.gitlab.com/charts/advanced/geo/) does not support [Unified URLs](secondary_proxy/index.md#set-up-a-unified-url-for-geo-sites). See [GitLab issue #3522](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/3522) for more details. ### Limitations on replication/verification diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index 7666a450648..39d1f5ae602 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -202,7 +202,7 @@ keys must be manually replicated to the **secondary** site. ``` 1. Navigate to the Primary Node GitLab Instance: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. 1. Select **Add site**.  @@ -222,19 +222,23 @@ keys must be manually replicated to the **secondary** site. gitlab-ctl restart ``` - Check if there are any common issue with your Geo setup by running: + Check if there are any common issues with your Geo setup by running: ```shell gitlab-rake gitlab:geo:check ``` + If any of the checks fail, check the [troubleshooting documentation](troubleshooting.md). + 1. SSH into a **Rails or Sidekiq server on your primary** site and login as root to verify the - **secondary** site is reachable or there are any common issue with your Geo setup: + **secondary** site is reachable or there are any common issues with your Geo setup: ```shell gitlab-rake gitlab:geo:check ``` + If any of the checks fail, check the [troubleshooting documentation](troubleshooting.md). + Once added to the Geo administration page and restarted, the **secondary** site automatically starts replicating missing data from the **primary** site in a process known as **backfill**. Meanwhile, the **primary** site starts to notify each **secondary** site of any changes, so @@ -305,7 +309,7 @@ method to be enabled. This is enabled by default, but if converting an existing On the **primary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility and access controls**. 1. Ensure "Enabled Git access protocols" is set to either "Both SSH and HTTP(S)" or "Only HTTP(S)". @@ -315,7 +319,7 @@ On the **primary** site: You can sign in to the **secondary** site with the same credentials you used with the **primary** site. After you sign in: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. 1. Verify that it's correctly identified as a **secondary** Geo site, and that Geo is enabled. diff --git a/doc/administration/geo/replication/container_registry.md b/doc/administration/geo/replication/container_registry.md index b425e5dcc0d..01ba81b6dbe 100644 --- a/doc/administration/geo/replication/container_registry.md +++ b/doc/administration/geo/replication/container_registry.md @@ -12,7 +12,7 @@ You can set up a Container Registry on your **secondary** Geo site that mirrors ## Supported container registries Geo supports the following types of container registries: - + - [Docker](https://docs.docker.com/registry/) - [OCI](https://github.com/opencontainers/distribution-spec/blob/main/spec.md) @@ -26,7 +26,7 @@ The following container image formats are support by Geo: In addition, Geo also supports [BuildKit cache images](https://github.com/moby/buildkit). -## Supported storage +## Supported storage ### Docker @@ -34,7 +34,7 @@ For more information on supported registry storage drivers see [Docker registry storage drivers](https://docs.docker.com/registry/storage-drivers/) Read the [Load balancing considerations](https://docs.docker.com/registry/deploying/#load-balancing-considerations) -when deploying the Registry, and how to set up the storage driver for the GitLab integrated +when deploying the Registry, and how to set up the storage driver for the GitLab integrated [Container Registry](../../packages/container_registry.md#use-object-storage). ### Registries that support OCI artifacts @@ -160,7 +160,7 @@ For each application and Sidekiq node on the **secondary** site: To verify Container Registry replication is working, on the **secondary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. The initial replication, or "backfill", is probably still in progress. diff --git a/doc/administration/geo/replication/disable_geo.md b/doc/administration/geo/replication/disable_geo.md index f0658ae45a2..84bc2e034b9 100644 --- a/doc/administration/geo/replication/disable_geo.md +++ b/doc/administration/geo/replication/disable_geo.md @@ -36,7 +36,7 @@ to do that. To remove the **primary** site: 1. [Remove all secondary Geo sites](remove_geo_site.md) -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. 1. Select **Remove** for the **primary** node. 1. Confirm by selecting **Remove** when the prompt appears. diff --git a/doc/administration/geo/replication/object_storage.md b/doc/administration/geo/replication/object_storage.md index d2e10678f8c..0336a1669f9 100644 --- a/doc/administration/geo/replication/object_storage.md +++ b/doc/administration/geo/replication/object_storage.md @@ -41,7 +41,7 @@ whether they are stored on the local file system or in object storage. To enable GitLab replication: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. 1. Select **Edit** on the **secondary** site. 1. In the **Synchronization Settings** section, find the **Allow this secondary node to replicate content on Object Storage** diff --git a/doc/administration/geo/replication/remove_geo_site.md b/doc/administration/geo/replication/remove_geo_site.md index 0d6715a93b7..b136f6cc8b8 100644 --- a/doc/administration/geo/replication/remove_geo_site.md +++ b/doc/administration/geo/replication/remove_geo_site.md @@ -9,7 +9,7 @@ type: howto **Secondary** sites can be removed from the Geo cluster using the Geo administration page of the **primary** site. To remove a **secondary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. 1. Select the **Remove** button for the **secondary** site you want to remove. 1. Confirm by selecting **Remove** when the prompt appears. diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 26d192f62cd..d64ad2549e8 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -19,24 +19,24 @@ Here is a list of steps you should take to attempt to fix problem: Before attempting more advanced troubleshooting: -- Check [the health of the **secondary** node](#check-the-health-of-the-secondary-node). +- Check [the health of the **secondary** site](#check-the-health-of-the-secondary-site). - Check [if PostgreSQL replication is working](#check-if-postgresql-replication-is-working). -### Check the health of the **secondary** node +### Check the health of the **secondary** site -On the **primary** node: +On the **primary** site: -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Geo > Nodes**. +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Geo > Sites**. -We perform the following health checks on each **secondary** node +We perform the following health checks on each **secondary** site to help identify if something is wrong: -- Is the node running? -- Is the node's secondary database configured for streaming replication? -- Is the node's secondary tracking database configured? -- Is the node's secondary tracking database connected? -- Is the node's secondary tracking database up-to-date? +- Is the site running? +- Is the secondary site's database configured for streaming replication? +- Is the secondary site's tracking database configured? +- Is the secondary site's tracking database connected? +- Is the secondary site's tracking database up-to-date?  @@ -48,8 +48,8 @@ health check manually to get this information and a few more details. #### Health check Rake task -This Rake task can be run on an app node in the **primary** or **secondary** -Geo nodes: +This Rake task can be run on a **Rails** node in the **primary** or **secondary** +Geo sites: ```shell sudo gitlab-rake gitlab:geo:check @@ -275,11 +275,11 @@ sudo gitlab-rake gitlab:geo:check Checking Geo ... Finished ``` - Ensure you have added the secondary node in the Admin Area of the **primary** node. - Also ensure you entered the `external_url` or `gitlab_rails['geo_node_name']` - when adding the secondary node in the Admin Area of the **primary** node. - In GitLab 12.3 and earlier, edit the secondary node in the Admin Area of the **primary** - node and ensure that there is a trailing `/` in the `Name` field. + Ensure you have added the secondary site in the **Main menu > Admin > Geo > Sites** on the web interface for the **primary** site. + Also ensure you entered the `gitlab_rails['geo_node_name']` + when adding the secondary site in the Admin Area of the **primary** site. + In GitLab 12.3 and earlier, edit the secondary site in the Admin Area of the **primary** + site and ensure that there is a trailing `/` in the `Name` field. - Check returns `Exception: PG::UndefinedTable: ERROR: relation "geo_nodes" does not exist`. @@ -321,7 +321,7 @@ error messages (indicated by `Database replication working? ... no` in the This means that the `max_replication_slots` PostgreSQL variable needs to be set on the **primary** database. This setting defaults to 1. You may need to -increase this value if you have more **secondary** nodes. +increase this value if you have more **secondary** sites. Be sure to restart PostgreSQL for this to take effect. See the [PostgreSQL replication setup](../setup/database.md#postgresql-replication) guide for more details. @@ -329,13 +329,13 @@ Be sure to restart PostgreSQL for this to take effect. See the ### Message: `FATAL: could not start WAL streaming: ERROR: replication slot "geo_secondary_my_domain_com" does not exist`? This occurs when PostgreSQL does not have a replication slot for the -**secondary** node by that name. +**secondary** site by that name. -You may want to rerun the [replication process](../setup/database.md) on the **secondary** node . +You may want to rerun the [replication process](../setup/database.md) on the **secondary** site . ### Message: "Command exceeded allowed execution time" when setting up replication? -This may happen while [initiating the replication process](../setup/database.md#step-3-initiate-the-replication-process) on the **secondary** node, +This may happen while [initiating the replication process](../setup/database.md#step-3-initiate-the-replication-process) on the **secondary** site, and indicates your initial dataset is too large to be replicated in the default timeout (30 minutes). Re-run `gitlab-ctl replicate-geo-database`, but include a larger value for @@ -374,8 +374,8 @@ log data to build up in `pg_xlog`. Removing the unused slots can reduce the amou Slots where `active` is `f` are not active. -- When this slot should be active, because you have a **secondary** node configured using that slot, - sign in to that **secondary** node and check the [PostgreSQL logs](../../logs/index.md#postgresql-logs) +- When this slot should be active, because you have a **secondary** site configured using that slot, + sign in on the web interface for the **secondary** site and check the [PostgreSQL logs](../../logs/index.md#postgresql-logs) to view why the replication is not running. - If you are no longer using the slot (for example, you no longer have Geo enabled), you can remove it with in the @@ -398,12 +398,12 @@ These long-running queries are [planned to be removed in the future](https://gitlab.com/gitlab-org/gitlab/-/issues/34269), but as a workaround, we recommend enabling [`hot_standby_feedback`](https://www.postgresql.org/docs/10/hot-standby.html#HOT-STANDBY-CONFLICT). -This increases the likelihood of bloat on the **primary** node as it prevents +This increases the likelihood of bloat on the **primary** site as it prevents `VACUUM` from removing recently-dead rows. However, it has been used successfully in production on GitLab.com. To enable `hot_standby_feedback`, add the following to `/etc/gitlab/gitlab.rb` -on the **secondary** node: +on the **secondary** site: ```ruby postgresql['hot_standby_feedback'] = 'on' @@ -463,14 +463,14 @@ This happens if data is detected in the `projects` table. When one or more proje is aborted to prevent accidental data loss. To bypass this message, pass the `--force` option to the command. In GitLab 13.4, a seed project is added when GitLab is first installed. This makes it necessary to pass `--force` even -on a new Geo secondary node. There is an [issue to account for seed projects](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5618) +on a new Geo secondary site. There is an [issue to account for seed projects](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5618) when checking the database. ### Message: `Synchronization failed - Error syncing repository` WARNING: If large repositories are affected by this problem, -their resync may take a long time and cause significant load on your Geo nodes, +their resync may take a long time and cause significant load on your Geo sites, storage and network systems. If you see the error message `Synchronization failed - Error syncing repository` along with `fatal: fsck error in packed object`, this indicates @@ -483,7 +483,7 @@ it's possible to override the consistency checks instead. To do that, follow [the instructions in the Gitaly docs](../../gitaly/configure_gitaly.md#repository-consistency-checks). You can also get the error message `Synchronization failed - Error syncing repository` along with the following log messages, this indicates that the expected `geo` remote is not present in the `.git/config` file -of a repository on the secondary Geo node's file system: +of a repository on the secondary Geo site's file system: ```json { @@ -505,7 +505,7 @@ of a repository on the secondary Geo node's file system: To solve this: -1. Sign in to the secondary Geo node. +1. Sign in on the web interface for the secondary Geo site. 1. Back up [the `.git` folder](../../repository_storage_types.md#translate-hashed-storage-paths). @@ -538,7 +538,7 @@ To solve this: end ``` -### Very large repositories never successfully synchronize on the **secondary** node +### Very large repositories never successfully synchronize on the **secondary** site GitLab places a timeout on all repository clones, including project imports and Geo synchronization operations. If a fresh `git clone` of a repository @@ -546,7 +546,8 @@ on the **primary** takes more than the default three hours, you may be affected To increase the timeout: -1. On the **secondary** node, add the following line to `/etc/gitlab/gitlab.rb`: +1. On the **Sidekiq nodes on your secondary** site, +add the following line to `/etc/gitlab/gitlab.rb`: ```ruby gitlab_rails['gitlab_shell_git_timeout'] = 14400 @@ -563,9 +564,9 @@ long enough to accommodate a full clone of your largest repositories. ### New LFS objects are never replicated -If new LFS objects are never replicated to secondary Geo nodes, check the version of +If new LFS objects are never replicated to secondary Geo sites, check the version of GitLab you are running. GitLab versions 11.11.x or 12.0.x are affected by -[a bug that results in new LFS objects not being replicated to Geo secondary nodes](https://gitlab.com/gitlab-org/gitlab/-/issues/32696). +[a bug that results in new LFS objects not being replicated to Geo secondary sites](https://gitlab.com/gitlab-org/gitlab/-/issues/32696). To resolve the issue, upgrade to GitLab 12.1 or later. @@ -574,9 +575,9 @@ To resolve the issue, upgrade to GitLab 12.1 or later. During a [backfill](../index.md#backfill), failures are scheduled to be retried at the end of the backfill queue, therefore these failures only clear up **after** the backfill completes. -### Resetting Geo **secondary** node replication +### Resetting Geo **secondary** site replication -If you get a **secondary** node in a broken state and want to reset the replication state, +If you get a **secondary** site in a broken state and want to reset the replication state, to start again from scratch, there are a few steps that can help you: 1. Stop Sidekiq and the Geo LogCursor. @@ -617,8 +618,8 @@ to start again from scratch, there are a few steps that can help you: 1. Optional. Rename other data folders and create new ones. WARNING: - You may still have files on the **secondary** node that have been removed from the **primary** node, but this - removal has not been reflected. If you skip this step, these files are not removed from the Geo node. + You may still have files on the **secondary** site that have been removed from the **primary** site, but this + removal has not been reflected. If you skip this step, these files are not removed from the Geo **secondary** site. Any uploaded content (like file attachments, avatars, or LFS objects) is stored in a subfolder in one of these paths: @@ -667,7 +668,7 @@ to start again from scratch, there are a few steps that can help you: ### Design repository failures on mirrored projects and project imports -On the top bar, under **Menu > Admin > Geo > Nodes**, +On the top bar, under **Main menu > Admin > Geo > Sites**, if the Design repositories progress bar shows `Synced` and `Failed` greater than 100%, and negative `Queued`, the instance is likely affected by @@ -714,7 +715,7 @@ Counts: {"synced"=>3} ``` -#### If you are promoting a Geo secondary site running on a single server +#### If you are promoting a Geo secondary site running on a single node `gitlab-ctl promotion-preflight-checks` fails due to the existence of `failed` rows in the `geo_design_registry` table. Use the @@ -831,10 +832,10 @@ We recommend transferring each failing repository individually and checking for after each transfer. Follow the [single target `rsync` instructions](../../operations/moving_repositories.md#single-rsync-to-another-server) to transfer each affected repository from the primary to the secondary site. -## Fixing errors during a failover or when promoting a secondary to a primary node +## Fixing errors during a failover or when promoting a secondary to a primary site The following are possible error messages that might be encountered during failover or -when promoting a secondary to a primary node with strategies to resolve them. +when promoting a secondary to a primary site with strategies to resolve them. ### Message: `ActiveRecord::RecordInvalid: Validation failed: Name has already been taken` @@ -868,14 +869,14 @@ or `gitlab-ctl promote-to-primary-node`, either: ``` - Upgrade to GitLab 12.6.3 or later if it is safe to do so. For example, - if the failover was just a test. A + if the failover was just a test. A [caching-related bug](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/22021) was fixed. ### Message: `ActiveRecord::RecordInvalid: Validation failed: Enabled Geo primary node cannot be disabled` -If you disabled a secondary node, either with the [replication pause task](../index.md#pausing-and-resuming-replication) +If you disabled a secondary site, either with the [replication pause task](../index.md#pausing-and-resuming-replication) (GitLab 13.2) or by using the user interface (GitLab 13.1 and earlier), you must first -re-enable the node before you can continue. This is fixed in GitLab 13.4. +re-enable the site before you can continue. This is fixed in GitLab 13.4. This can be fixed in the database. @@ -894,7 +895,7 @@ This can be fixed in the database. ``` 1. Run the following command, replacing `https://<secondary url>/` with the URL - for your secondary server. You can use either `http` or `https`, but ensure that you + for your secondary node. You can use either `http` or `https`, but ensure that you end the URL with a slash (`/`): ```sql @@ -987,32 +988,31 @@ sudo gitlab-rake geo:set_secondary_as_primary ## Expired artifacts If you notice for some reason there are more artifacts on the Geo -secondary node than on the Geo primary node, you can use the Rake task +**secondary** site than on the Geo **primary** site, you can use the Rake task to [cleanup orphan artifact files](../../../raketasks/cleanup.md#remove-orphan-artifact-files). -On a Geo **secondary** node, this command also cleans up all Geo +On a Geo **secondary** site, this command also cleans up all Geo registry record related to the orphan files on disk. ## Fixing sign in errors ### Message: The redirect URI included is not valid -If you are able to sign in to the **primary** node, but you receive this error message -when attempting to sign in to a **secondary**, you should verify the Geo -node's URL matches its external URL. +If you are able to sign in to the web interface for the **primary** site, but you receive this error message +when attempting to sign in to a **secondary** web interface, you should verify the Geo +site's URL matches its external URL. -On the **primary** node: +On the **primary** site: -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Geo > Nodes**. +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Geo > Sites**. 1. Find the affected **secondary** site and select **Edit**. 1. Ensure the **URL** field matches the value found in `/etc/gitlab/gitlab.rb` - in `external_url "https://gitlab.example.com"` on the frontend servers of - the **secondary** node. + in `external_url "https://gitlab.example.com"` on the **Rails nodes of the secondary** site. ## Fixing common errors -This section documents common error messages reported in the Admin Area, and how to fix them. +This section documents common error messages reported in the Admin Area on the web interface, and how to fix them. ### Geo database configuration file is missing @@ -1029,11 +1029,11 @@ has the correct permissions. Geo cannot reuse an existing tracking database. It is safest to use a fresh secondary, or reset the whole secondary by following -[Resetting Geo secondary node replication](#resetting-geo-secondary-node-replication). +[Resetting Geo secondary site replication](#resetting-geo-secondary-site-replication). -### Geo node has a database that is writable which is an indication it is not configured for replication with the primary node +### Geo site has a database that is writable which is an indication it is not configured for replication with the primary site -This error message refers to a problem with the database replica on a **secondary** node, +This error message refers to a problem with the database replica on a **secondary** site, which Geo expects to have access to. It usually means, either: - An unsupported replication method was used (for example, logical replication). @@ -1043,24 +1043,24 @@ which Geo expects to have access to. It usually means, either: Geo **secondary** sites require two separate PostgreSQL instances: -- A read-only replica of the **primary** node. +- A read-only replica of the **primary** site. - A regular, writable instance that holds replication metadata. That is, the Geo tracking database. This error message indicates that the replica database in the **secondary** site is misconfigured and replication has stopped. To restore the database and resume replication, you can do one of the following: -- [Reset the Geo secondary site replication](#resetting-geo-secondary-node-replication). +- [Reset the Geo secondary site replication](#resetting-geo-secondary-site-replication). - [Set up a new secondary Geo Omnibus instance](../setup/index.md#using-omnibus-gitlab). If you set up a new secondary from scratch, you must also [remove the old site from the Geo cluster](remove_geo_site.md#removing-secondary-geo-sites). -### Geo node does not appear to be replicating the database from the primary node +### Geo site does not appear to be replicating the database from the primary site The most common problems that prevent the database from replicating correctly are: -- **Secondary** nodes cannot reach the **primary** node. Check credentials, firewall rules, and so on. -- SSL certificate problems. Make sure you copied `/etc/gitlab/gitlab-secrets.json` from the **primary** node. +- **Secondary** sites cannot reach the **primary** site. Check credentials, [firewall rules](../index.md#firewall-rules), and so on. +- SSL certificate problems. Make sure you copied `/etc/gitlab/gitlab-secrets.json` from the **primary** site. - Database storage disk is full. - Database replication slot is misconfigured. - Database is not using a replication slot or another alternative and cannot catch-up because WAL files were purged. @@ -1072,26 +1072,26 @@ Make sure you follow the [Geo database replication](../setup/database.md) instru If you are using Omnibus GitLab installation, something might have failed during upgrade. You can: - Run `sudo gitlab-ctl reconfigure`. -- Manually trigger the database migration by running: `sudo gitlab-rake db:migrate:geo` as root on the **secondary** node. +- Manually trigger the database migration by running: `sudo gitlab-rake db:migrate:geo` as root on the **secondary** site. ### GitLab indicates that more than 100% of repositories were synced This can be caused by orphaned records in the project registry. You can clear them [using a Rake task](../../../administration/raketasks/geo.md#remove-orphaned-project-registries). -### Geo Admin Area returns 404 error for a secondary node +### Geo Admin Area returns 404 error for a secondary site -Sometimes `sudo gitlab-rake gitlab:geo:check` indicates that the **secondary** node is -healthy, but a 404 Not Found error message for the **secondary** node is returned in the Geo Admin Area on -the **primary** node. +Sometimes `sudo gitlab-rake gitlab:geo:check` indicates that **Rails nodes of the secondary** sites are +healthy, but a 404 Not Found error message for the **secondary** site is returned in the Geo Admin Area on the web interface for +the **primary** site. To resolve this issue: -- Try restarting the **secondary** using `sudo gitlab-ctl restart`. -- Check `/var/log/gitlab/gitlab-rails/geo.log` to see if the **secondary** node is - using IPv6 to send its status to the **primary** node. If it is, add an entry to - the **primary** node using IPv4 in the `/etc/hosts` file. Alternatively, you should - [enable IPv6 on the **primary** node](https://docs.gitlab.com/omnibus/settings/nginx.html#setting-the-nginx-listen-address-or-addresses). +- Try restarting **each Rails, Sidekiq and Gitaly nodes on your secondary site** using `sudo gitlab-ctl restart`. +- Check `/var/log/gitlab/gitlab-rails/geo.log` on Sidekiq nodes to see if the **secondary** site is + using IPv6 to send its status to the **primary** site. If it is, add an entry to + the **primary** site using IPv4 in the `/etc/hosts` file. Alternatively, you should + [enable IPv6 on the **primary** site](https://docs.gitlab.com/omnibus/settings/nginx.html#setting-the-nginx-listen-address-or-addresses). ### Secondary site returns 502 errors with Geo proxying @@ -1167,7 +1167,7 @@ To fix this issue, set the primary site's internal URL to a URL that is: You may have problems if you're running a version of [Git LFS](https://git-lfs.github.com/) before 2.4.2. As noted in [this authentication issue](https://github.com/git-lfs/git-lfs/issues/3025), -requests redirected from the secondary to the primary node do not properly send the +requests redirected from the secondary to the primary site do not properly send the Authorization header. This may result in either an infinite `Authorization <-> Redirect` loop, or Authorization error messages. @@ -1194,13 +1194,13 @@ The partial failover to a secondary Geo *site* may be the result of a temporary/ 1. SSH into every Sidekiq, PostgresSQL, Gitaly, and Rails node in the **secondary** site and run one of the following commands: - - To promote the secondary node to primary: + - To promote the secondary site to primary: ```shell sudo gitlab-ctl geo promote ``` - - To promote the secondary node to primary **without any further confirmation**: + - To promote the secondary site to primary **without any further confirmation**: ```shell sudo gitlab-ctl geo promote --force @@ -1230,3 +1230,37 @@ If the above steps are **not successful**, proceed through the next steps: 1. Verify you can connect to the newly-promoted **primary** site using the URL used previously for the **secondary** site. 1. If successful, the **secondary** site is now promoted to the **primary** site. + +## Additional tools + +There are useful snippets for manipulating Geo internals in the [GitLab Rails Cheat Sheet](../../troubleshooting/gitlab_rails_cheat_sheet.md#geo). For example, you can find how to manually sync or verify a replicable in Rails console. + +## Check OS locale data compatibility + +If different operating systems or different operating system versions are deployed across Geo sites, we recommend that you perform a locale data compatibility check setting up Geo. + +Geo uses PostgreSQL and Streaming Replication to replicate data across Geo sites. PostgreSQL uses locale data provided by the operating system’s C library for sorting text. If the locale data in the C library is incompatible across Geo sites, erroneous query results that lead to [incorrect behavior on secondary sites](https://gitlab.com/gitlab-org/gitlab/-/issues/360723). See [here](https://wiki.postgresql.org/wiki/Locale_data_changes) for more details. + +On all hosts running PostgreSQL, across all Geo sites, run the following shell command: + +```shell +( echo "1-1"; echo "11" ) | LC_COLLATE=en_US.UTF-8 sort +``` + +The output will either look like: + +```plaintext +1-1 +11 +``` + +or the reverse order: + +```plaintext +11 +1-1 +``` + +If the output is identical on all hosts, then they running compatible versions of locale data. + +If the output differs on some hosts, then PostgreSQL replication will not work properly. We advise that you select operating system versions that are compatible. diff --git a/doc/administration/geo/replication/tuning.md b/doc/administration/geo/replication/tuning.md index 755ab45a76c..370c50c93db 100644 --- a/doc/administration/geo/replication/tuning.md +++ b/doc/administration/geo/replication/tuning.md @@ -14,7 +14,7 @@ in the background. On the **primary** site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. 1. Select **Edit** of the secondary site you want to tune. 1. Under **Tuning settings**, there are several variables that can be tuned to diff --git a/doc/administration/geo/replication/version_specific_upgrades.md b/doc/administration/geo/replication/version_specific_upgrades.md index 350310c7076..6211b5689b1 100644 --- a/doc/administration/geo/replication/version_specific_upgrades.md +++ b/doc/administration/geo/replication/version_specific_upgrades.md @@ -157,6 +157,13 @@ to perform the following additional steps for the zero-downtime upgrade: sudo gitlab-rake db:migrate ``` +1. Hot reload `puma` and `sidekiq` services: + + ```shell + sudo gitlab-ctl hup puma + sudo gitlab-ctl restart sidekiq + ``` + If you have already run the final `sudo gitlab-rake db:migrate` command on the deploy node and have encountered the [column rename issue](https://gitlab.com/gitlab-org/gitlab/-/issues/324160), you might see the following error: @@ -183,8 +190,8 @@ GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab each upgraded reference. Delay any upgrade attempts until this is in the [13.7.5 patch release.](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/3002). More details are available [in this issue](https://gitlab.com/gitlab-org/git/-/issues/79). -- A new secret is generated in `/etc/gitlab/gitlab-secrets.json`. - In an HA GitLab or GitLab Geo environment, secrets need to be the same on all nodes. +- A new secret is generated in `/etc/gitlab/gitlab-secrets.json`. + In an HA GitLab or GitLab Geo environment, secrets need to be the same on all nodes. Ensure this new secret is also accounted for if you are manually syncing the file across nodes, or manually specifying secrets in `/etc/gitlab/gitlab.rb`. @@ -247,7 +254,7 @@ the recommended procedure, see the ## Upgrading to GitLab 12.9 WARNING: -GitLab 12.9.0 through GitLab 12.9.3 are affected by +GitLab 12.9.0 through GitLab 12.9.3 are affected by [a bug that stops repository verification](https://gitlab.com/gitlab-org/gitlab/-/issues/213523). The issue is fixed in GitLab 12.9.4. Upgrade to GitLab 12.9.4 or later. @@ -401,6 +408,6 @@ For the recommended procedure, see the ## Upgrading to GitLab 12.0 WARNING: -This version is affected by a +This version is affected by a [bug that results in new LFS objects not being replicated to Geo secondary sites](https://gitlab.com/gitlab-org/gitlab/-/issues/32696). The issue is fixed in GitLab 12.1. Be sure to upgrade to GitLab 12.1 or later. diff --git a/doc/administration/geo/secondary_proxy/index.md b/doc/administration/geo/secondary_proxy/index.md index 6c1812b2754..731b5012663 100644 --- a/doc/administration/geo/secondary_proxy/index.md +++ b/doc/administration/geo/secondary_proxy/index.md @@ -12,11 +12,6 @@ type: howto > - [Disabled by default for different URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/325732) in GitLab 14.6 [with a flag](../../feature_flags.md) named `geo_secondary_proxy_separate_urls`. > - [Enabled by default for different URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/346112) in GitLab 15.1. -FLAG: -On self-managed GitLab, this feature is only available by default for Geo sites using a unified URL. See below to -[set up a unified URL for Geo sites](#set-up-a-unified-url-for-geo-sites). -The feature is not ready for production use with separate URLs. - Use Geo proxying to: - Have secondary sites serve read-write traffic by proxying to the primary site. @@ -110,10 +105,13 @@ gitlab: ## Geo proxying with Separate URLs -Since GitLab 15.1, Geo secondary proxying is enabled by default for separate URLs also. +> Geo secondary proxying for separate URLs is [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/346112) in GitLab 15.1. + +NOTE: +The feature flag described in this section is planned to be deprecated and removed in a future release. Support for read-only Geo secondary sites is proposed in [issue 366810](https://gitlab.com/gitlab-org/gitlab/-/issues/366810), you can upvote and share your use cases in that issue. -There are minor known issues linked in the -["Geo secondary proxying with separate URLs" epic](https://gitlab.com/groups/gitlab-org/-/epics/6865). +There are minor known issues linked in the +["Geo secondary proxying with separate URLs" epic](https://gitlab.com/groups/gitlab-org/-/epics/6865). You can also add feedback in the epic about any use-cases that are not possible anymore with proxying enabled. diff --git a/doc/administration/gitaly/configure_gitaly.md b/doc/administration/gitaly/configure_gitaly.md index 5b868c274cd..ac03c3ffc02 100644 --- a/doc/administration/gitaly/configure_gitaly.md +++ b/doc/administration/gitaly/configure_gitaly.md @@ -1089,9 +1089,10 @@ lots of CI fetch traffic. The pack-objects cache wraps `git pack-objects`, an internal part of Git that gets invoked indirectly via the PostUploadPack and -SSHUploadPack Gitaly RPCs. These are the RPCs that Gitaly runs when a -user does a Git fetch via HTTP or SSH, respectively. When the cache is -enabled, anything that uses PostUploadPack or SSHUploadPack can +SSHUploadPack Gitaly RPCs. Gitaly runs PostUploadPack when a +user does a Git fetch via HTTP, or SSHUploadPack when a +user does a Git fetch via SSH. +When the cache is enabled, anything that uses PostUploadPack or SSHUploadPack can benefit from it. It is orthogonal to: - The transport (HTTP or SSH). diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index b053da7ac9b..c7f7c4c58a5 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -72,7 +72,7 @@ the current status of these issues, please refer to the referenced issues and ep |:--------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------| | Gitaly Cluster + Geo - Issues retrying failed syncs | If Gitaly Cluster is used on a Geo secondary site, repositories that have failed to sync could continue to fail when Geo tries to resync them. Recovering from this state requires assistance from support to run manual steps. | No known solution prior to GitLab 15.0. In GitLab 15.0 to 15.2, enable the [`gitaly_praefect_generated_replica_paths` feature flag](#praefect-generated-replica-paths-gitlab-150-and-later). In GitLab 15.3, the feature flag is enabled by default. | | Praefect unable to insert data into the database due to migrations not being applied after an upgrade | If the database is not kept up to date with completed migrations, then the Praefect node is unable to perform normal operation. | Make sure the Praefect database is up and running with all migrations completed (For example: `/opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-migrate-status` should show a list of all applied migrations). Consider [requesting upgrade assistance](https://about.gitlab.com/support/scheduling-upgrade-assistance/) so your upgrade plan can be reviewed by support. | -| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup, it's best to snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. | +| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup, it's best to [shut down GitLab](../read_only_gitlab.md#shut-down-the-gitlab-ui), snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. | ### Snapshot backup and recovery limitations @@ -598,6 +598,7 @@ To migrate to Gitaly Cluster: 1. Create the required storage. Refer to [repository storage recommendations](praefect.md#repository-storage-recommendations). 1. Create and configure [Gitaly Cluster](praefect.md). +1. Configure the existing Gitaly instance [to use TPC](praefect.md#use-tcp-for-existing-gitlab-instances), if not already configured that way. 1. [Move the repositories](../operations/moving_repositories.md#move-repositories). To migrate to Gitaly Cluster, existing repositories stored outside Gitaly Cluster must be moved. There is no automatic migration but the moves can be scheduled with the GitLab API. @@ -660,7 +661,14 @@ The code removed from GitLab during the Gitaly migration project affected these performance workaround for these NFS-based deployments, we re-introduced some of the old Rugged code. This re-introduced code is informally referred to as the "Rugged patches". -### How it works +### Automatic detection + +> Automatic detection for Rugged [disabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95445) in GitLab 15.3. + +FLAG: +On self-managed GitLab, by default automatic detection of whether Rugged should be used (per storage) is not available. +To make it available, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named +`skip_rugged_auto_detect`. The Ruby methods that perform direct Git access are behind [feature flags](../../development/gitaly.md#legacy-rugged-code), disabled by default. It wasn't @@ -685,12 +693,13 @@ To see if GitLab can access the repository file system directly, we use the foll - GitLab Rails tries to read the metadata file directly. If it exists, and if the UUID's match, assume we have direct access. -Versions of GitLab 15.3 and later disable direct Git access by default. +Direct Git access is: -For versions of GitLab prior to 15.3, direct Git access is enabled by -default in Omnibus GitLab because it fills in the correct repository -paths in the GitLab configuration file `config/gitlab.yml`. This -satisfies the UUID check. +- [Disabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95445) by default in GitLab 15.3 and later for + compatibility with [Praefect-generated replica paths](#praefect-generated-replica-paths-gitlab-150-and-later). It + can be enabled if Rugged [feature flags](../../development/gitaly.md#legacy-rugged-code) are enabled. +- Enabled by default in GitLab 15.2 and earlier because it fills in the correct repository paths in the GitLab + configuration file `config/gitlab.yml`. This satisfies the UUID check. ### Transition to Gitaly Cluster diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index 488e6a0df5f..bd03aa1bdbc 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -97,7 +97,7 @@ If you [installed](https://about.gitlab.com/install/) GitLab using the Omnibus G ### Preparation -Before beginning, you should already have a working GitLab instance. +Before beginning, you should already have a working GitLab instance. [Learn how to install GitLab](https://about.gitlab.com/install/). Provision a PostgreSQL server. We recommend using the PostgreSQL that is shipped @@ -164,7 +164,8 @@ The replication state is internal to each instance of GitLab and should not be replicated. These instructions help set up a single PostgreSQL database, which creates a single point of -failure. Alternatively, [you can use PostgreSQL replication and failover](../postgresql/replication_and_failover.md). +failure. To avoid this, you can configure your own clustered PostgreSQL. Support for PostgreSQL replication and failover using Omnibus GitLab is being tracked in +[a relevant epic](https://gitlab.com/groups/gitlab-org/-/epics/7814). The following options are available: @@ -221,7 +222,7 @@ For using Omnibus-provided PgBouncer you need to take the following additional s recommend using the PostgreSQL that is shipped with Omnibus as the backend. The following instructions only work on Omnibus-provided PostgreSQL: -1. For Omnibus-provided PgBouncer, you need to use the hash of `praefect` user instead the of the +1. For Omnibus-provided PgBouncer, you need to use the hash of `praefect` password instead the of the actual password: ```sql @@ -331,7 +332,7 @@ To configure the additional connection, you must either: #### Configure a new PgBouncer database with `pool_mode = session` We recommend using PgBouncer with `session` pool mode. You can use the -[bundled PgBouncer](../postgresql/pgbouncer.md) or use an external PgBouncer and +[bundled PgBouncer](../postgresql/pgbouncer.md) or use an external PgBouncer and [configure it manually](https://www.pgbouncer.org/config.html). The following example uses the bundled PgBouncer and sets up two separate connection pools on PostgreSQL host, @@ -620,7 +621,7 @@ Updates to example must be made at: gitlab-ctl reconfigure ``` -1. To ensure that Praefect +1. To ensure that Praefect [has updated its Prometheus listen address](https://gitlab.com/gitlab-org/gitaly/-/issues/2734), [restart Praefect](../restart_gitlab.md#omnibus-gitlab-restart): @@ -884,7 +885,7 @@ For more information on Gitaly server configuration, see our gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN' ``` -1. Configure and `internal_api_url`, which is also needed for `git push` operations: +1. Configure an `internal_api_url`, which is also needed for `git push` operations: ```ruby # Configure the gitlab-shell API callback URL. Without this, `git push` will @@ -928,7 +929,7 @@ For more information on Gitaly server configuration, see our gitlab-ctl reconfigure ``` -1. To ensure that Gitaly +1. To ensure that Gitaly [has updated its Prometheus listen address](https://gitlab.com/gitlab-org/gitaly/-/issues/2734), [restart Gitaly](../restart_gitlab.md#omnibus-gitlab-restart): @@ -1114,7 +1115,7 @@ Particular attention should be shown to: 1. Check that the Praefect storage is configured to store new repositories: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository**. 1. Expand the **Repository storage** section. diff --git a/doc/administration/gitaly/recovery.md b/doc/administration/gitaly/recovery.md index bd4846a986d..4bbf25d7cdd 100644 --- a/doc/administration/gitaly/recovery.md +++ b/doc/administration/gitaly/recovery.md @@ -430,6 +430,44 @@ This command fails if: - The repository is already being tracked by the Praefect database. - The repository does not exist on disk. +### Manually track large numbers of repositories + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/6319) in GitLab 15.4. + +The `track-repositories` Praefect sub-command adds large batches of on-disk repositories to the Praefect database for tracking. This can +be useful when migrating an existing instance to new infrastructure and ingesting all existing repositories into a fresh Gitaly Cluster. + +```shell +# Omnibus GitLab install +sudo gitlab-ctl praefect track-repositories --input-path /path/to/input.json + +# Source install +sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml track-repositories -input-path /path/to/input.json +``` + +The command validates that all entries: + +- Are formatted correctly and contain required fields. +- Correspond to a valid Git repository on disk. +- Are not currently tracked in the Praefect database. + +If any entry fails these checks, the command aborts prior to attempting to track a repository. + +- `input-path` is the path to a file containing a list of repositories formatted as newline-delimited JSON objects. Objects must contain the following keys: + - `relative_path`: corresponds with `repository` in [track-repositories](#manually-track-repositories). + - `authoritative-storage`: the storage Praefect is to treat as the primary. + - `virtual-storage`: the virtual storage the repository is located in. + + For example: + + ```json + {"relative_path":"@hashed/f5/ca/f5ca38f748a1d6eaf726b8a42fb575c3c71f1864a8143301782de13da2d9202b.git","authoritative_storage":"gitaly-1","virtual_storage":"default"} + {"relative_path":"@hashed/f8/9f/f89f8d0e735a91c5269ab08d72fa27670d000e7561698d6e664e7b603f5c4e40.git","authoritative_storage":"gitaly-2","virtual_storage":"default"} + ``` + +- `-replicate-immediately`, causes the command to replicate the repository to its secondaries immediately. + Otherwise, replication jobs are scheduled for execution in the database and are picked up by a Praefect background process. + ### List virtual storage details > [Introduced](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4609) in GitLab 15.1. diff --git a/doc/administration/gitaly/troubleshooting.md b/doc/administration/gitaly/troubleshooting.md index e308fa9da43..285ec3d2631 100644 --- a/doc/administration/gitaly/troubleshooting.md +++ b/doc/administration/gitaly/troubleshooting.md @@ -20,7 +20,7 @@ and our advice on [parsing the `gitaly/current` file](../logs/log_parsing.md#par When using standalone Gitaly servers, you must make sure they are the same version as GitLab to ensure full compatibility: -1. On the top bar, select **Menu > Admin** on your GitLab instance. +1. On the top bar, select **Main menu > Admin** on your GitLab instance. 1. On the left sidebar, select **Overview > Gitaly Servers**. 1. Confirm all Gitaly servers indicate that they are up to date. diff --git a/doc/administration/housekeeping.md b/doc/administration/housekeeping.md index 31b7ac9fd3e..cb0156f8e2d 100644 --- a/doc/administration/housekeeping.md +++ b/doc/administration/housekeeping.md @@ -27,7 +27,7 @@ GitLab automatically runs `git gc` and `git repack` on repositories after Git pu You can change how often this happens or turn it off: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Repository maintenance**. 1. In the **Housekeeping** section, configure the [housekeeping options](#housekeeping-options). diff --git a/doc/administration/inactive_project_deletion.md b/doc/administration/inactive_project_deletion.md index 224b52d420e..824a444fdd2 100644 --- a/doc/administration/inactive_project_deletion.md +++ b/doc/administration/inactive_project_deletion.md @@ -1,23 +1,21 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Inactive project deletion **(FREE SELF)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85689) in GitLab 15.0 [with a flag](../administration/feature_flags.md) named `inactive_projects_deletion`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to -[enable the feature flag](../administration/feature_flags.md) named `inactive_projects_deletion`. -On GitLab.com, this feature is not available. This feature is not ready for production use. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85689) in GitLab 15.0 [with a flag](../administration/feature_flags.md) named `inactive_projects_deletion`. Disabled by default. +> - [Feature flag `inactive_projects_deletion`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96803) removed in GitLab 15.4. Administrators of large GitLab instances can find that over time, projects become inactive and are no longer used. These projects take up unnecessary disk space. With inactive project deletion, you can identify these projects, warn the maintainers ahead of time, and then delete the projects if they remain inactive. When an inactive project is deleted, the action generates an audit event that it was performed by the first active administrator. +For the default setting on GitLab.com, see the [GitLab.com settings page](../user/gitlab_com/index.md#inactive-project-deletion). + ## Configure inactive project deletion You can configure inactive projects deletion or turn it off using either: @@ -62,7 +60,7 @@ You can use the [Application settings API](../api/settings.md#change-application To configure inactive projects with the GitLab UI: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Repository maintenance**. 1. In the **Inactive project deletion** section, configure the necessary options. diff --git a/doc/administration/index.md b/doc/administration/index.md index 3f2ae3170ab..58284a74bf7 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -82,7 +82,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. #### Customizing GitLab appearance -- [Header logo](../user/admin_area/appearance.md#navigation-bar): Change the logo on all pages and email headers. +- [Header logo](../user/admin_area/appearance.md#top-bar): Change the logo on all pages and email headers. - [Favicon](../user/admin_area/appearance.md#favicon): Change the default favicon to your own logo. - [Branded login page](../user/admin_area/appearance.md#sign-in--sign-up-pages): Customize the login page with your own logo, title, and description. - ["New Project" page](../user/admin_area/appearance.md#new-project-pages): Customize the text to be displayed on the page that opens whenever your users create a new project. diff --git a/doc/administration/instance_limits.md b/doc/administration/instance_limits.md index 31434a8ee6f..0cf2e3a1131 100644 --- a/doc/administration/instance_limits.md +++ b/doc/administration/instance_limits.md @@ -158,13 +158,19 @@ Set the limit to `0` to disable it. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80631) in GitLab 14.9. -This setting limits global search requests. +This setting limits global search requests as follows: | Limit | Default (requests per minute) | |-------------------------|-------------------------------| | Authenticated user | 30 | | Unauthenticated user | 10 | +Depending on the number of enabled [scopes](../user/search/advanced_search.md#global-search-scopes), a global search request can consume two to seven requests per minute. You may want to disable one or more scopes to use fewer requests. Global search requests that exceed the search rate limit per minute return the following error: + +```plaintext +This endpoint has been requested too many times. Try again later. +``` + ### Pipeline creation rate limit > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362475) in GitLab 15.0. @@ -581,7 +587,7 @@ This limit is [enabled on GitLab.com](../user/gitlab_com/index.md#gitlab-cicd). The total number of instance level CI/CD variables is limited at the instance level. This limit is checked each time a new instance level variable is created. If a new variable -would cause the total number of variables to exceed the limit, the new variable is created. +would cause the total number of variables to exceed the limit, the new variable is not created. On self-managed instances this limit is defined for the `default` plan. By default, this limit is set to `25`. @@ -648,7 +654,7 @@ installation, run the following in the [GitLab Rails console](operations/rails_c Plan.default.actual_limits.update!(ci_max_artifact_size_junit: 10) ``` -### Number of files per GitLab Pages web-site +### Number of files per GitLab Pages website The total number of file entries (including directories and symlinks) is limited to `200,000` per GitLab Pages website. @@ -663,6 +669,14 @@ For example, to change the limit to `100`: Plan.default.actual_limits.update!(pages_file_entries: 100) ``` +### Number of custom domains per GitLab Pages website + +The total number of custom domains per GitLab Pages website is limited to `150` for [GitLab SaaS](../subscriptions/gitlab_com/index.md). + +The default limit for [GitLab self-managed](../subscriptions/self_managed/index.md) is `0` (unlimited). +To set a limit on your self-managed instance, use the +[Admin Area](pages/index.md#set-maximum-number-of-gitlab-pages-custom-domains-for-a-project). + ### Number of registered runners per scope > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321368) in GitLab 13.12. Disabled by default. @@ -736,7 +750,7 @@ You can change these limits in the [GitLab Rails console](operations/rails_conso - To update the maximum YAML size, update `max_yaml_size_bytes` with the new value in megabytes: ```ruby - ApplicationSetting.update!(max_yaml_size_bytes: 2.megabytes) + ApplicationSetting.update(max_yaml_size_bytes: 2.megabytes) ``` The `max_yaml_size_bytes` value is not directly tied to the size of the YAML file, @@ -745,7 +759,7 @@ You can change these limits in the [GitLab Rails console](operations/rails_conso - To update the maximum YAML depth, update `max_yaml_depth` with the new value in megabytes: ```ruby - ApplicationSetting.update!(max_yaml_depth: 125) + ApplicationSetting.update(max_yaml_depth: 125) ``` ### Limit dotenv variables diff --git a/doc/administration/integration/kroki.md b/doc/administration/integration/kroki.md index a6fcfe6c80f..fb4659175b0 100644 --- a/doc/administration/integration/kroki.md +++ b/doc/administration/integration/kroki.md @@ -56,7 +56,7 @@ read the [Kroki installation](https://docs.kroki.io/kroki/setup/install/#_images You need to enable Kroki integration from Settings under Admin Area. To do that, log in with an administrator account and follow these steps: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. Go to **Settings > General**. 1. Expand the **Kroki** section. 1. Select **Enable Kroki** checkbox. diff --git a/doc/administration/integration/mailgun.md b/doc/administration/integration/mailgun.md index c007116d213..37e81f220cf 100644 --- a/doc/administration/integration/mailgun.md +++ b/doc/administration/integration/mailgun.md @@ -43,7 +43,7 @@ After configuring your Mailgun domain for the webhook endpoints, you're ready to enable the Mailgun integration: 1. Sign in to GitLab as an [Administrator](../../user/permissions.md) user. -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Main menu >** **{admin}** **Admin**. 1. On the left sidebar, go to **Settings > General** and expand the **Mailgun** section. 1. Select the **Enable Mailgun** check box. 1. Enter the Mailgun HTTP webhook signing key as described in diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index 58c9e01a151..de790c7ce40 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -211,7 +211,7 @@ stop; After configuring your local PlantUML server, you're ready to enable the PlantUML integration: 1. Sign in to GitLab as an [Administrator](../../user/permissions.md) user. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, go to **Settings > General** and expand the **PlantUML** section. 1. Select the **Enable PlantUML** checkbox. 1. Set the PlantUML instance as `https://gitlab.example.com/-/plantuml/`, diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 88b320941de..41984bbe7a7 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -1,6 +1,6 @@ --- -stage: Ecosystem -group: Integrations +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -114,7 +114,7 @@ they receive a `Connection failed` message. By default, terminal sessions do not expire. To limit the terminal session lifetime in your GitLab instance: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. Select [**Settings > Web terminal**](../../user/admin_area/settings/index.md#general). 1. Set a `max session time`. diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index a4ab0e07020..14749a9c7f6 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -437,6 +437,16 @@ If you need to manually remove job artifacts associated with multiple jobs while ```ruby builds_to_clear = builds_with_artifacts.where("finished_at < ?", 1.week.ago) builds_to_clear.find_each do |build| + Ci::JobArtifacts::DeleteService.new(build).execute + build.update!(artifacts_expire_at: Time.now) + end + ``` + + In [GitLab 15.3 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/372537), use the following instead: + + ```ruby + builds_to_clear = builds_with_artifacts.where("finished_at < ?", 1.week.ago) + builds_to_clear.find_each do |build| build.artifacts_expire_at = Time.now build.erase_erasable_artifacts! end @@ -489,7 +499,7 @@ If you need to manually remove **all** job artifacts associated with multiple jo print "Ci::Build ID #{build.id}... " if build.erasable? - build.erase(erased_by: admin_user) + Ci::BuildEraseService.new(build, admin_user).execute puts "Erased" else puts "Skipped (Nothing to erase or not erasable)" @@ -497,6 +507,9 @@ If you need to manually remove **all** job artifacts associated with multiple jo end ``` + In [GitLab 15.3 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/369132), replace + `Ci::BuildEraseService.new(build, admin_user).execute` with `build.erase(erased_by: admin_user)`. + `1.week.ago` is a Rails `ActiveSupport::Duration` method which calculates a new date or time in the past. Other valid examples are: diff --git a/doc/administration/job_logs.md b/doc/administration/job_logs.md index d2837bfa96e..84da4e31d92 100644 --- a/doc/administration/job_logs.md +++ b/doc/administration/job_logs.md @@ -134,8 +134,8 @@ For more information, see [delete references to missing artifacts](raketasks/che ## Incremental logging architecture -> - [Deployed behind a feature flag](../user/feature_flags.md), disabled by default. -> - Enabled on GitLab.com. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/18169) in GitLab 10.8 [with a flag](feature_flags.md) named `ci_enable_live_trace`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/241471) in GitLab 13.6. > - [Recommended for production use](https://gitlab.com/groups/gitlab-org/-/epics/4275) in GitLab 13.6. > - [Recommended for production use with AWS S3](https://gitlab.com/gitlab-org/gitlab/-/issues/273498) in GitLab 13.7. > - To use in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-incremental-logging). diff --git a/doc/administration/load_balancer.md b/doc/administration/load_balancer.md index 988bddcfe62..87b63c6272d 100644 --- a/doc/administration/load_balancer.md +++ b/doc/administration/load_balancer.md @@ -31,7 +31,7 @@ Configure your load balancers to pass connections on port 443 as 'TCP' rather than 'HTTP(S)' protocol. This passes the connection to the application nodes NGINX service untouched. NGINX has the SSL certificate and listen on port 443. -See [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Load Balancers terminate SSL without backend SSL @@ -41,8 +41,8 @@ The load balancers is be responsible for managing SSL certificates and terminating SSL. Because communication between the load balancers and GitLab isn't secure, -there is some additional configuration needed. See -[NGINX Proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) +there is some additional configuration needed. See the +[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination) for details. ### Load Balancers terminate SSL with backend SSL @@ -55,7 +55,7 @@ Traffic is secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL because the connection is secure all the way. However, configuration must be added to GitLab to configure SSL certificates. See -[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ## Ports @@ -115,14 +115,18 @@ Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`. It is strongly recommend that multi-node deployments configure load balancers to use the [readiness check](../user/admin_area/monitoring/health_check.md#readiness) to ensure a node is ready to accept traffic, before routing traffic to it. This is especially important when utilizing Puma, as there is a brief period during a restart where Puma doesn't accept requests. -<!-- ## Troubleshooting +## Troubleshooting -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### The health check is returning a `408` HTTP code via the load balancer -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +If you are using [AWS's Classic Load Balancer](https://docs.aws.amazon.com/en_en/elasticloadbalancing/latest/classic/elb-ssl-security-policy.html#ssl-ciphers) +in GitLab 15.0 or later, you must to enable the `AES256-GCM-SHA384` cipher in NGINX. +See [AES256-GCM-SHA384 SSL cipher no longer allowed by default by NGINX](https://docs.gitlab.com/omnibus/update/gitlab_15_changes.html#aes256-gcm-sha384-ssl-cipher-no-longer-allowed-by-default-by-nginx) +for more information. + +The default ciphers for a GitLab version can be +viewed in the [`files/gitlab-cookbooks/gitlab/attributes/default.rb`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb) +file and selecting the Git tag that correlates with your target GitLab version +(for example `15.0.5+ee.0`). If required by your load balancer, you can then define +[custom SSL ciphers](https://docs.gitlab.com/omnibus/settings/ssl.html#use-custom-ssl-ciphers) +for NGINX. diff --git a/doc/administration/maintenance_mode/index.md b/doc/administration/maintenance_mode/index.md index 00e28e650ea..60de8e2fd3a 100644 --- a/doc/administration/maintenance_mode/index.md +++ b/doc/administration/maintenance_mode/index.md @@ -21,7 +21,7 @@ Maintenance Mode allows most external actions that do not change internal state. There are three ways to enable Maintenance Mode as an administrator: - **Web UI**: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Maintenance Mode**, and toggle **Enable Maintenance Mode**. You can optionally add a message for the banner as well. @@ -45,7 +45,7 @@ There are three ways to enable Maintenance Mode as an administrator: There are three ways to disable Maintenance Mode: - **Web UI**: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Maintenance Mode**, and toggle **Enable Maintenance Mode**. You can optionally add a message for the banner as well. @@ -173,7 +173,7 @@ it is recommended that you disable all cron jobs except for those related to Geo To monitor queues and disable jobs: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. ### Incident management diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md index 2553638291d..4b62a8ae931 100644 --- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md +++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md @@ -38,12 +38,12 @@ This project can be used to: ## Create the self monitoring project -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling** and expand **Self monitoring**. 1. Toggle **Self monitoring** on. 1. After your GitLab instance creates the project, GitLab displays a link to the project in the text above the **Self monitoring** toggle. You can also find it - from the top bar by selecting **Menu > Project**, then selecting **Your projects**. + from the top bar by selecting **Main menu > Projects**. ## Delete the self monitoring project @@ -51,7 +51,7 @@ WARNING: Deleting the self monitoring project removes any changes made to the project. If you create the project again, it's created in its default state. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, go to **Settings > Metrics and profiling** and expand **Self monitoring**. 1. Toggle **Self monitoring** off. 1. In the confirmation dialog that opens, select **Delete self monitoring project**. diff --git a/doc/administration/monitoring/performance/gitlab_configuration.md b/doc/administration/monitoring/performance/gitlab_configuration.md index 128ddad6555..74db35eebc2 100644 --- a/doc/administration/monitoring/performance/gitlab_configuration.md +++ b/doc/administration/monitoring/performance/gitlab_configuration.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w GitLab Performance Monitoring is disabled by default. To enable it and change any of its settings: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling** (`/admin/application_settings/metrics_and_profiling`). 1. Add the necessary configuration changes. diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md index 3a76e2e4578..6e9ea0d8d42 100644 --- a/doc/administration/monitoring/performance/grafana_configuration.md +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -62,7 +62,7 @@ repository. After setting up Grafana, you can enable a link to access it easily from the GitLab sidebar: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling** and expand **Metrics - Grafana**. 1. Select the **Add a link to Grafana** checkbox. @@ -72,14 +72,14 @@ GitLab sidebar: - *Otherwise,* enter the full URL of the Grafana instance. 1. Select **Save changes**. -GitLab displays your link in the **Menu > Admin > Monitoring > Metrics Dashboard**. +GitLab displays your link in the **Main menu > Admin > Monitoring > Metrics Dashboard**. ## Required Scopes > [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5822) in GitLab 13.10. When setting up Grafana through the process above, no scope shows in the screen at -**Menu > Admin > Applications > GitLab Grafana**. However, the `read_user` scope is +**Main menu > Admin > Applications > GitLab Grafana**. However, the `read_user` scope is required and is provided to the application automatically. Setting any scope other than `read_user` without also including `read_user` leads to this error when you try to log in using GitLab as the OAuth provider: diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index 759f485c109..c23046158e1 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -108,7 +108,7 @@ The performance bar is disabled by default for non-administrators. To enable it for a given group: 1. Sign in as a user with administrator access. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling** (`admin/application_settings/metrics_and_profiling`), and expand **Profiling - Performance bar**. diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index a2def8a9f64..00dae8e4dd5 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w To enable the GitLab Prometheus metrics: 1. Log in to GitLab as a user with administrator access. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Find the **Metrics - Prometheus** section, and select **Add link to Prometheus**. 1. [Restart GitLab](../../restart_gitlab.md#omnibus-gitlab-restart) for the changes to take effect. @@ -146,6 +146,8 @@ The following metrics are available: | `gitlab_snowplow_failed_events_total` | Counter | 14.1 | Total number of GitLab Snowplow product intelligence events emission failures | | | `gitlab_snowplow_successful_events_total` | Counter | 14.1 | Total number of GitLab Snowplow product intelligence events emission successes | | | `gitlab_ci_build_trace_errors_total` | Counter | 14.4 | Total amount of different error types on a build trace | `error_reason` | +| `gitlab_presentable_object_cacheless_render_real_duration_seconds` | Histogram | 15.3 | Duration of real time spent caching and representing specific web request objects | `controller`, `action` | +| `cached_object_operations_total` | Counter | 15.3 | Total number of objects cached for specific web requests | `controller`, `action` | ## Metrics controlled by a feature flag @@ -390,7 +392,7 @@ Some basic Ruby runtime metrics are available: ## Redis metrics These client metrics are meant to complement Redis server metrics. -These metrics are broken down per +These metrics are broken down per [Redis instance](https://docs.gitlab.com/omnibus/settings/redis.html#running-with-multiple-redis-instances). These metrics all have a `storage` label which indicates the Redis instance (`cache`, `shared_state`, and so on). diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index 6f6ac5c5d4b..c4aa607fa4d 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -277,11 +277,6 @@ To use an external Prometheus server: static_configs: - targets: - 1.1.1.1:9168 - - job_name: gitlab_exporter_process - metrics_path: "/process" - static_configs: - - targets: - - 1.1.1.1:9168 - job_name: gitaly static_configs: - targets: diff --git a/doc/administration/nfs.md b/doc/administration/nfs.md index 29f00fec3f7..9967cce5b73 100644 --- a/doc/administration/nfs.md +++ b/doc/administration/nfs.md @@ -95,34 +95,22 @@ We are investigating the use of ## Improving NFS performance with GitLab NFS performance with GitLab can in some cases be improved with -[direct Git access](gitaly/index.md#direct-access-to-git-in-gitlab) using -[Rugged](https://github.com/libgit2/rugged). +[direct Git access](gitaly/index.md#direct-access-to-git-in-gitlab) using [Rugged](https://github.com/libgit2/rugged). -Versions of GitLab after 12.2 and prior to 15.3 automatically detect if -Rugged can and should be used per storage. +Depending on the GitLab version, GitLab [automatically detects](gitaly/index.md#automatic-detection) if Rugged can and should +be used per storage. -NOTE: -GitLab 15.3 and later disables this automatic detection. Auto-detection can be enabled via the -`skip_rugged_auto_detect` feature flag: - -```ruby -Feature.disable(:skip_rugged_auto_detect) -``` - -In addition, if you previously enabled Rugged using the feature flag and -you want to use automatic detection instead, you must unset the feature -flag: +If the Rugged feature flag is explicitly set to either `true` or `false`, GitLab uses the value explicitly set. If you +previously enabled Rugged using the feature flag and you want to use automatic detection instead, you must unset +the feature flag: ```shell sudo gitlab-rake gitlab:features:unset_rugged ``` -If the Rugged feature flag is explicitly set to either `true` or `false`, GitLab uses the value explicitly set. - -From GitLab 12.7, Rugged is only automatically enabled for use with Puma -if the [Puma thread count is set to `1`](../install/requirements.md#puma-settings). - -To use Rugged with a Puma thread count of more than `1`, enable Rugged using the [feature flag](../development/gitaly.md#legacy-rugged-code). +From GitLab 12.7, Rugged is only automatically enabled for use with Puma if the +[Puma thread count is set to `1`](../install/requirements.md#puma-settings). To use Rugged with a Puma thread count of +more than `1`, enable Rugged using the [feature flag](../development/gitaly.md#legacy-rugged-code). ## NFS server diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md index 0299d5f8b0c..fd9ab9b5972 100644 --- a/doc/administration/object_storage.md +++ b/doc/administration/object_storage.md @@ -26,7 +26,7 @@ GitLab has been tested by vendors and customers on a number of object storage pr ### Known compatibility issues -- Dell EMC ECS: Prior to GitLab 13.3, there is a +- Dell EMC ECS: Prior to GitLab 13.3, there is a [known bug in GitLab Workhorse that prevents HTTP Range Requests from working with CI job artifacts](https://gitlab.com/gitlab-org/gitlab/-/issues/223806). Be sure to upgrade to GitLab 13.3.0 or above if you use S3 storage with this hardware. @@ -528,7 +528,7 @@ supported by consolidated configuration form, refer to the following guides: | Object storage type | Supported by consolidated configuration? | |---------------------|------------------------------------------| -| [Backups](../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | **{dotted-circle}** No | +| [Backups](../raketasks/backup_gitlab.md#upload-backups-to-a-remote-cloud-storage) | **{dotted-circle}** No | | [Job artifacts](job_artifacts.md#using-object-storage) including archived job logs | **{check-circle}** Yes | | [LFS objects](lfs/index.md#storing-lfs-objects-in-remote-object-storage) | **{check-circle}** Yes | | [Uploads](uploads.md#using-object-storage) | **{check-circle}** Yes | @@ -578,7 +578,7 @@ real bucket into multiple virtual buckets. If your object storage bucket is called `my-gitlab-objects` you can configure uploads to go into `my-gitlab-objects/uploads`, artifacts into `my-gitlab-objects/artifacts`, etc. The application will act as if -these are separate buckets. Note that use of bucket prefixes +these are separate buckets. Note that use of bucket prefixes [may not work correctly with Helm backups](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/3376). Helm-based installs require separate buckets to @@ -693,7 +693,7 @@ configuration. When configured either with an instance profile or with the consolidated object configuration, GitLab Workhorse properly uploads files to S3 buckets that have [SSE-S3 or SSE-KMS encryption enabled by default](https://docs.aws.amazon.com/kms/latest/developerguide/services-s3.html). -Customer master keys (CMKs) and SSE-C encryption are +Customer master keys (CMKs) and SSE-C encryption are [not supported since this requires sending the encryption keys in every request](https://gitlab.com/gitlab-org/gitlab/-/issues/226006). ##### Server-side encryption headers @@ -701,7 +701,7 @@ Customer master keys (CMKs) and SSE-C encryption are > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38240) in GitLab 13.3. Setting a default encryption on an S3 bucket is the easiest way to -enable encryption, but you may want to +enable encryption, but you may want to [set a bucket policy to ensure only encrypted objects are uploaded](https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-store-kms-encrypted-objects/). To do this, you must configure GitLab to send the proper encryption headers in the `storage_options` configuration section: diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index a3240a6041b..8523b881730 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -106,7 +106,7 @@ users as long as a large file exists. To disable writes to the `authorized_keys` file: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Performance optimization**. 1. Clear the **Use authorized_keys file to authenticate SSH keys** checkbox. @@ -125,7 +125,7 @@ This overview is brief. Refer to the above instructions for more context. 1. [Rebuild the `authorized_keys` file](../raketasks/maintenance.md#rebuild-authorized_keys-file). 1. Enable writes to the `authorized_keys` file. - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Performance optimization**. 1. Select the **Use authorized_keys file to authenticate SSH keys** checkbox. diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index a6e66abdbdb..179958c6df1 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -18,7 +18,7 @@ Keep your GitLab instance up and running smoothly. - [Multiple Sidekiq processes](extra_sidekiq_processes.md): Configure multiple Sidekiq processes to ensure certain queues always have dedicated workers, no matter the number of jobs that must be processed. **(FREE SELF)** - [Sidekiq routing rules](extra_sidekiq_routing.md): Configure the routing rules to route a job from a worker to a desirable queue. **(FREE SELF)** - [Puma](puma.md): Understand Puma and puma-worker-killer. -- Speed up SSH operations by +- Speed up SSH operations by [Authorizing SSH users via a fast, indexed lookup to the GitLab database](fast_ssh_key_lookup.md), and/or by [doing away with user SSH keys stored on GitLab entirely in favor of SSH certificates](ssh_certificates.md). - [File System Performance Benchmarking](filesystem_benchmarking.md): File system diff --git a/doc/administration/operations/puma.md b/doc/administration/operations/puma.md index 5ce469d3e63..8e7594dfc2d 100644 --- a/doc/administration/operations/puma.md +++ b/doc/administration/operations/puma.md @@ -156,10 +156,8 @@ For deployments where NFS is used to store Git repositories, GitLab uses [direct Git access](../gitaly/index.md#direct-access-to-git-in-gitlab) to improve performance by using [Rugged](https://github.com/libgit2/rugged). -Rugged usage is automatically enabled if direct Git access -[is available](../gitaly/index.md#how-it-works) -and Puma is running single threaded, unless it is disabled by a -[feature flag](../../development/gitaly.md#legacy-rugged-code). +Rugged usage is automatically enabled if direct Git access [is available](../gitaly/index.md#automatic-detection) and +Puma is running single threaded, unless it is disabled by a [feature flag](../../development/gitaly.md#legacy-rugged-code). MRI Ruby uses a Global VM Lock (GVL). GVL allows MRI Ruby to be multi-threaded, but running at most on a single core. diff --git a/doc/administration/operations/rails_console.md b/doc/administration/operations/rails_console.md index 430dfbc637c..627dfbeb66c 100644 --- a/doc/administration/operations/rails_console.md +++ b/doc/administration/operations/rails_console.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Rails console **(FREE SELF)** -At the heart of GitLab is a web application +At the heart of GitLab is a web application [built using the Ruby on Rails framework](https://about.gitlab.com/blog/2018/10/29/why-we-use-rails-to-build-gitlab/). The [Rails console](https://guides.rubyonrails.org/command_line.html#rails-console). provides a way to interact with your GitLab instance from the command line, and also grants access to the amazing tools built right into Rails. @@ -19,7 +19,7 @@ with no consequences, you are strongly advised to do so in a test environment. The Rails console is for GitLab system administrators who are troubleshooting a problem or need to retrieve some data that can only be done through direct -access of the GitLab application. Basic knowledge of Ruby is needed (try +access of the GitLab application. Basic knowledge of Ruby is needed (try [this 30-minute tutorial](https://try.ruby-lang.org/) for a quick introduction). Rails experience is useful but not required. @@ -136,7 +136,7 @@ root 1 ``` -Some basic knowledge of Ruby will be very useful. Try +Some basic knowledge of Ruby will be very useful. Try [this 30-minute tutorial](https://try.ruby-lang.org/) for a quick introduction. Rails experience is helpful but not essential. @@ -202,6 +202,44 @@ sudo chmod 700 /scripts sudo gitlab-rails runner /scripts/helloworld.rb ``` +## Find specific methods for an object + +```ruby +Array.methods.select { |m| m.to_s.include? "sing" } +Array.methods.grep(/sing/) +``` + +## Find method source + +```ruby +instance_of_object.method(:foo).source_location + +# Example for when we would call project.private? +project.method(:private?).source_location +``` + +## Limiting output + +Adding a semicolon(`;`) and a follow-up statement at the end of a statement prevents the default implicit return output. This can be used if you are already explicitly printing details and potentially have a lot of return output: + +```ruby +puts ActiveRecord::Base.descendants; :ok +Project.select(&:pages_deployed?).each {|p| puts p.pages_url }; true +``` + +## Get or store the result of last operation + +Underscore(`_`) represents the implicit return of the previous statement. You can use this to quickly assign a variable from the output of the previous command: + +```ruby +Project.last +# => #<Project id:2537 root/discard>> +project = _ +# => #<Project id:2537 root/discard>> +project.id +# => 2537 +``` + ## Active Record objects ### Looking up database-persisted objects @@ -331,6 +369,15 @@ D, [2020-03-05T17:18:30.406047 #910] DEBUG -- : User Load (2.6ms) SELECT "use For more on different ways to retrieve data from the database using Active Record, please see the [Active Record Query Interface documentation](https://guides.rubyonrails.org/active_record_querying.html). +## Query the database using an Active Record model + +```ruby +m = Model.where('attribute like ?', 'ex%') + +# for example to query the projects +projects = Project.where('path like ?', 'Oumua%') +``` + ### Modifying Active Record objects In the previous section, we learned about retrieving database records using diff --git a/doc/administration/operations/ssh_certificates.md b/doc/administration/operations/ssh_certificates.md index 1e405189342..8069dad4d8d 100644 --- a/doc/administration/operations/ssh_certificates.md +++ b/doc/administration/operations/ssh_certificates.md @@ -159,7 +159,7 @@ users (especially if they're renewed) than you have deploy keys. Users can still bypass SSH certificate authentication by manually uploading an SSH public key to their profile, relying on the `~/.ssh/authorized_keys` fallback to authenticate it. There's -currently no feature to prevent this, +currently no feature to prevent this, [but there's an open request for adding it](https://gitlab.com/gitlab-org/gitlab/-/issues/23260). Such a restriction can currently be hacked in by, for example, providing a diff --git a/doc/administration/package_information/supported_os.md b/doc/administration/package_information/supported_os.md index e4d7ea72cdc..5ccabd66ed0 100644 --- a/doc/administration/package_information/supported_os.md +++ b/doc/administration/package_information/supported_os.md @@ -24,6 +24,7 @@ The following lists the currently supported OSs and their possible EOL dates. | OpenSUSE 15.3 | GitLab CE / GitLab EE 14.5.0 | x86_64, aarch64 | [OpenSUSE Install Documentation](https://about.gitlab.com/install/#opensuse-leap-15-3) | Nov 2022 | <https://en.opensuse.org/Lifetime> | | RHEL 8 | GitLab CE / GitLab EE 12.8.1 | x86_64, arm64 | [Use CentOS Install Documentation](https://about.gitlab.com/install/#centos-7) | May 2029 | [RHEL Details](https://access.redhat.com/support/policy/updates/errata/#Life_Cycle_Dates) | | SLES 12 | GitLab EE 9.0.0 | x86_64 | [Use OpenSUSE Install Documentation](https://about.gitlab.com/install/#opensuse-leap-15-3) | Oct 2027 | <https://www.suse.com/lifecycle/> | +| SLES 15 | GitLab EE 14.8.0 | x86_64 | [Use OpenSUSE Install Documentation](https://about.gitlab.com/install/#opensuse-leap-15-3) | Dec 2024 | <https://www.suse.com/lifecycle/> | | Oracle Linux | GitLab CE / GitLab EE 8.14.0 | x86_64 | [Use CentOS Install Documentation](https://about.gitlab.com/install/#centos-7) | Jul 2024 | <https://www.oracle.com/a/ocom/docs/elsp-lifetime-069338.pdf> | | Scientific Linux | GitLab CE / GitLab EE 8.14.0 | x86_64 | [Use CentOS Install Documentation](https://about.gitlab.com/install/#centos-7) | June 2024 | <https://scientificlinux.org/downloads/sl-versions/sl7/> | | Ubuntu 18.04 | GitLab CE / GitLab EE 10.7.0 | amd64 | [Ubuntu Install Documentation](https://about.gitlab.com/install/#ubuntu) | April 2023 | <https://wiki.ubuntu.com/Releases> | diff --git a/doc/administration/packages/container_registry.md b/doc/administration/packages/container_registry.md index 8b115ca1af4..537840ce785 100644 --- a/doc/administration/packages/container_registry.md +++ b/doc/administration/packages/container_registry.md @@ -23,7 +23,7 @@ may or may not be available by default. The Container Registry is automatically enabled and available on your GitLab domain, port 5050 if: -- You're using the built-in [Let's Encrypt integration](https://docs.gitlab.com/omnibus/settings/ssl.html#lets-encrypt-integration), and +- You're using the built-in [Let's Encrypt integration](https://docs.gitlab.com/omnibus/settings/ssl.html#enable-the-lets-encrypt-integration), and - You're using GitLab 12.5 or later. Otherwise, the Container Registry is not enabled. To enable it: @@ -199,7 +199,7 @@ a wildcard certificate if hosted under a subdomain of your existing GitLab domain, for example, `registry.gitlab.example.com`. As well as manually generated SSL certificates (explained here), certificates automatically -generated by Let's Encrypt are also [supported in Omnibus installs](https://docs.gitlab.com/omnibus/settings/ssl.html#host-services). +generated by Let's Encrypt are also [supported in Omnibus installs](https://docs.gitlab.com/omnibus/settings/ssl.html). Let's assume that you want the container Registry to be accessible at `https://registry.gitlab.example.com`. @@ -322,7 +322,7 @@ the Container Registry by themselves, follow the steps below. In GitLab, tokens for the Container Registry expire every five minutes. To increase the token duration: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Container Registry**. 1. For the **Authorization token duration (minutes)**, update the value. @@ -906,7 +906,7 @@ project.container_repositories.find_each do |repo| puts repo.attributes # Start the tag cleanup - puts Projects::ContainerRepository::CleanupTagsService.new(repo, user, policy.attributes.except("created_at", "updated_at")).execute() + puts Projects::ContainerRepository::CleanupTagsService.new(container_repository: repo, current_user: user, params: policy.attributes.except("created_at", "updated_at")).execute end ``` @@ -1202,7 +1202,7 @@ Before diving in to the following sections, here's some basic troubleshooting: been synchronized (for example, via NTP). 1. If you are using an S3-backed Registry, double check that the IAM - permissions and the S3 credentials (including region) are correct. See + permissions and the S3 credentials (including region) are correct. See [the sample IAM policy](https://docs.docker.com/registry/storage-drivers/s3/) for more details. @@ -1631,7 +1631,7 @@ wrong. However, since all communications between Docker clients and servers are done over HTTPS, it's a bit difficult to decrypt the traffic quickly even if you know the private key. What can we do instead? -One way would be to disable HTTPS by setting up an +One way would be to disable HTTPS by setting up an [insecure Registry](https://docs.docker.com/registry/insecure/). This could introduce a security hole and is only recommended for local testing. If you have a production system and can't or don't want to do this, there is another way: diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index e545fb3cd1b..024fb12a51f 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -156,7 +156,7 @@ Watch the [video tutorial](https://youtu.be/dD8c7WNcc6s) for this configuration. **Requirements:** - [Wildcard DNS setup](#dns-configuration) -- Wildcard TLS certificate +- TLS certificate. Can be either Wildcard, or any other type meeting the [requirements](../../user/project/pages/custom_domains_ssl_tls_certification/index.md#manual-addition-of-ssltls-certificates). --- @@ -191,6 +191,12 @@ to use the HTTPS protocol. WARNING: Multiple wildcards for one instance is not supported. Only one wildcard per instance can be assigned. +WARNING: +GitLab Pages does not update the OAuth application if changes are made to the redirect URI. +Before you reconfigure, remove the `gitlab_pages` section from `/etc/gitlab/gitlab-secrets.json`, +then run `gitlab-ctl reconfigure`. For more information, read +[GitLab Pages does not regenerate OAuth](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/3947). + ### Wildcard domains with TLS-terminating Load Balancer **Requirements:** @@ -368,7 +374,7 @@ world. Custom domains and TLS are supported. If you don't have IPv6, you can omit the IPv6 address. -1. If you haven't named your certificate and key `example.io.crt` and `example.io.key` respectively, +1. If you haven't named your certificate `example.io.crt` and your key `example.io.key`, then you need to also add the full paths as shown below: ```ruby @@ -397,7 +403,7 @@ domain as a custom domain to their project. If your user base is private or otherwise trusted, you can disable the verification requirement: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Pages**. 1. Clear the **Require users to prove ownership of custom domains** checkbox. @@ -414,7 +420,7 @@ sites served under a custom domain. To enable it: 1. Choose an email address on which you want to receive notifications about expiring domains. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Pages**. 1. Enter the email address for receiving notifications and accept Let's Encrypt's Terms of Service. @@ -467,7 +473,7 @@ pre-existing applications must modify the GitLab Pages OAuth application. Follow this: 1. Enable [access control](#access-control). -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Applications**. 1. Expand **GitLab Pages**. 1. Clear the `api` scope's checkbox and select the desired scope's checkbox (for example, @@ -486,7 +492,7 @@ This can be helpful to restrict information published with Pages websites to the of your instance only. To do that: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Pages**. 1. Select the **Disable public access to Pages sites** checkbox. @@ -523,7 +529,7 @@ For Omnibus, this is fixed by [installing a custom CA in Omnibus GitLab](https:/ > [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/548) in GitLab 14.8. -If GitLab has been [configured to require mutual TLS](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-2-way-ssl-client-authentication), you need to add the client certificates to Pages: +If GitLab has been [configured to require mutual TLS](https://docs.gitlab.com/omnibus/settings/ssl.html#enable-2-way-ssl-client-authentication), you need to add the client certificates to Pages: 1. Configure in `/etc/gitlab/gitlab.rb`: @@ -699,7 +705,7 @@ Prerequisites: To set the global maximum pages size for a project: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Pages**. 1. Enter a value under **Maximum size of pages**. @@ -713,7 +719,7 @@ Prerequisites: To set the maximum size of each GitLab Pages site in a group, overriding the inherited setting: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Pages**. 1. Enter a value under **Maximum size** in MB. @@ -727,11 +733,24 @@ Prerequisites: To set the maximum size of GitLab Pages site in a project, overriding the inherited setting: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Pages**. 1. Enter a value under **Maximum size of pages** in MB. 1. Select **Save changes**. +## Set maximum number of GitLab Pages custom domains for a project + +Prerequisite: + +- You must be an administrator of a self-managed GitLab instance. + +To set the maximum number of GitLab Pages custom domains for a project: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > Preferences**, and expand **Pages**. +1. Enter a value for **Maximum number of custom domains per project**. Use `0` for unlimited domains. +1. Select **Save changes**. + ## Running GitLab Pages on a separate server You can run the GitLab Pages daemon on a separate server to decrease the load on @@ -933,7 +952,7 @@ The following settings are: | `connection` | Various connection options described below. | | NOTE: -If you want to stop using and disconnect the NFS server, you need to +If you want to stop using and disconnect the NFS server, you need to [explicitly disable local storage](#disable-pages-local-storage), and it's only possible after upgrading to GitLab 13.11. #### S3-compatible connection settings @@ -1379,7 +1398,7 @@ Upgrading to an [officially supported operating system](https://about.gitlab.com This problem comes from the permissions of the GitLab Pages OAuth application. To fix it: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Applications > GitLab Pages**. 1. Edit the application. 1. Under **Scopes**, ensure that the `api` scope is selected. @@ -1502,3 +1521,12 @@ To fix that: - Store your deployments locally, by commenting out that line. 1. Save the changes you made to your `gitlab.rb` file, then [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + +### 404 error `The page you're looking for could not be found` + +If you get a `404 Page Not Found` response from GitLab Pages: + +1. Check `.gitlab-ci.yml` contains the job `pages:`. +1. Check the current project's pipeline to confirm the job `pages:deploy` is being run. + +Without the `pages:deploy` job, the updates to your GitLab Pages site are never published. diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md index 5ab508dbaca..14ac05e0293 100644 --- a/doc/administration/pages/source.md +++ b/doc/administration/pages/source.md @@ -364,9 +364,8 @@ world. Custom domains and TLS are supported. ``` 1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in - order to enable the pages daemon. In `gitlab_pages_options` the - `-pages-domain`, `-listen-http` and `-listen-https` must match the `host`, - `external_http` and `external_https` settings that you set above respectively. + order to enable the pages daemon. In `gitlab_pages_options`, you must match the + `-pages-domain` with `host`, `-listen-http` with `external_http`, and `-listen-https` with `external_https` settings. The `-root-cert` and `-root-key` settings are the wildcard TLS certificates of the `example.io` domain: @@ -486,7 +485,7 @@ The default for the maximum size of unpacked archives per project is 100 MB. To change this value: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Pages**. 1. Update the value for **Maximum size of pages (MB)**. diff --git a/doc/administration/polling.md b/doc/administration/polling.md index 5699bf78c04..7f1e7a047cf 100644 --- a/doc/administration/polling.md +++ b/doc/administration/polling.md @@ -26,7 +26,7 @@ The default value (`1`) is recommended for the majority of GitLab installations. To adjust the polling interval multiplier: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Polling interval multiplier**. 1. Set a value for the polling interval multiplier. This multiplier is applied to all resources at diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index a900da52aba..c4401b49180 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -326,14 +326,20 @@ database: gitlabhq_production Database migrations can be stuck in an incomplete state, with a `down` status in the output of the `sudo gitlab-rake db:migrate:status` command. -To complete these migrations, use the following Rake task: +1. To complete these migrations, use the following Rake task: -```shell -sudo gitlab-rake db:migrate -``` + ```shell + sudo gitlab-rake db:migrate + ``` + +1. After the command completes, run `sudo gitlab-rake db:migrate:status` to check if all migrations are completed (have an `up` status). + +1. Hot reload `puma` and `sidekiq` services: -After the command completes, run `sudo gitlab-rake db:migrate:status` to check if all -migrations are completed (have an `up` status). + ```shell + sudo gitlab-ctl hup puma + sudo gitlab-ctl restart sidekiq + ``` ## Rebuild database indexes diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index f3a9845d129..00bd71af6c5 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -45,7 +45,7 @@ Note the following: compatible as described in the [Version history](../../user/project/settings/import_export.md#version-history). - The project import option must be enabled: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility and access controls**. 1. Under **Import sources**, check the "Project export enabled" option. diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md index 27b899dd1b1..fc0ff23c5b1 100644 --- a/doc/administration/raketasks/storage.md +++ b/doc/administration/raketasks/storage.md @@ -109,7 +109,7 @@ sudo gitlab-rake gitlab:storage:migrate_to_hashed ID_FROM=50 ID_TO=100 To monitor the progress in GitLab: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. Watch how long the `hashed_storage:hashed_storage_project_migrate` queue takes to finish. After it reaches zero, you can confirm every project diff --git a/doc/administration/raketasks/uploads/migrate.md b/doc/administration/raketasks/uploads/migrate.md index c73840cb9ff..216c0875645 100644 --- a/doc/administration/raketasks/uploads/migrate.md +++ b/doc/administration/raketasks/uploads/migrate.md @@ -79,7 +79,8 @@ The Rake task uses three parameters to find uploads to migrate: NOTE: These parameters are mainly internal to the structure of GitLab, you may want to refer to the task list -instead below. +instead below. After running these individual tasks, we recommend that you run the [all-in-one Rake task](#all-in-one-rake-task) +to migrate any uploads not included in the listed types. This task also accepts an environment variable which you can use to override the default batch size: diff --git a/doc/administration/redis/replication_and_failover.md b/doc/administration/redis/replication_and_failover.md index c4b83b66738..b775b579fd4 100644 --- a/doc/administration/redis/replication_and_failover.md +++ b/doc/administration/redis/replication_and_failover.md @@ -343,7 +343,7 @@ NOTE: If you are using an external Redis Sentinel instance, be sure to exclude the `requirepass` parameter from the Sentinel configuration. This parameter causes clients to report `NOAUTH -Authentication required.`. +Authentication required.`. [Redis Sentinel 3.2.x does not support password authentication](https://github.com/antirez/redis/issues/3279). Now that the Redis servers are all set up, let's configure the Sentinel diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index fcc32100bc8..5d676dac000 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -18,30 +18,30 @@ full list of reference architectures, see > - **Test requests per second (RPS) rates:** API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS > - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)** -| Service | Nodes | Configuration | GCP | AWS | Azure | -|-----------------------------------------------------|----------------|-------------------------|------------------|----------------|----------------| -| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| PostgreSQL<sup>1</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | `D8s v3` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | `D16s v3` | -| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` | -| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Service | Nodes | Configuration | GCP | AWS | Azure | +|------------------------------------------|-------|-------------------------|------------------|----------------|-----------| +| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| PostgreSQL<sup>1</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | `D8s v3` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | `D16s v3` | +| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` | +| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Object storage<sup>4</sup> | - | - | - | - | - | +| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -142,10 +142,12 @@ Before starting, you should take note of the following requirements / guidance f This reference architecture was built and tested on Google Cloud Platform (GCP) using the [Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms) -CPU platform. On different hardware you may find that adjustments, either lower -or higher, are required for your CPU or node counts. For more information, see -our [Sysbench](https://github.com/akopytov/sysbench)-based -[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks). +CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)). + +Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported. + +NOTE: +Any "burstable" instance types are not recommended due to inconsistent performance. ### Supported infrastructure @@ -186,10 +188,11 @@ To set up GitLab and its components to accommodate up to 10,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. +1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) + to have shared disk storage service as an alternative to Gitaly or object + storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. -1. [Configure NFS](#configure-nfs) - to have shared disk storage service for certain GitLab operations (non Gitaly or Object Storage). The servers start on the same 10.6.0.0/24 private network range, and can connect to each other freely on these addresses. @@ -259,7 +262,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather than `HTTP(S)` protocol. This will pass the connection to the application node's NGINX service untouched. NGINX will have the SSL certificate and listen on port 443. -See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Load balancer terminates SSL without backend SSL @@ -270,7 +273,7 @@ terminating SSL. Since communication between the load balancer and GitLab will not be secure, there is some additional configuration needed. See the -[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) +[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination) for details. ### Load balancer terminates SSL with backend SSL @@ -283,7 +286,7 @@ Traffic will also be secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be added to GitLab to configure SSL certificates. See -[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Readiness checks @@ -2009,7 +2012,7 @@ On each node perform the following: 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -1. If you're [using NFS](#configure-nfs): +1. If you're [using NFS](#configure-nfs-optional): 1. If necessary, install the NFS client utility packages using the following commands: @@ -2069,7 +2072,7 @@ On each node perform the following: When you specify `https` in the `external_url`, as in the previous example, GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the certificates aren't present, NGINX will fail to start. For more information, see -the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). ### GitLab Rails post-configuration @@ -2175,7 +2178,7 @@ To configure the Monitoring node: ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -It's recommended over [NFS](#configure-nfs) and in general it's better +It's recommended over [NFS](#configure-nfs-optional) and in general it's better in larger setups as object storage is typically much more performant, reliable, and scalable. @@ -2187,7 +2190,6 @@ GitLab has been tested on a number of object storage providers: - [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm) - [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html) - [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) -- On-premises hardware and appliances from various storage vendors. - MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation. There are two ways of specifying object storage configuration in GitLab: @@ -2197,25 +2199,13 @@ There are two ways of specifying object storage configuration in GitLab: - [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). -Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. - -For configuring object storage in GitLab 13.1 and earlier, or for storage types not -supported by consolidated configuration form, refer to the following guides based -on what features you intend to use: - -|Object storage type|Supported by consolidated configuration?| -|-------------------|----------------------------------------| -| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No | -| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes | -| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes | -| [Uploads](../uploads.md#using-object-storage) | Yes | -| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No | -| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes | -| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | -| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | -| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | -| [Terraform state files](../terraform_state.md#using-object-storage) | Yes | +The consolidated form is used in the following examples when available. + +NOTE: +When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). +The previous [background upload](../../development/uploads/index.md#direct-upload) mode, +which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2228,29 +2218,13 @@ in the future. </a> </div> -## Enable incremental logging +### Enable incremental logging GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. -## Configure Advanced Search - -You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) -for faster, more advanced code search across your entire GitLab instance. - -Elasticsearch cluster design and requirements are dependent on your specific -data. For recommended best practices about how to set up your Elasticsearch -cluster alongside your instance, read how to -[choose the optimal cluster configuration](../../integration/advanced_search/elasticsearch.md#guidance-on-choosing-optimal-cluster-configuration). - -<div align="right"> - <a type="button" class="btn btn-default" href="#setup-components"> - Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> - </a> -</div> - -## Configure NFS +## Configure NFS (optional) [Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) are recommended over NFS wherever possible for improved performance. @@ -2258,14 +2232,24 @@ are recommended over NFS wherever possible for improved performance. See how to [configure NFS](../nfs.md). WARNING: -Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be -unavailable from GitLab 15.0. No further enhancements are planned for this feature. +Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) +after the release of GitLab 15.6. No further enhancements are planned for this feature. Read: - [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). - About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). +## Configure Advanced Search + +You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) +for faster, more advanced code search across your entire GitLab instance. + +Elasticsearch cluster design and requirements are dependent on your specific +data. For recommended best practices about how to set up your Elasticsearch +cluster alongside your instance, read how to +[choose the optimal cluster configuration](../../integration/advanced_search/elasticsearch.md#guidance-on-choosing-optimal-cluster-configuration). + <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> @@ -2292,6 +2276,10 @@ to be complex. **This setup is only recommended** if you have strong working knowledge and experience in Kubernetes. The rest of this section assumes this. +NOTE: +**Gitaly Cluster is not supported to be run in Kubernetes**. +Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details. + ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats @@ -2302,11 +2290,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK and CPU requirements should translate to most other providers. We hope to update this in the future with further specific cloud provider details. -| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | -|-----------------------------------------------|-------|-------------------------|-----------------|--------------|---------------------------------| -| Webservice | 4 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | 127.5 vCPU, 118 GB memory | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory | -| Supporting services such as NGINX, Prometheus | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory | +| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | +|---------------------|-------|-------------------------|-----------------|--------------|---------------------------------| +| Webservice | 4 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | 127.5 vCPU, 118 GB memory | +| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory | +| Supporting services | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory | - For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. @@ -2316,25 +2304,25 @@ future with further specific cloud provider details. Next are the backend components that run on static compute VMs via Omnibus (or External PaaS services where applicable): -| Service | Nodes | Configuration | GCP | AWS | -|------------------------------------------|----------------|-----------------------|------------------|----------------| -| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| PostgreSQL<sup>1</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | -| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | +| Service | Nodes | Configuration | GCP | AWS | +|------------------------------------------|-------|-----------------------|------------------|--------------| +| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| PostgreSQL<sup>1</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | +| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Object storage<sup>4</sup> | - | - | - | - | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -2353,7 +2341,7 @@ card "Kubernetes via Helm Charts" as kubernetes { collections "**Sidekiq** x4" as sidekiq #ff8dd1 } - card "**Supporting Services**" as support + card "**Supporting Services** x2" as support } card "**Internal Load Balancer**" as ilb #9370DB @@ -2419,29 +2407,42 @@ documents how to apply the calculated configuration to the Helm Chart. #### Webservice -Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_. -Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using +Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_. +Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using the [recommended topology](#cluster-topology) because four worker processes are created by default and each pod has other small processes running. For 10,000 users we recommend a total Puma worker count of around 80. With the [provided recommendations](#cluster-topology) this allows the deployment of up to 20 Webservice pods with 4 workers per pod and 5 pods per node. Expand available resources using -the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional +the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional Webservice pod. For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources). #### Sidekiq -Sidekiq pods should generally have 1 vCPU and 2 GB of memory. +Sidekiq pods should generally have 0.9 CPU and 2 GB of memory. [The provided starting point](#cluster-topology) allows the deployment of up to -14 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory +14 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory ratio for each additional pod. For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources). +### Supporting + +The Supporting Node Pool is designed to house all supporting deployments that don't need to be +on the Webservice and Sidekiq pools. + +This includes various deployments related to the Cloud Provider's implementation and supporting +GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/). + +If you wish to make any additional deployments, such as for Monitoring, it's recommended +to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed +specifically to accommodate several additional deployments. However, if your deployments don't fit into the +pool as given, you can increase the node pool accordingly. + <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md index 1a774603863..96b1e541f92 100644 --- a/doc/administration/reference_architectures/1k_users.md +++ b/doc/administration/reference_architectures/1k_users.md @@ -72,10 +72,12 @@ Before starting, you should take note of the following requirements / guidance f This reference architecture was built and tested on Google Cloud Platform (GCP) using the [Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms) -CPU platform. On different hardware you may find that adjustments, either lower -or higher, are required for your CPU or node counts. For more information, see -our [Sysbench](https://github.com/akopytov/sysbench)-based -[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks). +CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)). + +Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported. + +NOTE: +Any "burstable" instance types are not recommended due to inconsistent performance. ### Supported infrastructure diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 51d3e3d39a6..423dbc7abfb 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -18,30 +18,30 @@ full list of reference architectures, see > - **Test requests per second (RPS) rates:** API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS > - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)** -| Service | Nodes | Configuration | GCP | AWS | Azure | -|---------------------------------------------------|----------------|-------------------------|------------------|----------------|----------------| -| External load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | -| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| PostgreSQL<sup>1</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | `D16s v3` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | -| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | `D32s v3` | -| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| GitLab Rails | 5 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` | -| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Service | Nodes | Configuration | GCP | AWS | Azure | +|------------------------------------------|-------|-------------------------|------------------|--------------|-----------| +| External load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| PostgreSQL<sup>1</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | `D16s v3` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | `D32s v3` | +| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| GitLab Rails | 5 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` | +| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Object storage<sup>4</sup> | - | - | - | - | - | +| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -142,10 +142,12 @@ Before starting, you should take note of the following requirements / guidance f This reference architecture was built and tested on Google Cloud Platform (GCP) using the [Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms) -CPU platform. On different hardware you may find that adjustments, either lower -or higher, are required for your CPU or node counts. For more information, see -our [Sysbench](https://github.com/akopytov/sysbench)-based -[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks). +CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)). + +Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported. + +NOTE: +Any "burstable" instance types are not recommended due to inconsistent performance. ### Supported infrastructure @@ -186,11 +188,11 @@ To set up GitLab and its components to accommodate up to 25,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. +1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) + to have shared disk storage service as an alternative to Gitaly or object + storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. -1. [Configure NFS](#configure-nfs) - to have shared disk storage service for certain GitLab operations (non - Gitaly or Object Storage). The servers start on the same 10.6.0.0/24 private network range, and can connect to each other freely on these addresses. @@ -262,7 +264,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather than `HTTP(S)` protocol. This will pass the connection to the application node's NGINX service untouched. NGINX will have the SSL certificate and listen on port 443. -See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Load balancer terminates SSL without backend SSL @@ -273,7 +275,7 @@ terminating SSL. Since communication between the load balancer and GitLab will not be secure, there is some additional configuration needed. See the -[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) +[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination) for details. ### Load balancer terminates SSL with backend SSL @@ -285,8 +287,8 @@ end users will see. Traffic will also be secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be -added to GitLab to configure SSL certificates. See -[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +added to GitLab to configure SSL certificates. See the +[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Readiness checks @@ -2015,7 +2017,7 @@ On each node perform the following: 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -1. If you're [using NFS](#configure-nfs): +1. If you're [using NFS](#configure-nfs-optional): 1. If necessary, install the NFS client utility packages using the following commands: @@ -2074,7 +2076,7 @@ On each node perform the following: When you specify `https` in the `external_url`, as in the previous example, GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the certificates aren't present, NGINX will fail to start. For more information, see -the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). ### GitLab Rails post-configuration @@ -2179,9 +2181,9 @@ To configure the Monitoring node: ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -Object storage is also recommended over [NFS](#configure-nfs) and in general -it's better in larger setups as object storage is typically much more performant, -reliable, and scalable. +It's recommended over [NFS](#configure-nfs-optional) and in general it's better +in larger setups as object storage is typically much more performant, reliable, +and scalable. GitLab has been tested on a number of object storage providers: @@ -2191,7 +2193,6 @@ GitLab has been tested on a number of object storage providers: - [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm) - [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html) - [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) -- On-premises hardware and appliances from various storage vendors. - MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation. There are two ways of specifying object storage configuration in GitLab: @@ -2201,25 +2202,13 @@ There are two ways of specifying object storage configuration in GitLab: - [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). -Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. - -For configuring object storage in GitLab 13.1 and earlier, or for storage types not -supported by consolidated configuration form, refer to the following guides based -on what features you intend to use: - -|Object storage type|Supported by consolidated configuration?| -|-------------------|----------------------------------------| -| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No | -| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes | -| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes | -| [Uploads](../uploads.md#using-object-storage) | Yes | -| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No | -| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes | -| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | -| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | -| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | -| [Terraform state files](../terraform_state.md#using-object-storage) | Yes | +The consolidated form is used in the following examples when available. + +NOTE: +When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). +The previous [background upload](../../development/uploads/index.md#direct-upload) mode, +which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2232,12 +2221,28 @@ in the future. </a> </div> -## Enable incremental logging +### Enable incremental logging GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. +## Configure NFS (optional) + +[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) +are recommended over NFS wherever possible for improved performance. + +See how to [configure NFS](../nfs.md). + +WARNING: +Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) +after the release of GitLab 15.6. No further enhancements are planned for this feature. + +Read: + +- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). +- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). + ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) @@ -2254,22 +2259,6 @@ cluster alongside your instance, read how to </a> </div> -## Configure NFS - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be -unavailable from GitLab 15.0. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Cloud Native Hybrid reference architecture with Helm Charts (alternative) As an alternative approach, you can also run select components of GitLab as Cloud Native @@ -2290,6 +2279,10 @@ to be complex. **This setup is only recommended** if you have strong working knowledge and experience in Kubernetes. The rest of this section assumes this. +NOTE: +**Gitaly Cluster is not supported to be run in Kubernetes**. +Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details. + ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats @@ -2300,11 +2293,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK and CPU requirements should translate to most other providers. We hope to update this in the future with further specific cloud provider details. -| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | -|-----------------------------------------------|-------|-------------------------|-----------------|-----------------|---------------------------------| -| Webservice | 7 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | 223 vCPU, 206.5 GB memory | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory | -| Supporting services such as NGINX, Prometheus | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory | +| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | +|---------------------|-------|-------------------------|-----------------|--------------|---------------------------------| +| Webservice | 7 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | 223 vCPU, 206.5 GB memory | +| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory | +| Supporting services | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory | - For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. @@ -2314,25 +2307,25 @@ future with further specific cloud provider details. Next are the backend components that run on static compute VMs via Omnibus (or External PaaS services where applicable): -| Service | Nodes | Configuration | GCP | AWS | -|------------------------------------------|----------------|------------------------|------------------|----------------| -| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| PostgreSQL<sup>1</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6GB memory | `n1-highcpu-4` | `c5.xlarge` | -| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | -| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | +| Service | Nodes | Configuration | GCP | AWS | +|------------------------------------------|-------|------------------------|------------------|--------------| +| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| PostgreSQL<sup>1</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6GB memory | `n1-highcpu-4` | `c5.xlarge` | +| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | +| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Object storage<sup>4</sup> | - | - | - | - | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -2347,11 +2340,11 @@ card "Kubernetes via Helm Charts" as kubernetes { card "**External Load Balancer**" as elb #6a9be7 together { - collections "**Webservice** x4" as gitlab #32CD32 + collections "**Webservice** x7" as gitlab #32CD32 collections "**Sidekiq** x4" as sidekiq #ff8dd1 } - card "**Supporting Services**" as support + card "**Supporting Services** x2" as support } card "**Internal Load Balancer**" as ilb #9370DB @@ -2417,29 +2410,42 @@ documents how to apply the calculated configuration to the Helm Chart. #### Webservice -Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_. -Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using +Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_. +Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using the [recommended topology](#cluster-topology) because four worker processes are created by default and each pod has other small processes running. For 25,000 users we recommend a total Puma worker count of around 140. With the [provided recommendations](#cluster-topology) this allows the deployment of up to 35 Webservice pods with 4 workers per pod and 5 pods per node. Expand available resources using -the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional +the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional Webservice pod. For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources). #### Sidekiq -Sidekiq pods should generally have 1 vCPU and 2 GB of memory. +Sidekiq pods should generally have 0.9 CPU and 2 GB of memory. [The provided starting point](#cluster-topology) allows the deployment of up to -14 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory +14 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory ratio for each additional pod. For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources). +### Supporting + +The Supporting Node Pool is designed to house all supporting deployments that don't need to be +on the Webservice and Sidekiq pools. + +This includes various deployments related to the Cloud Provider's implementation and supporting +GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/). + +If you wish to make any additional deployments, such as for Monitoring, it's recommended +to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed +specifically to accommodate several additional deployments. However, if your deployments don't fit into the +pool as given, you can increase the node pool accordingly. + <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 8ca58da55ca..99cc6d47f6a 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -19,22 +19,22 @@ For a full list of reference architectures, see > - **Test requests per second (RPS) rates:** API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS > - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)** -| Service | Nodes | Configuration | GCP | AWS | Azure | -|------------------------------------------|----------------|-------------------------|-----------------|----------------|----------------| -| Load balancer<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | -| Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` | `D2s v3` | -| Gitaly | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| GitLab Rails | 2 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | -| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Service | Nodes | Configuration | GCP | AWS | Azure | +|----------------------------|-------|------------------------|-----------------|--------------|----------| +| Load balancer<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | +| Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` | `D2s v3` | +| Gitaly | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| GitLab Rails | 2 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | +| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Object storage<sup>4</sup> | - | - | - | - | - | +| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run as reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run as reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. <!-- markdownlint-enable MD029 --> NOTE: @@ -78,10 +78,12 @@ Before starting, you should take note of the following requirements / guidance f This reference architecture was built and tested on Google Cloud Platform (GCP) using the [Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms) -CPU platform. On different hardware you may find that adjustments, either lower -or higher, are required for your CPU or node counts. For more information, see -our [Sysbench](https://github.com/akopytov/sysbench)-based -[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks). +CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)). + +Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported. + +NOTE: +Any "burstable" instance types are not recommended due to inconsistent performance. ### Supported infrastructure @@ -110,11 +112,11 @@ To set up GitLab and its components to accommodate up to 2,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, - more advanced code search across your entire GitLab instance. 1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) to have shared disk storage service as an alternative to Gitaly or object storage. +1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, + more advanced code search across your entire GitLab instance. ## Configure the external load balancer @@ -147,7 +149,7 @@ of `HTTP(S)`. This will pass the connection unaltered to the application node's NGINX service, which has the SSL certificate and listens to port 443. For details about managing SSL certificates and configuring NGINX, see the -[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) ### Load balancer terminates SSL without backend SSL @@ -157,7 +159,7 @@ terminating SSL. Due to communication between the load balancer and GitLab not being secure, you'll need to complete some additional configuration. For details, see the -[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl). +[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination). ### Load balancer terminates SSL with backend SSL @@ -169,7 +171,7 @@ Traffic will be secure between the load balancers and NGINX in this scenario, and there's no need to add a configuration for proxied SSL. However, you'll need to add a configuration to GitLab to configure SSL certificates. For details about managing SSL certificates and configuring NGINX, see the -[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). ### Readiness checks @@ -750,7 +752,7 @@ On each node perform the following: When you specify `https` in the `external_url`, as in the previous example, GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the certificates aren't present, NGINX will fail to start. For more information, see -the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). ### GitLab Rails post-configuration @@ -885,10 +887,10 @@ running [Prometheus](../monitoring/prometheus/index.md) and ## Configure the object storage -GitLab supports using an object storage service for holding several types of -data, and is recommended over [NFS](#configure-nfs-optional). In general, -object storage services are better for larger environments, as object storage -is typically much more performant, reliable, and scalable. +GitLab supports using an object storage service for holding numerous types of data. +It's recommended over [NFS](#configure-nfs-optional) and in general it's better +in larger setups as object storage is typically much more performant, reliable, +and scalable. GitLab has been tested on a number of object storage providers: @@ -898,7 +900,6 @@ GitLab has been tested on a number of object storage providers: - [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm) - [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html) - [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) -- On-premises hardware and appliances from various storage vendors. - MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation. There are two ways of specifying object storage configuration in GitLab: @@ -908,29 +909,13 @@ There are two ways of specifying object storage configuration in GitLab: - [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). -Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. - -GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes. - -In GitLab 13.6 and later, it's also recommended to switch to [Incremental logging](../job_logs.md#incremental-logging-architecture), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed. +The consolidated form is used in the following examples when available. -For configuring object storage in GitLab 13.1 and earlier, or for storage types not -supported by consolidated configuration form, refer to the following guides based -on what features you intend to use: - -|Object storage type|Supported by consolidated configuration?| -|-------------------|----------------------------------------| -| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No | -| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes | -| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes | -| [Uploads](../uploads.md#using-object-storage) | Yes | -| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No | -| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes | -| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | -| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | -| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | -| [Terraform state files](../terraform_state.md#using-object-storage) | Yes | +NOTE: +When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). +The previous [background upload](../../development/uploads/index.md#direct-upload) mode, +which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -943,21 +928,11 @@ in the future. </a> </div> -## Configure Advanced Search **(PREMIUM SELF)** - -You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) -for faster, more advanced code search across your entire GitLab instance. +### Enable incremental logging -Elasticsearch cluster design and requirements are dependent on your specific -data. For recommended best practices about how to set up your Elasticsearch -cluster alongside your instance, read how to -[choose the optimal cluster configuration](../../integration/advanced_search/elasticsearch.md#guidance-on-choosing-optimal-cluster-configuration). +GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. -<div align="right"> - <a type="button" class="btn btn-default" href="#setup-components"> - Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> - </a> -</div> +While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. ## Configure NFS (optional) @@ -968,14 +943,30 @@ possible. See how to [configure NFS](../nfs.md). WARNING: -Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be -unavailable from GitLab 15.0. No further enhancements are planned for this feature. +Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) +after the release of GitLab 15.6. No further enhancements are planned for this feature. Read: - [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). - About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). +## Configure Advanced Search **(PREMIUM SELF)** + +You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) +for faster, more advanced code search across your entire GitLab instance. + +Elasticsearch cluster design and requirements are dependent on your specific +data. For recommended best practices about how to set up your Elasticsearch +cluster alongside your instance, read how to +[choose the optimal cluster configuration](../../integration/advanced_search/elasticsearch.md#guidance-on-choosing-optimal-cluster-configuration). + +<div align="right"> + <a type="button" class="btn btn-default" href="#setup-components"> + Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> + </a> +</div> + ## Cloud Native Hybrid reference architecture with Helm Charts (alternative) As an alternative approach, you can also run select components of GitLab as Cloud Native @@ -998,6 +989,10 @@ to be complex. **This setup is only recommended** if you have strong working knowledge and experience in Kubernetes. The rest of this section assumes this. +NOTE: +**Gitaly Cluster is not supported to be run in Kubernetes**. +Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details. + ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats @@ -1008,11 +1003,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK and CPU requirements should translate to most other providers. We hope to update this in the future with further specific cloud provider details. -| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | -|-----------------------------------------------|-------|------------------------|-----------------|--------------|---------------------------------| -| Webservice | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | 23.7 vCPU, 16.9 GB memory | -| Sidekiq | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 3.9 vCPU, 11.8 GB memory | -| Supporting services such as NGINX, Prometheus | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | 1.9 vCPU, 5.5 GB memory | +| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | +|---------------------|-------|------------------------|-----------------|--------------|---------------------------------| +| Webservice | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | 23.7 vCPU, 16.9 GB memory | +| Sidekiq | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.8 vCPU, 25.9 GB memory | +| Supporting services | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | 1.9 vCPU, 5.5 GB memory | - For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. @@ -1022,18 +1017,18 @@ future with further specific cloud provider details. Next are the backend components that run on static compute VMs via Omnibus (or External PaaS services where applicable): -| Service | Nodes | Configuration | GCP | AWS | -|----------------------------|----------------|------------------------|-----------------|----------------| -| PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | -| Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` | -| Gitaly | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| Object storage<sup>3</sup> | Not applicable | Not applicable | Not applicable | Not applicable | +| Service | Nodes | Configuration | GCP | AWS | +|----------------------------|-------|------------------------|-----------------|-------------| +| PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | +| Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` | +| Gitaly | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| Object storage<sup>3</sup> | - | - | - | - | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. -3. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +3. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. <!-- markdownlint-enable MD029 --> NOTE: @@ -1048,9 +1043,10 @@ card "Kubernetes via Helm Charts" as kubernetes { together { collections "**Webservice** x3" as gitlab #32CD32 - card "**Sidekiq**" as sidekiq #ff8dd1 - collections "**Supporting Services** x2" as support + collections "**Sidekiq** x2" as sidekiq #ff8dd1 } + + collections "**Supporting Services** x2" as support } card "**Gitaly**" as gitaly #FF8C00 @@ -1081,29 +1077,42 @@ documents how to apply the calculated configuration to the Helm Chart. #### Webservice -Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_. -Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using +Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_. +Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using the [recommended topology](#cluster-topology) because two worker processes are created by default and each pod has other small processes running. For 2,000 users we recommend a total Puma worker count of around 12. With the [provided recommendations](#cluster-topology) this allows the deployment of up to 3 Webservice pods with 4 workers per pod and 1 pod per node. Expand available resources using -the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional +the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional Webservice pod. For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources). #### Sidekiq -Sidekiq pods should generally have 1 vCPU and 2 GB of memory. +Sidekiq pods should generally have 0.9 CPU and 2 GB of memory. [The provided starting point](#cluster-topology) allows the deployment of up to -4 Sidekiq pods. Expand available resources using the 1 vCPU to 2 GB memory +4 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory ratio for each additional pod. For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources). +### Supporting + +The Supporting Node Pool is designed to house all supporting deployments that don't need to be +on the Webservice and Sidekiq pools. + +This includes various deployments related to the Cloud Provider's implementation and supporting +GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/). + +If you wish to make any additional deployments, such as for Monitoring, it's recommended +to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed +specifically to accommodate several additional deployments. However, if your deployments don't fit into the +pool as given, you can increase the node pool accordingly. + <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index db81ae7ae3c..5c227e3dc27 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -28,29 +28,29 @@ For a full list of reference architectures, see > - **Test requests per second (RPS) rates:** API: 60 RPS, Web: 6 RPS, Git (Pull): 6 RPS, Git (Push): 1 RPS > - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)** -| Service | Nodes | Configuration | GCP | AWS | Azure | -|--------------------------------------------|----------------|-----------------------|-----------------|----------------|----------------| -| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | -| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | -| GitLab Rails | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | -| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Service | Nodes | Configuration | GCP | AWS | Azure | +|-------------------------------------------|-------|-----------------------|-----------------|--------------|----------| +| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | +| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | +| GitLab Rails | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | +| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Object storage<sup>4</sup> | - | - | - | - | - | +| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -148,10 +148,12 @@ Before starting, you should take note of the following requirements / guidance f This reference architecture was built and tested on Google Cloud Platform (GCP) using the [Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms) -CPU platform. On different hardware you may find that adjustments, either lower -or higher, are required for your CPU or node counts. For more information, see -our [Sysbench](https://github.com/akopytov/sysbench)-based -[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks). +CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)). + +Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported. + +NOTE: +Any "burstable" instance types are not recommended due to inconsistent performance. ### Supported infrastructure @@ -192,11 +194,11 @@ To set up GitLab and its components to accommodate up to 3,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, - more advanced code search across your entire GitLab instance. 1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) to have shared disk storage service as an alternative to Gitaly or object storage. +1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, + more advanced code search across your entire GitLab instance. The servers start on the same 10.6.0.0/24 private network range, and can connect to each other freely on these addresses. @@ -263,7 +265,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather than `HTTP(S)` protocol. This will pass the connection to the application node's NGINX service untouched. NGINX will have the SSL certificate and listen on port 443. -See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Load balancer terminates SSL without backend SSL @@ -274,7 +276,7 @@ terminating SSL. Since communication between the load balancer and GitLab will not be secure, there is some additional configuration needed. See the -[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) +[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination) for details. ### Load balancer terminates SSL with backend SSL @@ -286,8 +288,8 @@ end users will see. Traffic will also be secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be -added to GitLab to configure SSL certificates. See -[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +added to GitLab to configure SSL certificates. See the +[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Readiness checks @@ -2005,7 +2007,7 @@ On each node perform the following: When you specify `https` in the `external_url`, as in the previous example, GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the certificates aren't present, NGINX will fail to start. For more information, see -the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). ### GitLab Rails post-configuration @@ -2128,7 +2130,6 @@ GitLab has been tested on a number of object storage providers: - [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm) - [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html) - [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) -- On-premises hardware and appliances from various storage vendors. - MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation. There are two ways of specifying object storage configuration in GitLab: @@ -2138,25 +2139,13 @@ There are two ways of specifying object storage configuration in GitLab: - [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). -Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. - -For configuring object storage in GitLab 13.1 and earlier, or for storage types not -supported by consolidated configuration form, refer to the following guides based -on what features you intend to use: - -|Object storage type|Supported by consolidated configuration?| -|-------------------|----------------------------------------| -| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No | -| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes | -| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes | -| [Uploads](../uploads.md#using-object-storage) | Yes | -| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No | -| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes | -| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | -| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | -| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | -| [Terraform state files](../terraform_state.md#using-object-storage) | Yes | +The consolidated form is used in the following examples when available. + +NOTE: +When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). +The previous [background upload](../../development/uploads/index.md#direct-upload) mode, +which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2169,12 +2158,28 @@ in the future. </a> </div> -## Enable incremental logging +### Enable incremental logging GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. +## Configure NFS (optional) + +[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) +are recommended over NFS wherever possible for improved performance. + +See how to [configure NFS](../nfs.md). + +WARNING: +Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) +after the release of GitLab 15.6. No further enhancements are planned for this feature. + +Read: + +- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). +- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). + ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) @@ -2191,22 +2196,6 @@ cluster alongside your instance, read how to </a> </div> -## Configure NFS (optional) - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be -unavailable from GitLab 15.0. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Supported modifications for lower user counts (HA) The 3k GitLab reference architecture is the smallest we recommend that achieves High Availability (HA). @@ -2252,6 +2241,10 @@ to be complex. **This setup is only recommended** if you have strong working knowledge and experience in Kubernetes. The rest of this section assumes this. +NOTE: +**Gitaly Cluster is not supported to be run in Kubernetes**. +Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details. + ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats @@ -2262,11 +2255,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK and CPU requirements should translate to most other providers. We hope to update this in the future with further specific cloud provider details. -| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | -|-----------------------------------------------|-------|-------------------------|-----------------|--------------|---------------------------------| -| Webservice | 2 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | 31.8 vCPU, 24.8 GB memory | -| Sidekiq | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 11.8 vCPU, 38.9 GB memory | -| Supporting services such as NGINX, Prometheus | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | 3.9 vCPU, 11.8 GB memory | +| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | +|---------------------|-------|-------------------------|-----------------|--------------|---------------------------------| +| Webservice | 2 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | 31.8 vCPU, 24.8 GB memory | +| Sidekiq | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 11.8 vCPU, 38.9 GB memory | +| Supporting services | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | 3.9 vCPU, 11.8 GB memory | - For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. @@ -2276,24 +2269,24 @@ future with further specific cloud provider details. Next are the backend components that run on static compute VMs via Omnibus (or External PaaS services where applicable): -| Service | Nodes | Configuration | GCP | AWS | -|-------------------------------------------|----------------|-----------------------|-----------------|----------------| -| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | -| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | +| Service | Nodes | Configuration | GCP | AWS | +|-------------------------------------------|-------|-----------------------|-----------------|-------------| +| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | +| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Object storage<sup>4</sup> | - | - | - | - | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -2308,11 +2301,11 @@ card "Kubernetes via Helm Charts" as kubernetes { card "**External Load Balancer**" as elb #6a9be7 together { - collections "**Webservice** x4" as gitlab #32CD32 - collections "**Sidekiq** x4" as sidekiq #ff8dd1 + collections "**Webservice** x2" as gitlab #32CD32 + collections "**Sidekiq** x3" as sidekiq #ff8dd1 } - card "**Supporting Services**" as support + card "**Supporting Services** x2" as support } card "**Internal Load Balancer**" as ilb #9370DB @@ -2375,29 +2368,42 @@ documents how to apply the calculated configuration to the Helm Chart. #### Webservice -Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_. -Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using +Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_. +Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using the [recommended topology](#cluster-topology) because four worker processes are created by default and each pod has other small processes running. For 3,000 users we recommend a total Puma worker count of around 16. With the [provided recommendations](#cluster-topology) this allows the deployment of up to 4 Webservice pods with 4 workers per pod and 2 pods per node. Expand available resources using -the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional +the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional Webservice pod. For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources). #### Sidekiq -Sidekiq pods should generally have 1 vCPU and 2 GB of memory. +Sidekiq pods should generally have 0.9 CPU and 2 GB of memory. [The provided starting point](#cluster-topology) allows the deployment of up to -8 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory +8 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory ratio for each additional pod. For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources). +### Supporting + +The Supporting Node Pool is designed to house all supporting deployments that don't need to be +on the Webservice and Sidekiq pools. + +This includes various deployments related to the Cloud Provider's implementation and supporting +GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/). + +If you wish to make any additional deployments, such as for Monitoring, it's recommended +to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed +specifically to accommodate several additional deployments. However, if your deployments don't fit into the +pool as given, you can increase the node pool accordingly. + <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 535e483cb7d..bddec55ba71 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -18,30 +18,30 @@ full list of reference architectures, see > - **Test requests per second (RPS) rates:** API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS > - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)** -| Service | Nodes | Configuration | GCP | AWS | Azure | -|---------------------------------------------------|----------------|-------------------------|------------------|----------------|----------------| -| External load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | -| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| PostgreSQL<sup>1</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | `D32s v3` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | -| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` | `D64s v3` | -| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| GitLab Rails | 12 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` | -| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Service | Nodes | Configuration | GCP | AWS | Azure | +|------------------------------------------|-------|-------------------------|------------------|---------------|-----------| +| External load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | +| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| PostgreSQL<sup>1</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | `D32s v3` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | +| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` | `D64s v3` | +| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| GitLab Rails | 12 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` | +| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Object storage<sup>4</sup> | - | - | - | - | - | +| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -142,10 +142,12 @@ Before starting, you should take note of the following requirements / guidance f This reference architecture was built and tested on Google Cloud Platform (GCP) using the [Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms) -CPU platform. On different hardware you may find that adjustments, either lower -or higher, are required for your CPU or node counts. For more information, see -our [Sysbench](https://github.com/akopytov/sysbench)-based -[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks). +CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)). + +Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported. + +NOTE: +Any "burstable" instance types are not recommended due to inconsistent performance. ### Supported infrastructure @@ -186,10 +188,11 @@ To set up GitLab and its components to accommodate up to 50,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. +1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) + to have shared disk storage service as an alternative to Gitaly or object + storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. -1. [Configure NFS](#configure-nfs) - to have shared disk storage service for certain GitLab operations (non Gitaly or Object Storage). The servers start on the same 10.6.0.0/24 private network range, and can connect to each other freely on these addresses. @@ -268,7 +271,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather than `HTTP(S)` protocol. This will pass the connection to the application node's NGINX service untouched. NGINX will have the SSL certificate and listen on port 443. -See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Load balancer terminates SSL without backend SSL @@ -279,7 +282,7 @@ terminating SSL. Since communication between the load balancer and GitLab will not be secure, there is some additional configuration needed. See the -[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) +[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination) for details. ### Load balancer terminates SSL with backend SSL @@ -291,8 +294,8 @@ end users will see. Traffic will also be secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be -added to GitLab to configure SSL certificates. See -[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +added to GitLab to configure SSL certificates. See the +[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Readiness checks @@ -2031,7 +2034,7 @@ On each node perform the following: 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -1. If you're [using NFS](#configure-nfs): +1. If you're [using NFS](#configure-nfs-optional): 1. If necessary, install the NFS client utility packages using the following commands: @@ -2090,7 +2093,7 @@ On each node perform the following: When you specify `https` in the `external_url`, as in the previous example, GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the certificates aren't present, NGINX will fail to start. For more information, see -the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). ### GitLab Rails post-configuration @@ -2195,7 +2198,7 @@ To configure the Monitoring node: ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -It's recommended over [NFS](#configure-nfs) and in general it's better +It's recommended over [NFS](#configure-nfs-optional) and in general it's better in larger setups as object storage is typically much more performant, reliable, and scalable. @@ -2207,7 +2210,6 @@ GitLab has been tested on a number of object storage providers: - [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm) - [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html) - [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) -- On-premises hardware and appliances from various storage vendors. - MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation. There are two ways of specifying object storage configuration in GitLab: @@ -2217,25 +2219,13 @@ There are two ways of specifying object storage configuration in GitLab: - [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). -Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. - -For configuring object storage in GitLab 13.1 and earlier, or for storage types not -supported by consolidated configuration form, refer to the following guides based -on what features you intend to use: - -|Object storage type|Supported by consolidated configuration?| -|-------------------|----------------------------------------| -| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No | -| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes | -| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes | -| [Uploads](../uploads.md#using-object-storage) | Yes | -| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No | -| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes | -| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | -| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | -| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | -| [Terraform state files](../terraform_state.md#using-object-storage) | Yes | +The consolidated form is used in the following examples when available. + +NOTE: +When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). +The previous [background upload](../../development/uploads/index.md#direct-upload) mode, +which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2248,12 +2238,28 @@ in the future. </a> </div> -## Enable incremental logging +### Enable incremental logging GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. +## Configure NFS (optional) + +[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) +are recommended over NFS wherever possible for improved performance. + +See how to [configure NFS](../nfs.md). + +WARNING: +Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) +after the release of GitLab 15.6. No further enhancements are planned for this feature. + +Read: + +- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). +- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). + ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) @@ -2270,22 +2276,6 @@ cluster alongside your instance, read how to </a> </div> -## Configure NFS - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be -unavailable from GitLab 15.0. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Cloud Native Hybrid reference architecture with Helm Charts (alternative) As an alternative approach, you can also run select components of GitLab as Cloud Native @@ -2306,6 +2296,10 @@ to be complex. **This setup is only recommended** if you have strong working knowledge and experience in Kubernetes. The rest of this section assumes this. +NOTE: +**Gitaly Cluster is not supported to be run in Kubernetes**. +Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details. + ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats @@ -2316,11 +2310,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK and CPU requirements should translate to most other providers. We hope to update this in the future with further specific cloud provider details. -| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | -|-----------------------------------------------|-------|-------------------------|-----------------|--------------|---------------------------------| -| Webservice | 16 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `m5.8xlarge` | 510 vCPU, 472 GB memory | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory | -| Supporting services such as NGINX, Prometheus | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory | +| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | +|---------------------|-------|-------------------------|-----------------|--------------|---------------------------------| +| Webservice | 16 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `m5.8xlarge` | 510 vCPU, 472 GB memory | +| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory | +| Supporting services | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory | - For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. @@ -2330,25 +2324,25 @@ future with further specific cloud provider details. Next are the backend components that run on static compute VMs via Omnibus (or External PaaS services where applicable): -| Service | Nodes | Configuration | GCP | AWS | -|------------------------------------------|----------------|------------------------|------------------|----------------| -| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| PostgreSQL<sup>1</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | -| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` | -| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | +| Service | Nodes | Configuration | GCP | AWS | +|------------------------------------------|-------|------------------------|------------------|---------------| +| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| PostgreSQL<sup>1</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | +| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` | +| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Object storage<sup>4</sup> | - | - | - | - | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -2363,11 +2357,11 @@ card "Kubernetes via Helm Charts" as kubernetes { card "**External Load Balancer**" as elb #6a9be7 together { - collections "**Webservice** x4" as gitlab #32CD32 + collections "**Webservice** x16" as gitlab #32CD32 collections "**Sidekiq** x4" as sidekiq #ff8dd1 } - card "**Supporting Services**" as support + card "**Supporting Services** x2" as support } card "**Internal Load Balancer**" as ilb #9370DB @@ -2433,29 +2427,42 @@ documents how to apply the calculated configuration to the Helm Chart. #### Webservice -Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_. -Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using +Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_. +Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using the [recommended topology](#cluster-topology) because four worker processes are created by default and each pod has other small processes running. For 50,000 users we recommend a total Puma worker count of around 320. With the [provided recommendations](#cluster-topology) this allows the deployment of up to 80 Webservice pods with 4 workers per pod and 5 pods per node. Expand available resources using -the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional +the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional Webservice pod. For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources). #### Sidekiq -Sidekiq pods should generally have 1 vCPU and 2 GB of memory. +Sidekiq pods should generally have 0.9 CPU and 2 GB of memory. [The provided starting point](#cluster-topology) allows the deployment of up to -14 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory +14 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory ratio for each additional pod. For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources). +### Supporting + +The Supporting Node Pool is designed to house all supporting deployments that don't need to be +on the Webservice and Sidekiq pools. + +This includes various deployments related to the Cloud Provider's implementation and supporting +GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/). + +If you wish to make any additional deployments, such as for Monitoring, it's recommended +to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed +specifically to accommodate several additional deployments. However, if your deployments don't fit into the +pool as given, you can increase the node pool accordingly. + <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index e19440e7660..0e599df7c1f 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -25,29 +25,29 @@ costly-to-operate environment by using the > - **Test requests per second (RPS) rates:** API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS > - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)** -| Service | Nodes | Configuration | GCP | AWS | Azure | -|--------------------------------------------|----------------|-------------------------|-----------------|----------------|----------------| -| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | -| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | `D8s v3` | -| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | -| GitLab Rails | 3 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | `F16s v2` | -| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | +| Service | Nodes | Configuration | GCP | AWS | Azure | +|-------------------------------------------|-------|-------------------------|-----------------|--------------|----------| +| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | +| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | `D8s v3` | +| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` | +| GitLab Rails | 3 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | `F16s v2`| +| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | +| Object storage<sup>4</sup> | - | - | - | - | - | +| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -145,10 +145,12 @@ Before starting, you should take note of the following requirements / guidance f This reference architecture was built and tested on Google Cloud Platform (GCP) using the [Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms) -CPU platform. On different hardware you may find that adjustments, either lower -or higher, are required for your CPU or node counts. For more information, see -our [Sysbench](https://github.com/akopytov/sysbench)-based -[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks). +CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)). + +Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported. + +NOTE: +Any "burstable" instance types are not recommended due to inconsistent performance. ### Supported infrastructure @@ -189,12 +191,11 @@ To set up GitLab and its components to accommodate up to 5,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, - more advanced code search across your entire GitLab instance. 1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) to have shared disk storage service as an alternative to Gitaly or object - storage. You can skip this step if you're not using GitLab Pages (which - requires NFS). + storage. +1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, + more advanced code search across your entire GitLab instance. The servers start on the same 10.6.0.0/24 private network range, and can connect to each other freely on these addresses. @@ -261,7 +262,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather than `HTTP(S)` protocol. This passes the connection to the application node's NGINX service untouched. NGINX has the SSL certificate and listen on port 443. -See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Load balancer terminates SSL without backend SSL @@ -272,7 +273,7 @@ terminating SSL. Since communication between the load balancer and GitLab is not secure, there is some additional configuration needed. See the -[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) +[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination) for details. ### Load balancer terminates SSL with backend SSL @@ -284,8 +285,8 @@ end users see. Traffic is also secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection is secure all the way. However, configuration needs to be -added to GitLab to configure SSL certificates. See -[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +added to GitLab to configure SSL certificates. See the +[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html) for details on managing SSL certificates and configuring NGINX. ### Readiness checks @@ -2005,7 +2006,7 @@ On each node perform the following: When you specify `https` in the `external_url`, as in the previous example, GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the certificates aren't present, NGINX fails to start. For more information, see -the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). ### GitLab Rails post-configuration @@ -2128,7 +2129,6 @@ GitLab has been tested on a number of object storage providers: - [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm) - [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html) - [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) -- On-premises hardware and appliances from various storage vendors. - MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation. There are two ways of specifying object storage configuration in GitLab: @@ -2138,25 +2138,13 @@ There are two ways of specifying object storage configuration in GitLab: - [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its own object storage [connection and configuration](../object_storage.md#connection-settings). -Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up. - -For configuring object storage in GitLab 13.1 and earlier, or for storage types not -supported by consolidated configuration form, refer to the following guides based -on what features you intend to use: - -|Object storage type|Supported by consolidated configuration?| -|-------------------|----------------------------------------| -| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No | -| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes | -| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes | -| [Uploads](../uploads.md#using-object-storage) | Yes | -| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No | -| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes | -| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | -| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | -| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | -| [Terraform state files](../terraform_state.md#using-object-storage) | Yes | +The consolidated form is used in the following examples when available. + +NOTE: +When using the [storage-specific form](../object_storage.md#storage-specific-configuration) +in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). +The previous [background upload](../../development/uploads/index.md#direct-upload) mode, +which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2169,12 +2157,28 @@ in the future. </a> </div> -## Enable incremental logging +### Enable incremental logging GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes. While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. +## Configure NFS (optional) + +[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) +are recommended over NFS wherever possible for improved performance. + +See how to [configure NFS](../nfs.md). + +WARNING: +Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) +after the release of GitLab 15.6. No further enhancements are planned for this feature. + +Read: + +- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). +- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). + ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) @@ -2191,22 +2195,6 @@ cluster alongside your instance, read how to </a> </div> -## Configure NFS (optional) - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be -unavailable from GitLab 15.0. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Cloud Native Hybrid reference architecture with Helm Charts (alternative) As an alternative approach, you can also run select components of GitLab as Cloud Native @@ -2227,6 +2215,10 @@ to be complex. **This setup is only recommended** if you have strong working knowledge and experience in Kubernetes. The rest of this section assumes this. +NOTE: +**Gitaly Cluster is not supported to be run in Kubernetes**. +Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details. + ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats @@ -2251,24 +2243,24 @@ future with further specific cloud provider details. Next are the backend components that run on static compute VMs via Omnibus (or External PaaS services where applicable): -| Service | Nodes | Configuration | GCP | AWS | -|-------------------------------------------|----------------|-----------------------|-----------------|----------------| -| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | -| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | -| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | -| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | -| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | +| Service | Nodes | Configuration | GCP | AWS | +|-------------------------------------------|-------|-----------------------|-----------------|--------------| +| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | +| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | +| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | +| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | +| Object storage<sup>4</sup> | - | - | - | - | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. -4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. +4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section. 5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`. <!-- markdownlint-enable MD029 --> @@ -2283,11 +2275,11 @@ card "Kubernetes via Helm Charts" as kubernetes { card "**External Load Balancer**" as elb #6a9be7 together { - collections "**Webservice** x4" as gitlab #32CD32 - collections "**Sidekiq** x4" as sidekiq #ff8dd1 + collections "**Webservice** x5" as gitlab #32CD32 + collections "**Sidekiq** x3" as sidekiq #ff8dd1 } - card "**Supporting Services**" as support + card "**Supporting Services** x2" as support } card "**Internal Load Balancer**" as ilb #9370DB @@ -2350,29 +2342,42 @@ documents how to apply the calculated configuration to the Helm Chart. #### Webservice -Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_. -Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using +Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_. +Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using the [recommended topology](#cluster-topology) because four worker processes are created by default and each pod has other small processes running. For 5,000 users we recommend a total Puma worker count of around 40. With the [provided recommendations](#cluster-topology) this allows the deployment of up to 10 Webservice pods with 4 workers per pod and 2 pods per node. Expand available resources using -the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional +the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional Webservice pod. For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources). #### Sidekiq -Sidekiq pods should generally have 1 vCPU and 2 GB of memory. +Sidekiq pods should generally have 0.9 CPU and 2 GB of memory. [The provided starting point](#cluster-topology) allows the deployment of up to -8 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory +8 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory ratio for each additional pod. For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources). +### Supporting + +The Supporting Node Pool is designed to house all supporting deployments that don't need to be +on the Webservice and Sidekiq pools. + +This includes various deployments related to the Cloud Provider's implementation and supporting +GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/). + +If you wish to make any additional deployments, such as for Monitoring, it's recommended +to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed +specifically to accommodate several additional deployments. However, if your deployments don't fit into the +pool as given, you can increase the node pool accordingly. + <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> diff --git a/doc/administration/repository_checks.md b/doc/administration/repository_checks.md index a97c8611239..4615499d6fa 100644 --- a/doc/administration/repository_checks.md +++ b/doc/administration/repository_checks.md @@ -20,7 +20,7 @@ committed to a repository. GitLab administrators can: To check a project's repository using GitLab UI: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Projects**. 1. Select the project to check. 1. In the **Repository check** section, select **Trigger repository check**. @@ -32,7 +32,7 @@ project page in the Admin Area. If the checks fail, see [what to do](#what-to-do Instead of checking repositories manually, GitLab can be configured to run the checks periodically: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository** (`/admin/application_settings/repository`). 1. Expand the **Repository maintenance** section. 1. Enable **Enable repository checks**. @@ -68,7 +68,7 @@ You can run [`git fsck`](https://git-scm.com/docs/git-fsck) using the command li 1. Run the check. For example: ```shell - sudo /opt/gitlab/embedded/bin/git -C /var/opt/gitlab/git-data/repositories/@hashed/0b/91/0b91...f9.git fsck + sudo -u git /opt/gitlab/embedded/bin/git -C /var/opt/gitlab/git-data/repositories/@hashed/0b/91/0b91...f9.git fsck ``` ## What to do if a check failed @@ -81,7 +81,15 @@ If a repository check fails, locate the error in the [`repocheck.log` file](logs If periodic repository checks cause false alarms, you can clear all repository check states: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository** (`/admin/application_settings/repository`). 1. Expand the **Repository maintenance** section. 1. Select **Clear all repository checks**. + +### Error: `failed to parse commit <commit SHA> from object database for commit-graph` + +You can see a `failed to parse commit <commit SHA> from object database for commit-graph` error in repository check logs. This error occurs if your `commit-graph` cache is out +of date. The `commit-graph` cache is an auxiliary cache and is not required for regular Git operations. + +While the message can be safely ignored, see the issue [error: Could not read from object database for commit-graph](https://gitlab.com/gitlab-org/gitaly/-/issues/2359) +for more details. diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index f001ba35bca..482cbd97e37 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -146,7 +146,7 @@ Omnibus stores the repositories in a `repositories` subdirectory of the `git-dat After you [configure](#configure-repository-storage-paths) multiple repository storage paths, you can choose where new repositories are stored: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository** and expand the **Repository storage** section. 1. Enter values in the **Storage nodes for new repositories** fields. diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index 7ce629b5d71..865daba9e89 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -79,7 +79,7 @@ Administrators can look up a project's hashed path from its name or ID using: To look up a project's hash path in the Admin Area: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Projects** and select the project. The **Gitaly relative path** is displayed there and looks similar to: diff --git a/doc/administration/restart_gitlab.md b/doc/administration/restart_gitlab.md index 6625039504a..e5ec12054b8 100644 --- a/doc/administration/restart_gitlab.md +++ b/doc/administration/restart_gitlab.md @@ -102,7 +102,7 @@ depend on those files. ## Installations from source -If you have followed the official installation guide to +If you have followed the official installation guide to [install GitLab from source](../install/installation.md), run the following command to restart GitLab: ```shell diff --git a/doc/administration/server_hooks.md b/doc/administration/server_hooks.md index 1a47dc4ccf2..4148abf1bdc 100644 --- a/doc/administration/server_hooks.md +++ b/doc/administration/server_hooks.md @@ -30,7 +30,7 @@ alternatives to server hooks include: To create server hooks for a repository: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. Go to **Overview > Projects** and select the project you want to add a server hook to. 1. On the page that appears, locate the value of **Gitaly relative path**. This path is where server hooks must be located. - If you are using [hashed storage](repository_storage_types.md#hashed-storage), see @@ -56,8 +56,7 @@ If the server hook code is properly implemented, it should execute when the Git ## Create global server hooks for all repositories -To create a Git hook that applies to all repositories, set a global server hook. The default global server hook directory -is in the GitLab Shell directory. Any server hook added there applies to all repositories, including: +To create a Git hook that applies to all repositories, set a global server hook. Global server hooks also apply to: - [Project and group wiki](../user/project/wiki/index.md) repositories. Their storage directory names are in the format `<id>.wiki.git`. @@ -66,36 +65,36 @@ is in the GitLab Shell directory. Any server hook added there applies to all rep ### Choose a server hook directory -Before creating a global server hook, you must choose a directory for it. The default global server hook directory: +Before creating a global server hook, you must choose a directory for it. -- For Omnibus GitLab installations is usually `/opt/gitlab/embedded/service/gitlab-shell/hooks`. -- For an installation from source is usually `/home/git/gitlab-shell/hooks`. +For Omnibus GitLab installations, the directory is set in `gitlab.rb` under `gitaly['custom_hooks_dir']`. You can either: -To use a different directory for global server hooks, set `custom_hooks_dir` in Gitaly configuration: +- Use the default suggestion of the `/var/opt/gitlab/gitaly/custom_hooks` directory by uncommenting it. +- Add your own setting. -- For Omnibus installations, set in `gitlab.rb`. -- For source installations, the configuration location depends on the GitLab version. For: - - GitLab 13.0 and earlier, set in `gitlab-shell/config.yml`. - - GitLab 13.1 and later, set in `gitaly/config.toml` under the `[hooks]` section. However, GitLab honors the - `custom_hooks_dir` value in `gitlab-shell/config.yml` if the value in `gitaly/config.toml` is blank or non-existent. +For installations from source: + +- The directory is set in a configuration file. The location of the configuration file depends on the GitLab version: + - For GitLab 13.1 and later, the directory is set in `gitaly/config.toml` under the `[hooks]` section. However, + GitLab honors the `custom_hooks_dir` value in `gitlab-shell/config.yml` if the value in `gitaly/config.toml` is blank + or non-existent. + - For GitLab 13.0 and earlier, the directory set in `gitlab-shell/config.yml`. +- The default directory is `/home/git/gitlab-shell/hooks`. ### Create the global server hook To create a global server hook for all repositories: 1. On the GitLab server, go to the configured global server hook directory. -1. In the configured global server hook directory: - - To create a single server hook, create a file with a name that matches the hook type. For example, for a - `pre-receive` server hook, the filename should be `pre-receive` with no extension. - - To create many server hooks, create a directory for the hooks that matches the hook type. For example, for a - `pre-receive` server hook, the directory name should be `pre-receive.d`. Put the files for the hook in that directory. -1. Inside this new directory, add your server hook. Server hooks can be in any programming language. Ensure the +1. In the configured global server hook directory, create a directory for the hooks that matches the hook type. For example, for a `pre-receive` server hook, the directory name should be `pre-receive.d`. +1. Inside this new directory, add your server hooks. Server hooks can be in any programming language. Ensure the [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) at the top reflects the language type. For example, if the script is in Ruby the shebang is probably `#!/usr/bin/env ruby`. 1. Make the hook file executable, ensure that it's owned by the Git user, and ensure it does not match the backup file pattern (`*~`). -If the server hook code is properly implemented, it should execute when the Git hook is next triggered. +If the server hook code is properly implemented, it should execute when the Git hook is next triggered. Hooks are executed in alphabetical order by filename in the hook type +subdirectories. ## Chained server hooks diff --git a/doc/administration/sidekiq/extra_sidekiq_processes.md b/doc/administration/sidekiq/extra_sidekiq_processes.md index 1cd3771c94d..b774b0d3e14 100644 --- a/doc/administration/sidekiq/extra_sidekiq_processes.md +++ b/doc/administration/sidekiq/extra_sidekiq_processes.md @@ -95,7 +95,7 @@ To start multiple processes: To view the Sidekiq processes in GitLab: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. ## Negate settings diff --git a/doc/administration/static_objects_external_storage.md b/doc/administration/static_objects_external_storage.md index 21949388f19..a288b19f164 100644 --- a/doc/administration/static_objects_external_storage.md +++ b/doc/administration/static_objects_external_storage.md @@ -16,7 +16,7 @@ storage such as a content delivery network (CDN). To configure external storage for static objects: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository**. 1. Expand the **External storage for repository static objects** section. 1. Enter the base URL and an arbitrary token. When you [set up external storage](#set-up-external-storage), diff --git a/doc/administration/system_hooks.md b/doc/administration/system_hooks.md index cb141a18ae6..0d70744e942 100644 --- a/doc/administration/system_hooks.md +++ b/doc/administration/system_hooks.md @@ -56,7 +56,7 @@ for Push and Tag events, but we never display commits. To create a system hook: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **System Hooks**. 1. Provide the **URL** and **Secret Token**. 1. Select the checkbox next to each optional **Trigger** you want to enable. diff --git a/doc/administration/terraform_state.md b/doc/administration/terraform_state.md index 7a8d7774948..5a272025987 100644 --- a/doc/administration/terraform_state.md +++ b/doc/administration/terraform_state.md @@ -78,8 +78,8 @@ Terraform state files are stored locally, follow the steps below. ## Using object storage **(FREE SELF)** -Instead of storing Terraform state files on disk, we recommend the use of -[one of the supported object storage options](object_storage.md#options). +Instead of storing Terraform state files on disk, we recommend the use of +[one of the supported object storage options](object_storage.md#options). This configuration relies on valid credentials to be configured already. [Read more about using object storage with GitLab](object_storage.md). diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md index aa4dbec4f95..40bb15ecb70 100644 --- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md +++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md @@ -28,22 +28,6 @@ mentioned above, we recommend running these scripts under the supervision of a Support Engineer, who can also verify that they continue to work as they should and, if needed, update the script for the latest version of GitLab. -## Find specific methods for an object - -```ruby -Array.methods.select { |m| m.to_s.include? "sing" } -Array.methods.grep(/sing/) -``` - -## Find method source - -```ruby -instance_of_object.method(:foo).source_location - -# Example for when we would call project.private? -project.method(:private?).source_location -``` - ## Attributes View available attributes, formatted using pretty print (`pp`). @@ -78,28 +62,6 @@ Notify.test_email(e, "Test email for #{n}", 'Test email').deliver_now Notify.test_email(u.email, "Test email for #{u.name}", 'Test email').deliver_now ``` -## Limiting output - -Adding a semicolon(`;`) and a follow-up statement at the end of a statement prevents the default implicit return output. This can be used if you are already explicitly printing details and potentially have a lot of return output: - -```ruby -puts ActiveRecord::Base.descendants; :ok -Project.select(&:pages_deployed?).each {|p| puts p.pages_url }; true -``` - -## Get or store the result of last operation - -Underscore(`_`) represents the implicit return of the previous statement. You can use this to quickly assign a variable from the output of the previous command: - -```ruby -Project.last -# => #<Project id:2537 root/discard>> -project = _ -# => #<Project id:2537 root/discard>> -project.id -# => 2537 -``` - ## Open object in `irb` Sometimes it is easier to go through a method if you are in the context of the object. You can shim into the namespace of `Object` to let you open `irb` in the context of any object: @@ -115,15 +77,6 @@ irb(#<Project>)> web_url # => "https://gitlab-example/root/discard" ``` -## Query the database using an ActiveRecord Model - -```ruby -m = Model.where('attribute like ?', 'ex%') - -# for example to query the projects -projects = Project.where('path like ?', 'Oumua%') -``` - ## View all keys in cache ```ruby @@ -166,18 +119,6 @@ Benchmark.bm do |x| end ``` -## Feature flags - -### Show all feature flags that are enabled - -```ruby -# Regular output -Feature.all - -# Nice output -Feature.all.map {|f| [f.name, f.state]} -``` - ## Projects ### Clear a project's cache @@ -813,30 +754,6 @@ subgroup.members.map(&:errors).map(&:full_messages) subgroup.members_and_requesters.map(&:errors).map(&:full_messages) ``` -## Authentication - -### Re-enable standard web sign-in form - -Re-enable the standard username and password-based sign-in form if it was disabled as a [Sign-in restriction](../../user/admin_area/settings/sign_in_restrictions.md#password-authentication-enabled). - -You can use this method when a configured external authentication provider (through SSO or an LDAP configuration) is facing an outage and direct sign-in access to GitLab is required. - -```ruby -Gitlab::CurrentSettings.update!(password_authentication_enabled_for_web: true) -``` - -## SCIM - -### Find groups using an SQL query - -Find and store an array of groups based on an SQL query: - -```ruby -# Finds groups and subgroups that end with '%oup' -Group.find_by_sql("SELECT * FROM namespaces WHERE name LIKE '%oup'") -=> [#<Group id:3 @test-group>, #<Group id:4 @template-group/template-subgroup>] -``` - ## Routes ### Remove redirecting routes @@ -1173,6 +1090,38 @@ This content has been converted to a Rake task, see [verify database values can ## Geo +### Reverify all uploads (or any SSF data type which is verified) + +1. SSH into a GitLab Rails node in the primary Geo site. +1. Open [Rails console](../operations/rails_console.md). +1. Mark all uploads as "pending verification": + + ```ruby + Upload.verification_state_table_class.each_batch do |relation| + relation.update_all(verification_state: 0) + end + ``` + +1. This will cause the primary to start checksumming all Uploads. +1. When a primary successfully checksums a record, then all secondaries rechecksum as well, and they compare the values. + +A similar thing can be done for all Models handled by the [Geo Self-Service Framework](../../development/geo/framework.md) which have implemented verification: + +- `LfsObject` +- `MergeRequestDiff` +- `Packages::PackageFile` +- `Terraform::StateVersion` +- `SnippetRepository` +- `Ci::PipelineArtifact` +- `PagesDeployment` +- `Upload` +- `Ci::JobArtifact` +- `Ci::SecureFile` + +NOTE: +`GroupWikiRepository` is not in the previous list since verification is not implemented. +There is an [issue to implement this functionality in the Admin UI](https://gitlab.com/gitlab-org/gitlab/-/issues/364729). + ### Artifacts #### Find failed artifacts @@ -1362,54 +1311,3 @@ Prints the metrics saved in `conversational_development_index_metrics`. ```shell rake gitlab:usage_data:generate_and_send ``` - -## Elasticsearch - -### Configuration attributes - -Open the rails console (`gitlab rails c`) and run the following command to see all the available attributes: - -```ruby -ApplicationSetting.last.attributes -``` - -Among other attributes, the output contains all the settings available in the [Elasticsearch Integration page](../../integration/advanced_search/elasticsearch.md), such as `elasticsearch_indexing`, `elasticsearch_url`, `elasticsearch_replicas`, and `elasticsearch_pause_indexing`. - -#### Setting attributes - -You can then set anyone of Elasticsearch integration settings by issuing a command similar to: - -```ruby -ApplicationSetting.last.update(elasticsearch_url: '<your ES URL and port>') - -#or - -ApplicationSetting.last.update(elasticsearch_indexing: false) -``` - -#### Getting attributes - -You can then check if the settings have been set in the [Elasticsearch Integration page](../../integration/advanced_search/elasticsearch.md) or in the rails console by issuing: - -```ruby -Gitlab::CurrentSettings.elasticsearch_url - -#or - -Gitlab::CurrentSettings.elasticsearch_indexing -``` - -#### Changing the Elasticsearch password - -```ruby -es_url = Gitlab::CurrentSettings.current_application_settings - -# Confirm the current ElasticSearch URL -es_url.elasticsearch_url - -# Set the ElasticSearch URL -es_url.elasticsearch_url = "http://<username>:<password>@your.es.host:<port>" - -# Save the change -es_url.save! -``` diff --git a/doc/administration/troubleshooting/linux_cheat_sheet.md b/doc/administration/troubleshooting/linux_cheat_sheet.md index 6ff6e562a7d..c1a428018c2 100644 --- a/doc/administration/troubleshooting/linux_cheat_sheet.md +++ b/doc/administration/troubleshooting/linux_cheat_sheet.md @@ -204,7 +204,7 @@ or you can build it from source if you have the Rust compiler. #### How to use the tool -First run the tool with `summary` flag to get a summary of the top processes sorted by time spent actively performing tasks. +First run the tool with `summary` flag to get a summary of the top processes sorted by time spent actively performing tasks. You can also sort based on total time, # of system calls made, PID #, and # of child processes using the `-s` or `--sort` flag. The number of results defaults to 25 processes, but can be changed using the `-c`/`--count` option. See `--help` for full details. diff --git a/doc/administration/troubleshooting/ssl.md b/doc/administration/troubleshooting/ssl.md index e1cd92a788f..c5f3f0ed8d1 100644 --- a/doc/administration/troubleshooting/ssl.md +++ b/doc/administration/troubleshooting/ssl.md @@ -13,7 +13,7 @@ main SSL documentation: - [Omnibus SSL Configuration](https://docs.gitlab.com/omnibus/settings/ssl.html). - [Self-signed certificates or custom Certification Authorities for GitLab Runner](https://docs.gitlab.com/runner/configuration/tls-self-signed.html). -- [Manually configuring HTTPS](https://docs.gitlab.com/omnibus/settings/nginx.html#manually-configuring-https). +- [Configure HTTPS manually](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-https-manually). ## Using an internal CA certificate with GitLab @@ -251,3 +251,13 @@ You must specify that Git should use OpenSSL: ```shell git config --system http.sslbackend openssl ``` + +Alternatively, you can ignore SSL verification by running: + +WARNING: +Proceed with caution when [ignoring SSL](https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpsslVerify) +due to the potential security issues associated with disabling this option at global level. Use this option _only_ when troubleshooting, and reinstate SSL verification immediately after. + +```shell +git config --global http.sslVerify false +``` diff --git a/doc/administration/troubleshooting/tracing_correlation_id.md b/doc/administration/troubleshooting/tracing_correlation_id.md index ee59b7c2504..917e27bab70 100644 --- a/doc/administration/troubleshooting/tracing_correlation_id.md +++ b/doc/administration/troubleshooting/tracing_correlation_id.md @@ -6,6 +6,6 @@ remove_date: '2022-11-12' This document was moved to [another location](../logs/tracing_correlation_id.md). <!-- This redirect file can be deleted after 2022-11-12. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> <!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
\ No newline at end of file diff --git a/doc/administration/user_settings.md b/doc/administration/user_settings.md index 2e879f8789d..0a3f351c695 100644 --- a/doc/administration/user_settings.md +++ b/doc/administration/user_settings.md @@ -8,10 +8,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w GitLab administrators can modify user settings for the entire GitLab instance. -## Prevent users from creating top-level groups +## Prevent new users from creating top-level groups -By default, new users can create top-level groups. To disable your users' -ability to create top-level groups: +By default, new users can create top-level groups. To disable new users' +ability to create top-level groups (does not affect existing users' setting): **Omnibus GitLab installations** @@ -33,6 +33,13 @@ ability to create top-level groups: 1. [Restart GitLab](restart_gitlab.md#installations-from-source). +### Prevent existing users from creating top-level groups + +Administrators can: + +- Use the Admin Area to [prevent an existing user from creating top-level groups](../user/admin_area/index.md#prevent-a-user-from-creating-groups). +- Use the [modify an existing user API endpoint](../api/users.md#user-modification) to change the `can_create_group` setting. + ## Prevent users from changing their usernames By default, new users can change their usernames. To disable your users' diff --git a/doc/administration/whats-new.md b/doc/administration/whats-new.md index 05c769cf20c..d8003d579ac 100644 --- a/doc/administration/whats-new.md +++ b/doc/administration/whats-new.md @@ -6,37 +6,39 @@ info: For assistance with this What's new page, see https://about.gitlab.com/han # What's new **(FREE)** -With each monthly release, GitLab includes some of the highlights from the last 10 -GitLab versions in the **What's new** feature. To access it: +You can view some of the highlights from the last 10 +GitLab versions in the **What's new** feature. It lists new features available in different +[GitLab tiers](https://about.gitlab.com/pricing/). -1. In the top navigation bar, select the **{question}** icon. -1. Select **What's new** from the menu. - -The **What's new** describes new features available in multiple -[GitLab tiers](https://about.gitlab.com/pricing/). While all users can see the -feature list, the feature list is tailored to your subscription type: +All users can see the feature list, but the entries might differ depending on the subscription type: -- Features only available to self-managed installations are not shown on GitLab.com. - Features only available on GitLab.com are not shown to self-managed installations. +- Features only available to self-managed installations are not shown on GitLab.com. -## Self-managed installations + NOTE: + For self-managed installations, the updated **What's new** is included + in the first patch release after a new version, such as `13.10.1`. -Due to our release post process, the content for **What's new** is not finalized -when a new version (`.0` release) is cut. The updated **What's new** is included -in the first patch release, such as `13.10.1`. +## Access What's new + +To access the **What's new** feature: + +1. On the top bar, select the **{question}** icon. +1. Select **What's new** from the menu. -## Configure What's new variant +## Configure What's new -You can configure the What's new variant: +You can configure **What's new** to display features based on the tier, +or you can hide it. To configure it: -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Settings > Preferences**, then expand **What's new**. -1. Choose one of the following options: +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > Preferences**. +1. Expand **What's new**, and choose one of the following options: | Option | Description | | ------ | ----------- | - | Enable What's new: All tiers | What's new presents new features from all tiers to help you keep track of all new features. | - | Enable What's new: Current tier only | What's new presents new features for your current subscription tier, while hiding new features not available to your subscription tier. | - | Disable What's new | What's new is disabled and can no longer be viewed. | + | Enable What's new: All tiers | Presents new features from all tiers. | + | Enable What's new: Current tier only | Presents new features for your current subscription tier, and hides new features outside of your tier. | + | Disable What's new | Disables this feature, so it no longer displays under the **{question}** icon. | -1. Save your changes. +1. Select **Save changes**. diff --git a/doc/api/api_resources.md b/doc/api/api_resources.md index aa942582e9c..7d613852d23 100644 --- a/doc/api/api_resources.md +++ b/doc/api/api_resources.md @@ -31,7 +31,10 @@ The following API resources are available in the project context: | [Commits](commits.md) | `/projects/:id/repository/commits`, `/projects/:id/statuses` | | [Container Registry](container_registry.md) | `/projects/:id/registry/repositories` | | [Custom attributes](custom_attributes.md) | `/projects/:id/custom_attributes` (also available for groups and users) | +| [Composer distributions](packages/composer.md) | `/projects/:id/packages/composer` (also available for groups) | +| [Conan distributions](packages/conan.md) | `/projects/:id/packages/conan` (also available standalone) | | [Debian distributions](packages/debian_project_distributions.md) | `/projects/:id/debian_distributions` (also available for groups) | +| [Debian packages](packages/debian.md) | `/projects/:id/packages/debian` (also available for groups) | | [Dependencies](dependencies.md) **(ULTIMATE)** | `/projects/:id/dependencies` | | [Deploy keys](deploy_keys.md) | `/projects/:id/deploy_keys` (also available standalone) | | [Deploy tokens](deploy_tokens.md) | `/projects/:id/deploy_tokens` (also available for groups and standalone) | @@ -43,6 +46,8 @@ The following API resources are available in the project context: | [Feature Flag User Lists](feature_flag_user_lists.md) | `/projects/:id/feature_flags_user_lists` | | [Feature Flags](feature_flags.md) | `/projects/:id/feature_flags` | | [Freeze Periods](freeze_periods.md) | `/projects/:id/freeze_periods` | +| [Go Proxy](packages/go_proxy.md) | `/projects/:id/packages/go` | +| [Helm repository](packages/helm.md) | `/projects/:id/packages/helm_repository` | | [Integrations](integrations.md) (Formerly "services") | `/projects/:id/integrations` | | [Invitations](invitations.md) | `/projects/:id/invitations` (also available for groups) | | [Issue boards](boards.md) | `/projects/:id/boards` | @@ -51,8 +56,10 @@ The following API resources are available in the project context: | [Issues](issues.md) | `/projects/:id/issues` (also available for groups and standalone) | | [Iterations](iterations.md) **(PREMIUM)** | `/projects/:id/iterations` (also available for groups) | | [Jobs](jobs.md) | `/projects/:id/jobs`, `/projects/:id/pipelines/.../jobs` | +| [Jobs Artifacts](job_artifacts.md) | `/projects/:id/jobs/:job_id/artifacts` | | [Labels](labels.md) | `/projects/:id/labels` | | [Managed licenses](managed_licenses.md) **(ULTIMATE)** | `/projects/:id/managed_licenses` | +| [Maven repository](packages/maven.md) | `/projects/:id/packages/maven` (also available for groups and standalone) | | [Members](members.md) | `/projects/:id/members` (also available for groups) | | [Merge request approvals](merge_request_approvals.md) **(PREMIUM)** | `/projects/:id/approvals`, `/projects/:id/merge_requests/.../approvals` | | [Merge requests](merge_requests.md) | `/projects/:id/merge_requests` (also available for groups and standalone) | @@ -60,6 +67,8 @@ The following API resources are available in the project context: | [Metadata](metadata.md) | `/metadata` | | [Notes](notes.md) (comments) | `/projects/:id/issues/.../notes`, `/projects/:id/snippets/.../notes`, `/projects/:id/merge_requests/.../notes` (also available for groups) | | [Notification settings](notification_settings.md) | `/projects/:id/notification_settings` (also available for groups and standalone) | +| [NPM repository](packages/npm.md) | `/projects/:id/packages/npm` | +| [NuGet packages](packages/nuget.md) | `/projects/:id/packages/nuget` (also available for groups) | | [Packages](packages.md) | `/projects/:id/packages` | | [Pages domains](pages_domains.md) | `/projects/:id/pages` (also available standalone) | | [Pipeline schedules](pipeline_schedules.md) | `/projects/:id/pipeline_schedules` | @@ -78,6 +87,7 @@ The following API resources are available in the project context: | [Protected branches](protected_branches.md) | `/projects/:id/protected_branches` | | [Protected environments](protected_environments.md) | `/projects/:id/protected_environments` | | [Protected tags](protected_tags.md) | `/projects/:id/protected_tags` | +| [PyPI packages](packages/pypi.md) | `/projects/:id/packages/pypi` (also available for groups) | | [Release links](releases/links.md) | `/projects/:id/releases/.../assets/links` | | [Releases](releases/index.md) | `/projects/:id/releases` | | [Remote mirrors](remote_mirrors.md) | `/projects/:id/remote_mirrors` | @@ -85,9 +95,11 @@ The following API resources are available in the project context: | [Repository files](repository_files.md) | `/projects/:id/repository/files` | | [Repository submodules](repository_submodules.md) | `/projects/:id/repository/submodules` | | [Resource label events](resource_label_events.md) | `/projects/:id/issues/.../resource_label_events`, `/projects/:id/merge_requests/.../resource_label_events` (also available for groups) | +| [Ruby gems](packages/rubygems.md) | `/projects/:id/packages/rubygems` | | [Runners](runners.md) | `/projects/:id/runners` (also available standalone) | | [Search](search.md) | `/projects/:id/search` (also available for groups and standalone) | | [Tags](tags.md) | `/projects/:id/repository/tags` | +| [Terraform modules](packages/terraform-modules.md) | `/projects/:id/packages/terraform/mdoules` (also available standalone) | | [User-starred metrics dashboards](metrics_user_starred_dashboards.md ) | `/projects/:id/metrics/user_starred_dashboards` | | [Visual Review discussions](visual_review_discussions.md) **(PREMIUM)** | `/projects/:id/merge_requests/:merge_request_id/visual_review_discussions` | | [Vulnerabilities](vulnerabilities.md) **(ULTIMATE)** | `/vulnerabilities/:id` | diff --git a/doc/api/audit_events.md b/doc/api/audit_events.md index 80d7b23d642..5bff3cf49fc 100644 --- a/doc/api/audit_events.md +++ b/doc/api/audit_events.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/container_registry.md b/doc/api/container_registry.md index 0dd2106b4d1..f23641abb4f 100644 --- a/doc/api/container_registry.md +++ b/doc/api/container_registry.md @@ -80,6 +80,11 @@ Example response: } ``` +## Container Registry pagination + +By default, `GET` requests return 20 results at a time because the API results +are [paginated](index.md#pagination). + ## List registry repositories ### Within a project diff --git a/doc/api/custom_attributes.md b/doc/api/custom_attributes.md index 94f924c051d..e15ed4bceee 100644 --- a/doc/api/custom_attributes.md +++ b/doc/api/custom_attributes.md @@ -1,6 +1,6 @@ --- -stage: Ecosystem -group: Integrations +stage: Plan +group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/deployments.md b/doc/api/deployments.md index 01fa4e11884..8b99c21a385 100644 --- a/doc/api/deployments.md +++ b/doc/api/deployments.md @@ -58,6 +58,9 @@ Example response: "started_at": null, "status": "success", "tag": false, + "project": { + "ci_job_token_scope_enabled": false + }, "user": { "id": 1, "name": "Administrator", @@ -128,6 +131,9 @@ Example response: "started_at": null, "status": "success", "tag": false, + "project": { + "ci_job_token_scope_enabled": false + }, "user": { "id": 1, "name": "Administrator", @@ -226,6 +232,9 @@ Example response: "created_at": "2016-08-11T11:32:24.456Z", "started_at": null, "finished_at": "2016-08-11T11:32:35.145Z", + "project": { + "ci_job_token_scope_enabled": false + }, "user": { "id": 1, "name": "Administrator", diff --git a/doc/api/discussions.md b/doc/api/discussions.md index a5610adad79..60ce10590e1 100644 --- a/doc/api/discussions.md +++ b/doc/api/discussions.md @@ -1109,7 +1109,7 @@ GET /projects/:id/repository/commits/:commit_id/discussions | Attribute | Type | Required | Description | | ------------------- | ---------------- | ---------- | ------------ | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | -| `commit_id` | integer | yes | The ID of a commit | +| `commit_id` | string | yes | The SHA of a commit | ```json [ @@ -1237,7 +1237,7 @@ Diff comments contain also position: ```shell curl --header "PRIVATE-TOKEN: <your_access_token>"\ - "https://gitlab.example.com/api/v4/projects/5/repository/commits/11/discussions" + "https://gitlab.example.com/api/v4/projects/5/repository/commits/<commit_id>/discussions" ``` ### Get single commit discussion item @@ -1253,12 +1253,12 @@ Parameters: | Attribute | Type | Required | Description | | ------------------- | -------------- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | -| `commit_id` | integer | yes | The ID of a commit | -| `discussion_id` | integer | yes | The ID of a discussion item | +| `commit_id` | string | yes | The SHA of a commit | +| `discussion_id` | string | yes | The ID of a discussion item | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>"\ - "https://gitlab.example.com/api/v4/projects/5/repository/commits/11/discussions/<discussion_id>" + "https://gitlab.example.com/api/v4/projects/5/repository/commits/<commit_id>/discussions/<discussion_id>" ``` ### Create new commit thread @@ -1294,7 +1294,7 @@ Parameters: ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>"\ - "https://gitlab.example.com/api/v4/projects/5/repository/commits/11/discussions?body=comment" + "https://gitlab.example.com/api/v4/projects/5/repository/commits/<commit_id>/discussions?body=comment" ``` The rules for creating the API request are the same as when @@ -1314,15 +1314,15 @@ Parameters: | Attribute | Type | Required | Description | | ------------------- | -------------- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | -| `commit_id` | integer | yes | The ID of a commit | -| `discussion_id` | integer | yes | The ID of a thread | +| `commit_id` | string | yes | The SHA of a commit | +| `discussion_id` | string | yes | The ID of a thread | | `note_id` | integer | yes | The ID of a thread note | | `body` | string | yes | The content of the note/reply | | `created_at` | string | no | Date time string, ISO 8601 formatted, such `2016-03-11T03:45:40Z` (requires administrator or project/group owner rights) | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>"\ - "https://gitlab.example.com/api/v4/projects/5/repository/commits/11/discussions/<discussion_id>/notes?body=comment + "https://gitlab.example.com/api/v4/projects/5/repository/commits/<commit_id>/discussions/<discussion_id>/notes?body=comment ``` ### Modify an existing commit thread note @@ -1338,21 +1338,21 @@ Parameters: | Attribute | Type | Required | Description | | ------------------- | -------------- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | -| `commit_id` | integer | yes | The ID of a commit | -| `discussion_id` | integer | yes | The ID of a thread | +| `commit_id` | string | yes | The SHA of a commit | +| `discussion_id` | string | yes | The ID of a thread | | `note_id` | integer | yes | The ID of a thread note | | `body` | string | no | The content of a note | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>"\ - "https://gitlab.example.com/api/v4/projects/5/repository/commits/11/discussions/<discussion_id>/notes/1108?body=comment" + "https://gitlab.example.com/api/v4/projects/5/repository/commits/<commit_id>/discussions/<discussion_id>/notes/1108?body=comment" ``` Resolving a note: ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>"\ - "https://gitlab.example.com/api/v4/projects/5/repository/commits/11/discussions/<discussion_id>/notes/1108?resolved=true" + "https://gitlab.example.com/api/v4/projects/5/repository/commits/<commit_id>/discussions/<discussion_id>/notes/1108?resolved=true" ``` ### Delete a commit thread note @@ -1368,11 +1368,11 @@ Parameters: | Attribute | Type | Required | Description | | ------------------- | -------------- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | -| `commit_id` | integer | yes | The ID of a commit | -| `discussion_id` | integer | yes | The ID of a thread | +| `commit_id` | string | yes | The SHA of a commit | +| `discussion_id` | string | yes | The ID of a thread | | `note_id` | integer | yes | The ID of a thread note | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>"\ - "https://gitlab.example.com/api/v4/projects/5/repository/commits/11/discussions/636" + "https://gitlab.example.com/api/v4/projects/5/repository/commits/<commit_id>/discussions/<discussion_id>/notes/636" ``` diff --git a/doc/api/dora/metrics.md b/doc/api/dora/metrics.md index 99473ca7b4c..543b6583fde 100644 --- a/doc/api/dora/metrics.md +++ b/doc/api/dora/metrics.md @@ -105,3 +105,6 @@ parameter: | `deployment_frequency` | The number of successful deployments during the time period. | | `lead_time_for_changes` | The median number of seconds between the merge of the merge request (MR) and the deployment of the MR's commits for all MRs deployed during the time period. | | `time_to_restore_service` | The median number of seconds an incident was open during the time period. Available only for production environment. | + +NOTE: +The API returns the `monthly` and `all` intervals by calculating the median of the daily median values. This can introduce a slight inaccuracy in the returned data. diff --git a/doc/api/environments.md b/doc/api/environments.md index d67808bfd61..3f5f711e10b 100644 --- a/doc/api/environments.md +++ b/doc/api/environments.md @@ -68,6 +68,9 @@ Example response: "started_at": "2019-03-25T12:54:50.082Z", "finished_at": "2019-03-25T18:55:13.216Z", "duration": 21623.13423, + "project": { + "ci_job_token_scope_enabled": false + }, "user": { "id": 1, "name": "Administrator", @@ -180,6 +183,9 @@ Example of response "started_at": "2019-03-25T12:54:50.082Z", "finished_at": "2019-03-25T18:55:13.216Z", "duration": 21623.13423, + "project": { + "ci_job_token_scope_enabled": false + }, "user": { "id": 1, "name": "Administrator", diff --git a/doc/api/events.md b/doc/api/events.md index 3f032c72870..e4490459030 100644 --- a/doc/api/events.md +++ b/doc/api/events.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/graphql/audit_report.md b/doc/api/graphql/audit_report.md index 988db29912f..dc7eb10b810 100644 --- a/doc/api/graphql/audit_report.md +++ b/doc/api/graphql/audit_report.md @@ -1,6 +1,6 @@ --- -stage: Ecosystem -group: Integrations +stage: Govern +group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/graphql/getting_started.md b/doc/api/graphql/getting_started.md index 258f781528b..386ef6c403b 100644 --- a/doc/api/graphql/getting_started.md +++ b/doc/api/graphql/getting_started.md @@ -26,6 +26,7 @@ local computer. A GraphQL request can be made as a `POST` request to `/api/graph with the query as the payload. You can authorize your request by generating a [personal access token](../../user/profile/personal_access_tokens.md) to use as a bearer token. +This token requires at least the `read_api` scope. Example: @@ -36,6 +37,16 @@ curl "https://gitlab.com/api/graphql" --header "Authorization: Bearer $GRAPHQL_T --data "{\"query\": \"query {currentUser {name}}\"}" ``` +To nest strings in the query string, +wrap the data in single quotes or escape the strings with `\\`: + +```shell +curl "https://gitlab.com/api/graphql" --header "Authorization: Bearer $GRAPHQL_TOKEN" \ + --header "Content-Type: application/json" --request POST \ + --data '{"query": "query {project(fullPath: \"<group>/<subgroup>/<project>\") {jobs {nodes {id duration}}}}"}' + # or "{\"query\": \"query {project(fullPath: \\\"<group>/<subgroup>/<project>\\\") {jobs {nodes {id duration}}}}\"}" +``` + ### GraphiQL GraphiQL (pronounced "graphical") allows you to run queries directly against diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index ab5b5a92203..f9125e5f4d4 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -280,7 +280,7 @@ Returns [`Namespace`](#namespace). ### `Query.package` -Find a package. +Find a package. This field can only be resolved for one query in any single request. Returns [`PackageDetailsType`](#packagedetailstype). @@ -317,11 +317,11 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | | <a id="queryprojectsids"></a>`ids` | [`[ID!]`](#id) | Filter projects by IDs. | -| <a id="queryprojectsmembership"></a>`membership` | [`Boolean`](#boolean) | Limit projects that the current user is a member of. | -| <a id="queryprojectssearch"></a>`search` | [`String`](#string) | Search query for project name, path, or description. | +| <a id="queryprojectsmembership"></a>`membership` | [`Boolean`](#boolean) | Return only projects that the current user is a member of. | +| <a id="queryprojectssearch"></a>`search` | [`String`](#string) | Search query, which can be for the project name, a path, or a description. | | <a id="queryprojectssearchnamespaces"></a>`searchNamespaces` | [`Boolean`](#boolean) | Include namespace in project search. | -| <a id="queryprojectssort"></a>`sort` | [`String`](#string) | Sort order of results. | -| <a id="queryprojectstopics"></a>`topics` | [`[String!]`](#string) | Filters projects by topics. | +| <a id="queryprojectssort"></a>`sort` | [`String`](#string) | Sort order of results. Format: '<field_name>_<sort_direction>', for example: 'id_desc' or 'name_asc'. | +| <a id="queryprojectstopics"></a>`topics` | [`[String!]`](#string) | Filter projects by topics. | ### `Query.queryComplexity` @@ -742,6 +742,25 @@ Input type: `ApiFuzzingCiConfigurationCreateInput` | <a id="mutationapifuzzingciconfigurationcreateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationapifuzzingciconfigurationcreategitlabciyamleditpath"></a>`gitlabCiYamlEditPath` **{warning-solid}** | [`String`](#string) | **Deprecated:** The configuration snippet is now generated client-side. Deprecated in 14.6. | +### `Mutation.artifactDestroy` + +Input type: `ArtifactDestroyInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationartifactdestroyclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationartifactdestroyid"></a>`id` | [`CiJobArtifactID!`](#cijobartifactid) | ID of the artifact to delete. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationartifactdestroyartifact"></a>`artifact` | [`CiJobArtifact`](#cijobartifact) | Deleted artifact. | +| <a id="mutationartifactdestroyclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationartifactdestroyerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | + ### `Mutation.auditEventsStreamingHeadersCreate` Input type: `AuditEventsStreamingHeadersCreateInput` @@ -1409,7 +1428,9 @@ Input type: `CreateComplianceFrameworkInput` ### `Mutation.createCustomEmoji` -Available only when feature flag `custom_emoji` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. +WARNING: +**Introduced** in 13.6. +This feature is in Alpha. It can be changed or removed at any time. Input type: `CreateCustomEmojiInput` @@ -2271,7 +2292,9 @@ Input type: `DestroyContainerRepositoryTagsInput` ### `Mutation.destroyCustomEmoji` -Available only when feature flag `custom_emoji` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. +WARNING: +**Introduced** in 13.6. +This feature is in Alpha. It can be changed or removed at any time. Input type: `DestroyCustomEmojiInput` @@ -2786,6 +2809,7 @@ Input type: `ExternalAuditEventDestinationCreateInput` | <a id="mutationexternalauditeventdestinationcreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationexternalauditeventdestinationcreatedestinationurl"></a>`destinationUrl` | [`String!`](#string) | Destination URL. | | <a id="mutationexternalauditeventdestinationcreategrouppath"></a>`groupPath` | [`ID!`](#id) | Group path. | +| <a id="mutationexternalauditeventdestinationcreateverificationtoken"></a>`verificationToken` | [`String`](#string) | Verification token. | #### Fields @@ -3033,6 +3057,7 @@ Input type: `IssueMoveListInput` | <a id="mutationissuemovelistiid"></a>`iid` | [`String!`](#string) | IID of the issue to mutate. | | <a id="mutationissuemovelistmoveafterid"></a>`moveAfterId` | [`ID`](#id) | ID of issue that should be placed after the current issue. | | <a id="mutationissuemovelistmovebeforeid"></a>`moveBeforeId` | [`ID`](#id) | ID of issue that should be placed before the current issue. | +| <a id="mutationissuemovelistpositioninlist"></a>`positionInList` | [`Int`](#int) | Position of issue within the board list. Positions start at 0. Use -1 to move to the end of the list. | | <a id="mutationissuemovelistprojectpath"></a>`projectPath` | [`ID!`](#id) | Project the issue to mutate is in. | | <a id="mutationissuemovelisttolistid"></a>`toListId` | [`ID`](#id) | ID of the board list that the issue will be moved to. | @@ -3457,6 +3482,26 @@ Input type: `JiraImportUsersInput` | <a id="mutationjiraimportuserserrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationjiraimportusersjirausers"></a>`jiraUsers` | [`[JiraUser!]`](#jirauser) | Users returned from Jira, matched by email and name if possible. | +### `Mutation.jobArtifactsDestroy` + +Input type: `JobArtifactsDestroyInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationjobartifactsdestroyclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationjobartifactsdestroyid"></a>`id` | [`CiBuildID!`](#cibuildid) | ID of the job to mutate. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationjobartifactsdestroyclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationjobartifactsdestroydestroyedartifactscount"></a>`destroyedArtifactsCount` | [`Int!`](#int) | Number of artifacts deleted. | +| <a id="mutationjobartifactsdestroyerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationjobartifactsdestroyjob"></a>`job` | [`CiJob`](#cijob) | Job with artifacts to be deleted. | + ### `Mutation.jobCancel` Input type: `JobCancelInput` @@ -4326,7 +4371,7 @@ Input type: `ReleaseCreateInput` | <a id="mutationreleasecreatename"></a>`name` | [`String`](#string) | Name of the release. | | <a id="mutationreleasecreateprojectpath"></a>`projectPath` | [`ID!`](#id) | Full path of the project the release is associated with. | | <a id="mutationreleasecreateref"></a>`ref` | [`String`](#string) | Commit SHA or branch name to use if creating a new tag. | -| <a id="mutationreleasecreatereleasedat"></a>`releasedAt` | [`Time`](#time) | Date and time for the release. Defaults to the current date and time. | +| <a id="mutationreleasecreatereleasedat"></a>`releasedAt` | [`Time`](#time) | Date and time for the release. Defaults to the current time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). Only provide this field if creating an upcoming or historical release. | | <a id="mutationreleasecreatetagmessage"></a>`tagMessage` | [`String`](#string) | Message to use if creating a new annotated tag. | | <a id="mutationreleasecreatetagname"></a>`tagName` | [`String!`](#string) | Name of the tag to associate with the release. | @@ -4450,6 +4495,7 @@ Input type: `RunnerUpdateInput` | ---- | ---- | ----------- | | <a id="mutationrunnerupdateaccesslevel"></a>`accessLevel` | [`CiRunnerAccessLevel`](#cirunneraccesslevel) | Access level of the runner. | | <a id="mutationrunnerupdateactive"></a>`active` **{warning-solid}** | [`Boolean`](#boolean) | **Deprecated:** This was renamed. Please use `paused`. Deprecated in 14.8. | +| <a id="mutationrunnerupdateassociatedprojects"></a>`associatedProjects` | [`[ProjectID!]`](#projectid) | Projects associated with the runner. Available only for project runners. | | <a id="mutationrunnerupdateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationrunnerupdatedescription"></a>`description` | [`String`](#string) | Description of the runner. | | <a id="mutationrunnerupdateid"></a>`id` | [`CiRunnerID!`](#cirunnerid) | ID of the runner to update. | @@ -4575,6 +4621,47 @@ Input type: `ScanExecutionPolicyCommitInput` | <a id="mutationscanexecutionpolicycommitclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationscanexecutionpolicycommiterrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +### `Mutation.securityFindingCreateIssue` + +Input type: `SecurityFindingCreateIssueInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsecurityfindingcreateissueclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsecurityfindingcreateissueproject"></a>`project` | [`ProjectID!`](#projectid) | ID of the project to attach the issue to. | +| <a id="mutationsecurityfindingcreateissueuuid"></a>`uuid` | [`String!`](#string) | UUID of the security finding to be used to create an issue. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsecurityfindingcreateissueclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsecurityfindingcreateissueerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationsecurityfindingcreateissueissue"></a>`issue` | [`Issue`](#issue) | Issue created after mutation. | + +### `Mutation.securityFindingDismiss` + +Input type: `SecurityFindingDismissInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsecurityfindingdismissclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsecurityfindingdismisscomment"></a>`comment` | [`String`](#string) | Comment why finding should be dismissed. | +| <a id="mutationsecurityfindingdismissdismissalreason"></a>`dismissalReason` | [`VulnerabilityDismissalReason`](#vulnerabilitydismissalreason) | Reason why finding should be dismissed. | +| <a id="mutationsecurityfindingdismissuuid"></a>`uuid` | [`String!`](#string) | UUID of the finding to be dismissed. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsecurityfindingdismissclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsecurityfindingdismisserrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationsecurityfindingdismissuuid"></a>`uuid` | [`String`](#string) | UUID of dismissed finding. | + ### `Mutation.securityPolicyProjectAssign` Assigns the specified project(`security_policy_project_id`) as security policy project for the given project(`full_path`). If the project already has a security policy project, this reassigns the project's security policy project with the given `security_policy_project_id`. @@ -5094,6 +5181,8 @@ Input type: `UpdateDependencyProxyImageTtlGroupPolicyInput` ### `Mutation.updateDependencyProxySettings` +These settings can be adjusted by the group Owner or Maintainer. However, in GitLab 16.0, we will be limiting this to the Owner role. [GitLab-#364441](https://gitlab.com/gitlab-org/gitlab/-/issues/364441) proposes making this change to match the permissions level in the user interface. + Input type: `UpdateDependencyProxySettingsInput` #### Arguments @@ -5456,7 +5545,7 @@ Input type: `VulnerabilityCreateInput` | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mutationvulnerabilitycreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | -| <a id="mutationvulnerabilitycreateconfidence"></a>`confidence` | [`VulnerabilityConfidence`](#vulnerabilityconfidence) | Confidence of the vulnerability (defaults to `unknown`). | +| <a id="mutationvulnerabilitycreateconfidence"></a>`confidence` **{warning-solid}** | [`VulnerabilityConfidence`](#vulnerabilityconfidence) | **Deprecated:** This field will be removed from the Vulnerability domain model. Deprecated in 15.4. | | <a id="mutationvulnerabilitycreateconfirmedat"></a>`confirmedAt` | [`Time`](#time) | Timestamp of when the vulnerability state changed to confirmed (defaults to creation time if status is `confirmed`). | | <a id="mutationvulnerabilitycreatedescription"></a>`description` | [`String!`](#string) | Long text section that describes the vulnerability in more detail. | | <a id="mutationvulnerabilitycreatedetectedat"></a>`detectedAt` | [`Time`](#time) | Timestamp of when the vulnerability was first detected (defaults to creation time). | @@ -5728,6 +5817,7 @@ Input type: `WorkItemUpdateInput` | <a id="mutationworkitemupdatedescriptionwidget"></a>`descriptionWidget` | [`WorkItemWidgetDescriptionInput`](#workitemwidgetdescriptioninput) | Input for description widget. | | <a id="mutationworkitemupdatehierarchywidget"></a>`hierarchyWidget` | [`WorkItemWidgetHierarchyUpdateInput`](#workitemwidgethierarchyupdateinput) | Input for hierarchy widget. | | <a id="mutationworkitemupdateid"></a>`id` | [`WorkItemID!`](#workitemid) | Global ID of the work item. | +| <a id="mutationworkitemupdateiterationwidget"></a>`iterationWidget` | [`WorkItemWidgetIterationInput`](#workitemwidgetiterationinput) | Input for iteration widget. | | <a id="mutationworkitemupdatestartandduedatewidget"></a>`startAndDueDateWidget` | [`WorkItemWidgetStartAndDueDateUpdateInput`](#workitemwidgetstartandduedateupdateinput) | Input for start and due date widget. | | <a id="mutationworkitemupdatestateevent"></a>`stateEvent` | [`WorkItemStateEvent`](#workitemstateevent) | Close or reopen a work item. | | <a id="mutationworkitemupdatetitle"></a>`title` | [`String`](#string) | Title of the work item. | @@ -6025,6 +6115,7 @@ The connection type for [`BoardEpic`](#boardepic). | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="boardepicconnectioncount"></a>`count` | [`Int!`](#int) | Total count of collection. | | <a id="boardepicconnectionedges"></a>`edges` | [`[BoardEpicEdge]`](#boardepicedge) | A list of edges. | | <a id="boardepicconnectionnodes"></a>`nodes` | [`[BoardEpic]`](#boardepic) | A list of nodes. | | <a id="boardepicconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | @@ -6063,6 +6154,29 @@ The edge type for [`BoardList`](#boardlist). | <a id="boardlistedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="boardlistedgenode"></a>`node` | [`BoardList`](#boardlist) | The item at the end of the edge. | +#### `BranchRuleConnection` + +The connection type for [`BranchRule`](#branchrule). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="branchruleconnectionedges"></a>`edges` | [`[BranchRuleEdge]`](#branchruleedge) | A list of edges. | +| <a id="branchruleconnectionnodes"></a>`nodes` | [`[BranchRule]`](#branchrule) | A list of nodes. | +| <a id="branchruleconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `BranchRuleEdge` + +The edge type for [`BranchRule`](#branchrule). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="branchruleedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="branchruleedgenode"></a>`node` | [`BranchRule`](#branchrule) | The item at the end of the edge. | + #### `CiBuildNeedConnection` The connection type for [`CiBuildNeed`](#cibuildneed). @@ -6210,6 +6324,7 @@ The connection type for [`CiGroupVariable`](#cigroupvariable). | Name | Type | Description | | ---- | ---- | ----------- | | <a id="cigroupvariableconnectionedges"></a>`edges` | [`[CiGroupVariableEdge]`](#cigroupvariableedge) | A list of edges. | +| <a id="cigroupvariableconnectionlimit"></a>`limit` | [`Int!`](#int) | Maximum amount of group CI/CD variables. | | <a id="cigroupvariableconnectionnodes"></a>`nodes` | [`[CiGroupVariable]`](#cigroupvariable) | A list of nodes. | | <a id="cigroupvariableconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | @@ -6385,6 +6500,7 @@ The connection type for [`CiProjectVariable`](#ciprojectvariable). | Name | Type | Description | | ---- | ---- | ----------- | | <a id="ciprojectvariableconnectionedges"></a>`edges` | [`[CiProjectVariableEdge]`](#ciprojectvariableedge) | A list of edges. | +| <a id="ciprojectvariableconnectionlimit"></a>`limit` | [`Int!`](#int) | Maximum amount of project CI/CD variables. | | <a id="ciprojectvariableconnectionnodes"></a>`nodes` | [`[CiProjectVariable]`](#ciprojectvariable) | A list of nodes. | | <a id="ciprojectvariableconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | @@ -6959,6 +7075,29 @@ The edge type for [`DependencyProxyManifest`](#dependencyproxymanifest). | <a id="dependencyproxymanifestedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="dependencyproxymanifestedgenode"></a>`node` | [`DependencyProxyManifest`](#dependencyproxymanifest) | The item at the end of the edge. | +#### `DeploymentConnection` + +The connection type for [`Deployment`](#deployment). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="deploymentconnectionedges"></a>`edges` | [`[DeploymentEdge]`](#deploymentedge) | A list of edges. | +| <a id="deploymentconnectionnodes"></a>`nodes` | [`[Deployment]`](#deployment) | A list of nodes. | +| <a id="deploymentconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `DeploymentEdge` + +The edge type for [`Deployment`](#deployment). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="deploymentedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="deploymentedgenode"></a>`node` | [`Deployment`](#deployment) | The item at the end of the edge. | + #### `DesignAtVersionConnection` The connection type for [`DesignAtVersion`](#designatversion). @@ -7151,6 +7290,7 @@ The connection type for [`Epic`](#epic). | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="epicconnectioncount"></a>`count` | [`Int!`](#int) | Total count of collection. | | <a id="epicconnectionedges"></a>`edges` | [`[EpicEdge]`](#epicedge) | A list of edges. | | <a id="epicconnectionnodes"></a>`nodes` | [`[Epic]`](#epic) | A list of nodes. | | <a id="epicconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | @@ -7700,6 +7840,29 @@ The edge type for [`MemberInterface`](#memberinterface). | <a id="memberinterfaceedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="memberinterfaceedgenode"></a>`node` | [`MemberInterface`](#memberinterface) | The item at the end of the edge. | +#### `MergeAccessLevelConnection` + +The connection type for [`MergeAccessLevel`](#mergeaccesslevel). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergeaccesslevelconnectionedges"></a>`edges` | [`[MergeAccessLevelEdge]`](#mergeaccessleveledge) | A list of edges. | +| <a id="mergeaccesslevelconnectionnodes"></a>`nodes` | [`[MergeAccessLevel]`](#mergeaccesslevel) | A list of nodes. | +| <a id="mergeaccesslevelconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `MergeAccessLevelEdge` + +The edge type for [`MergeAccessLevel`](#mergeaccesslevel). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergeaccessleveledgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="mergeaccessleveledgenode"></a>`node` | [`MergeAccessLevel`](#mergeaccesslevel) | The item at the end of the edge. | + #### `MergeRequestAssigneeConnection` The connection type for [`MergeRequestAssignee`](#mergerequestassignee). @@ -8258,6 +8421,29 @@ The edge type for [`ProjectMember`](#projectmember). | <a id="projectmemberedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="projectmemberedgenode"></a>`node` | [`ProjectMember`](#projectmember) | The item at the end of the edge. | +#### `PushAccessLevelConnection` + +The connection type for [`PushAccessLevel`](#pushaccesslevel). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="pushaccesslevelconnectionedges"></a>`edges` | [`[PushAccessLevelEdge]`](#pushaccessleveledge) | A list of edges. | +| <a id="pushaccesslevelconnectionnodes"></a>`nodes` | [`[PushAccessLevel]`](#pushaccesslevel) | A list of nodes. | +| <a id="pushaccesslevelconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `PushAccessLevelEdge` + +The edge type for [`PushAccessLevel`](#pushaccesslevel). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="pushaccessleveledgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="pushaccessleveledgenode"></a>`node` | [`PushAccessLevel`](#pushaccesslevel) | The item at the end of the edge. | + #### `ReleaseAssetLinkConnection` The connection type for [`ReleaseAssetLink`](#releaseassetlink). @@ -9726,6 +9912,7 @@ Represents an epic on an issue board. | <a id="boardepiccolor"></a>`color` | [`String`](#string) | Color of the epic. Returns `null` if `epic_color_highlight` feature flag is disabled. | | <a id="boardepicconfidential"></a>`confidential` | [`Boolean`](#boolean) | Indicates if the epic is confidential. | | <a id="boardepiccreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of when the epic was created. | +| <a id="boardepicdefaultprojectforissuecreation"></a>`defaultProjectForIssueCreation` | [`Project`](#project) | Default Project for issue creation. Based on the project the user created the last issue in. | | <a id="boardepicdescendantcounts"></a>`descendantCounts` | [`EpicDescendantCount`](#epicdescendantcount) | Number of open and closed descendant epics and issues. | | <a id="boardepicdescendantweightsum"></a>`descendantWeightSum` | [`EpicDescendantWeights`](#epicdescendantweights) | Total weight of open and closed issues in the epic and its descendants. | | <a id="boardepicdescription"></a>`description` | [`String`](#string) | Description of the epic. | @@ -9795,7 +9982,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="boardepicancestorsiid"></a>`iid` | [`ID`](#id) | IID of the epic, e.g., "1". | | <a id="boardepicancestorsiidstartswith"></a>`iidStartsWith` | [`String`](#string) | Filter epics by IID for autocomplete. | | <a id="boardepicancestorsiids"></a>`iids` | [`[ID!]`](#id) | List of IIDs of epics, e.g., `[1, 2]`. | -| <a id="boardepicancestorsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument. | +| <a id="boardepicancestorsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="boardepicancestorsincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include epics from ancestor groups. | | <a id="boardepicancestorsincludedescendantgroups"></a>`includeDescendantGroups` | [`Boolean`](#boolean) | Include epics from descendant groups. | | <a id="boardepicancestorslabelname"></a>`labelName` | [`[String!]`](#string) | Filter epics by labels. | @@ -9833,7 +10020,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="boardepicchildreniid"></a>`iid` | [`ID`](#id) | IID of the epic, e.g., "1". | | <a id="boardepicchildreniidstartswith"></a>`iidStartsWith` | [`String`](#string) | Filter epics by IID for autocomplete. | | <a id="boardepicchildreniids"></a>`iids` | [`[ID!]`](#id) | List of IIDs of epics, e.g., `[1, 2]`. | -| <a id="boardepicchildrenin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument. | +| <a id="boardepicchildrenin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="boardepicchildrenincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include epics from ancestor groups. | | <a id="boardepicchildrenincludedescendantgroups"></a>`includeDescendantGroups` | [`Boolean`](#boolean) | Include epics from descendant groups. | | <a id="boardepicchildrenlabelname"></a>`labelName` | [`[String!]`](#string) | Filter epics by labels. | @@ -9937,6 +10124,32 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="branchcommit"></a>`commit` | [`Commit`](#commit) | Commit for the branch. | | <a id="branchname"></a>`name` | [`String!`](#string) | Name of the branch. | +### `BranchProtection` + +Branch protection details for a branch rule. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="branchprotectionallowforcepush"></a>`allowForcePush` | [`Boolean!`](#boolean) | Toggle force push to the branch for users with write access. | +| <a id="branchprotectioncodeownerapprovalrequired"></a>`codeOwnerApprovalRequired` | [`Boolean!`](#boolean) | Enforce code owner approvals before allowing a merge. | +| <a id="branchprotectionmergeaccesslevels"></a>`mergeAccessLevels` | [`MergeAccessLevelConnection`](#mergeaccesslevelconnection) | Details about who can merge when this branch is the source branch. (see [Connections](#connections)) | +| <a id="branchprotectionpushaccesslevels"></a>`pushAccessLevels` | [`PushAccessLevelConnection`](#pushaccesslevelconnection) | Details about who can push when this branch is the source branch. (see [Connections](#connections)) | + +### `BranchRule` + +List of branch rules for a project, grouped by branch name. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="branchrulebranchprotection"></a>`branchProtection` | [`BranchProtection!`](#branchprotection) | Branch protections configured for this branch rule. | +| <a id="branchrulecreatedat"></a>`createdAt` | [`Time!`](#time) | Timestamp of when the branch rule was created. | +| <a id="branchrulename"></a>`name` | [`String!`](#string) | Branch name, with wildcards, for the branch rules. | +| <a id="branchruleupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp of when the branch rule was last updated. | + ### `BurnupChartDailyTotals` Represents the total number of issues and their weights for a particular day. @@ -10050,6 +10263,18 @@ Represents the total number of issues and their weights for a particular day. | <a id="ciconfigstagegroups"></a>`groups` | [`CiConfigGroupConnection`](#ciconfiggroupconnection) | Groups of jobs for the stage. (see [Connections](#connections)) | | <a id="ciconfigstagename"></a>`name` | [`String`](#string) | Name of the stage. | +### `CiConfigVariable` + +CI/CD config variables. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="ciconfigvariabledescription"></a>`description` | [`String`](#string) | Description for the CI/CD config variable. | +| <a id="ciconfigvariablekey"></a>`key` | [`String`](#string) | Name of the variable. | +| <a id="ciconfigvariablevalue"></a>`value` | [`String`](#string) | Value of the variable. | + ### `CiGroup` #### Fields @@ -10139,6 +10364,7 @@ CI/CD variables for a GitLab instance. | <a id="cijobtags"></a>`tags` | [`[String!]`](#string) | Tags for the current job. | | <a id="cijobtriggered"></a>`triggered` | [`Boolean`](#boolean) | Whether the job was triggered. | | <a id="cijobuserpermissions"></a>`userPermissions` | [`JobPermissions!`](#jobpermissions) | Permissions for the current user on the resource. | +| <a id="cijobwebpath"></a>`webPath` | [`String`](#string) | Web path of the job. | ### `CiJobArtifact` @@ -10147,8 +10373,11 @@ CI/CD variables for a GitLab instance. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="cijobartifactdownloadpath"></a>`downloadPath` | [`String`](#string) | URL for downloading the artifact's file. | +| <a id="cijobartifactexpireat"></a>`expireAt` | [`Time`](#time) | Expiry date of the artifact. | | <a id="cijobartifactfiletype"></a>`fileType` | [`JobArtifactFileType`](#jobartifactfiletype) | File type of the artifact. | +| <a id="cijobartifactid"></a>`id` | [`CiJobArtifactID!`](#cijobartifactid) | ID of the artifact. | | <a id="cijobartifactname"></a>`name` | [`String`](#string) | File name of the artifact. | +| <a id="cijobartifactsize"></a>`size` | [`Int!`](#int) | Size of the artifact in bytes. | ### `CiJobTokenScopeType` @@ -10240,7 +10469,6 @@ CI/CD variables for a project. | <a id="cirunnerplatformname"></a>`platformName` | [`String`](#string) | Platform provided by the runner. | | <a id="cirunnerprivateprojectsminutescostfactor"></a>`privateProjectsMinutesCostFactor` | [`Float`](#float) | Private projects' "minutes cost factor" associated with the runner (GitLab.com only). | | <a id="cirunnerprojectcount"></a>`projectCount` | [`Int`](#int) | Number of projects that the runner is associated with. | -| <a id="cirunnerprojects"></a>`projects` | [`ProjectConnection`](#projectconnection) | Projects the runner is associated with. For project runners only. (see [Connections](#connections)) | | <a id="cirunnerpublicprojectsminutescostfactor"></a>`publicProjectsMinutesCostFactor` | [`Float`](#float) | Public projects' "minutes cost factor" associated with the runner (GitLab.com only). | | <a id="cirunnerrevision"></a>`revision` | [`String`](#string) | Revision of the runner. | | <a id="cirunnerrununtagged"></a>`runUntagged` | [`Boolean!`](#boolean) | Indicates the runner is able to run untagged jobs. | @@ -10256,7 +10484,7 @@ CI/CD variables for a project. ##### `CiRunner.jobs` -Jobs assigned to the runner. +Jobs assigned to the runner. This field can only be resolved for one runner in any single request. Returns [`CiJobConnection`](#cijobconnection). @@ -10270,6 +10498,26 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="cirunnerjobsstatuses"></a>`statuses` | [`[CiJobStatus!]`](#cijobstatus) | Filter jobs by status. | +##### `CiRunner.projects` + +Find projects the runner is associated with. For project runners only. + +Returns [`ProjectConnection`](#projectconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="cirunnerprojectsmembership"></a>`membership` | [`Boolean`](#boolean) | Return only projects that the current user is a member of. | +| <a id="cirunnerprojectssearch"></a>`search` | [`String`](#string) | Search query, which can be for the project name, a path, or a description. | +| <a id="cirunnerprojectssearchnamespaces"></a>`searchNamespaces` | [`Boolean`](#boolean) | Include namespace in project search. | +| <a id="cirunnerprojectssort"></a>`sort` **{warning-solid}** | [`String`](#string) | **Deprecated** in 15.4. Default sort order will change in 16.0. Specify `"id_asc"` if query results' order is important. | +| <a id="cirunnerprojectstopics"></a>`topics` | [`[String!]`](#string) | Filter projects by topics. | + ##### `CiRunner.status` Status of the runner. @@ -10339,6 +10587,7 @@ GitLab CI/CD configuration template. | <a id="clusteragentname"></a>`name` | [`String`](#string) | Name of the cluster agent. | | <a id="clusteragentproject"></a>`project` | [`Project`](#project) | Project this cluster agent is associated with. | | <a id="clusteragentupdatedat"></a>`updatedAt` | [`Time`](#time) | Timestamp the cluster agent was updated. | +| <a id="clusteragentvulnerabilityimages"></a>`vulnerabilityImages` | [`VulnerabilityContainerImageConnection`](#vulnerabilitycontainerimageconnection) | Container images reported on the agent vulnerabilities. (see [Connections](#connections)) | | <a id="clusteragentwebpath"></a>`webPath` | [`String`](#string) | Web path of the cluster agent. | #### Fields with arguments @@ -10950,6 +11199,60 @@ Group-level Dependency Proxy settings. | ---- | ---- | ----------- | | <a id="dependencyproxysettingenabled"></a>`enabled` | [`Boolean!`](#boolean) | Indicates whether the dependency proxy is enabled for the group. | +### `Deployment` + +The deployment of an environment. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="deploymentcommit"></a>`commit` | [`Commit`](#commit) | Commit details of the deployment. | +| <a id="deploymentcreatedat"></a>`createdAt` | [`Time`](#time) | When the deployment record was created. | +| <a id="deploymentfinishedat"></a>`finishedAt` | [`Time`](#time) | When the deployment finished. | +| <a id="deploymentid"></a>`id` | [`ID`](#id) | Global ID of the deployment. | +| <a id="deploymentiid"></a>`iid` | [`ID`](#id) | Project-level internal ID of the deployment. | +| <a id="deploymentjob"></a>`job` | [`CiJob`](#cijob) | Pipeline job of the deployment. | +| <a id="deploymentref"></a>`ref` | [`String`](#string) | Git-Ref that the deployment ran on. | +| <a id="deploymentsha"></a>`sha` | [`String`](#string) | Git-SHA that the deployment ran on. | +| <a id="deploymentstatus"></a>`status` | [`DeploymentStatus`](#deploymentstatus) | Status of the deployment. | +| <a id="deploymenttag"></a>`tag` | [`Boolean`](#boolean) | True or false if the deployment ran on a Git-tag. | +| <a id="deploymenttriggerer"></a>`triggerer` | [`UserCore`](#usercore) | User who executed the deployment. | +| <a id="deploymentupdatedat"></a>`updatedAt` | [`Time`](#time) | When the deployment record was updated. | + +### `DeploymentDetails` + +The details of the deployment. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="deploymentdetailscommit"></a>`commit` | [`Commit`](#commit) | Commit details of the deployment. | +| <a id="deploymentdetailscreatedat"></a>`createdAt` | [`Time`](#time) | When the deployment record was created. | +| <a id="deploymentdetailsfinishedat"></a>`finishedAt` | [`Time`](#time) | When the deployment finished. | +| <a id="deploymentdetailsid"></a>`id` | [`ID`](#id) | Global ID of the deployment. | +| <a id="deploymentdetailsiid"></a>`iid` | [`ID`](#id) | Project-level internal ID of the deployment. | +| <a id="deploymentdetailsjob"></a>`job` | [`CiJob`](#cijob) | Pipeline job of the deployment. | +| <a id="deploymentdetailsref"></a>`ref` | [`String`](#string) | Git-Ref that the deployment ran on. | +| <a id="deploymentdetailssha"></a>`sha` | [`String`](#string) | Git-SHA that the deployment ran on. | +| <a id="deploymentdetailsstatus"></a>`status` | [`DeploymentStatus`](#deploymentstatus) | Status of the deployment. | +| <a id="deploymentdetailstag"></a>`tag` | [`Boolean`](#boolean) | True or false if the deployment ran on a Git-tag. | +| <a id="deploymentdetailstags"></a>`tags` | [`[DeploymentTag!]`](#deploymenttag) | Git tags that contain this deployment. | +| <a id="deploymentdetailstriggerer"></a>`triggerer` | [`UserCore`](#usercore) | User who executed the deployment. | +| <a id="deploymentdetailsupdatedat"></a>`updatedAt` | [`Time`](#time) | When the deployment record was updated. | + +### `DeploymentTag` + +Tags for a given deployment. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="deploymenttagname"></a>`name` | [`String`](#string) | Name of this git tag. | +| <a id="deploymenttagpath"></a>`path` | [`String`](#string) | Path for this tag. | + ### `Design` A single design. @@ -11373,14 +11676,51 @@ Describes where code is deployed for a project. | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="environmentautodeleteat"></a>`autoDeleteAt` | [`Time`](#time) | When the environment is going to be deleted automatically. | +| <a id="environmentautostopat"></a>`autoStopAt` | [`Time`](#time) | When the environment is going to be stopped automatically. | +| <a id="environmentcreatedat"></a>`createdAt` | [`Time`](#time) | When the environment was created. | +| <a id="environmentenvironmenttype"></a>`environmentType` | [`String`](#string) | Folder name of the environment. | +| <a id="environmentexternalurl"></a>`externalUrl` | [`String`](#string) | External URL of the environment. | | <a id="environmentid"></a>`id` | [`ID!`](#id) | ID of the environment. | | <a id="environmentlatestopenedmostseverealert"></a>`latestOpenedMostSevereAlert` | [`AlertManagementAlert`](#alertmanagementalert) | Most severe open alert for the environment. If multiple alerts have equal severity, the most recent is returned. | | <a id="environmentname"></a>`name` | [`String!`](#string) | Human-readable name of the environment. | | <a id="environmentpath"></a>`path` | [`String!`](#string) | Path to the environment. | +| <a id="environmentslug"></a>`slug` | [`String`](#string) | Slug of the environment. | | <a id="environmentstate"></a>`state` | [`String!`](#string) | State of the environment, for example: available/stopped. | +| <a id="environmenttier"></a>`tier` | [`DeploymentTier`](#deploymenttier) | Deployment tier of the environment. | +| <a id="environmentupdatedat"></a>`updatedAt` | [`Time`](#time) | When the environment was updated. | #### Fields with arguments +##### `Environment.deployments` + +Deployments of the environment. This field can only be resolved for one project in any single request. + +Returns [`DeploymentConnection`](#deploymentconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="environmentdeploymentsorderby"></a>`orderBy` | [`DeploymentsOrderByInput`](#deploymentsorderbyinput) | Order by a specified field. | +| <a id="environmentdeploymentsstatuses"></a>`statuses` | [`[DeploymentStatus!]`](#deploymentstatus) | Statuses of the deployments. | + +##### `Environment.lastDeployment` + +Last deployment of the environment. + +Returns [`Deployment`](#deployment). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="environmentlastdeploymentstatus"></a>`status` | [`DeploymentStatus!`](#deploymentstatus) | Status of the Deployment. | + ##### `Environment.metricsDashboard` Metrics dashboard schema for the environment. @@ -11411,6 +11751,7 @@ Represents an epic. | <a id="epiccolor"></a>`color` | [`String`](#string) | Color of the epic. Returns `null` if `epic_color_highlight` feature flag is disabled. | | <a id="epicconfidential"></a>`confidential` | [`Boolean`](#boolean) | Indicates if the epic is confidential. | | <a id="epiccreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of when the epic was created. | +| <a id="epicdefaultprojectforissuecreation"></a>`defaultProjectForIssueCreation` | [`Project`](#project) | Default Project for issue creation. Based on the project the user created the last issue in. | | <a id="epicdescendantcounts"></a>`descendantCounts` | [`EpicDescendantCount`](#epicdescendantcount) | Number of open and closed descendant epics and issues. | | <a id="epicdescendantweightsum"></a>`descendantWeightSum` | [`EpicDescendantWeights`](#epicdescendantweights) | Total weight of open and closed issues in the epic and its descendants. | | <a id="epicdescription"></a>`description` | [`String`](#string) | Description of the epic. | @@ -11479,7 +11820,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="epicancestorsiid"></a>`iid` | [`ID`](#id) | IID of the epic, e.g., "1". | | <a id="epicancestorsiidstartswith"></a>`iidStartsWith` | [`String`](#string) | Filter epics by IID for autocomplete. | | <a id="epicancestorsiids"></a>`iids` | [`[ID!]`](#id) | List of IIDs of epics, e.g., `[1, 2]`. | -| <a id="epicancestorsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument. | +| <a id="epicancestorsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="epicancestorsincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include epics from ancestor groups. | | <a id="epicancestorsincludedescendantgroups"></a>`includeDescendantGroups` | [`Boolean`](#boolean) | Include epics from descendant groups. | | <a id="epicancestorslabelname"></a>`labelName` | [`[String!]`](#string) | Filter epics by labels. | @@ -11517,7 +11858,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="epicchildreniid"></a>`iid` | [`ID`](#id) | IID of the epic, e.g., "1". | | <a id="epicchildreniidstartswith"></a>`iidStartsWith` | [`String`](#string) | Filter epics by IID for autocomplete. | | <a id="epicchildreniids"></a>`iids` | [`[ID!]`](#id) | List of IIDs of epics, e.g., `[1, 2]`. | -| <a id="epicchildrenin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument. | +| <a id="epicchildrenin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="epicchildrenincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include epics from ancestor groups. | | <a id="epicchildrenincludedescendantgroups"></a>`includeDescendantGroups` | [`Boolean`](#boolean) | Include epics from descendant groups. | | <a id="epicchildrenlabelname"></a>`labelName` | [`[String!]`](#string) | Filter epics by labels. | @@ -11665,6 +12006,7 @@ Relationship between an epic and an issue. | <a id="epicissueepicissueid"></a>`epicIssueId` | [`ID!`](#id) | ID of the epic-issue relation. | | <a id="epicissueescalationpolicy"></a>`escalationPolicy` | [`EscalationPolicyType`](#escalationpolicytype) | Escalation policy associated with the issue. Available for issues which support escalation. | | <a id="epicissueescalationstatus"></a>`escalationStatus` | [`IssueEscalationStatus`](#issueescalationstatus) | Escalation status of the issue. | +| <a id="epicissuehasepic"></a>`hasEpic` | [`Boolean!`](#boolean) | Indicates if the issue belongs to an epic. Can return true and not show an associated epic when the user has no access to the epic. | | <a id="epicissuehealthstatus"></a>`healthStatus` | [`HealthStatus`](#healthstatus) | Current health status. | | <a id="epicissuehidden"></a>`hidden` | [`Boolean`](#boolean) | Indicates the issue is hidden because the author has been banned. Will always return `null` if `ban_user_feature_flag` feature flag is disabled. | | <a id="epicissuehumantimeestimate"></a>`humanTimeEstimate` | [`String`](#string) | Human-readable time estimate of the issue. | @@ -12144,7 +12486,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupcontainerrepositoriescount"></a>`containerRepositoriesCount` | [`Int!`](#int) | Number of container repositories in the group. | | <a id="groupcontainslockedprojects"></a>`containsLockedProjects` | [`Boolean!`](#boolean) | Includes at least one project where the repository size exceeds the limit. | | <a id="groupcrossprojectpipelineavailable"></a>`crossProjectPipelineAvailable` | [`Boolean!`](#boolean) | Indicates if the cross_project_pipeline feature is available for the namespace. | -| <a id="groupcustomemoji"></a>`customEmoji` | [`CustomEmojiConnection`](#customemojiconnection) | Custom emoji within this namespace. Available only when feature flag `custom_emoji` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. (see [Connections](#connections)) | +| <a id="groupcustomemoji"></a>`customEmoji` **{warning-solid}** | [`CustomEmojiConnection`](#customemojiconnection) | **Introduced** in 13.6. This feature is in Alpha. It can be changed or removed at any time. Custom emoji within this namespace. | | <a id="groupdependencyproxyblobcount"></a>`dependencyProxyBlobCount` | [`Int!`](#int) | Number of dependency proxy blobs cached in the group. | | <a id="groupdependencyproxyblobs"></a>`dependencyProxyBlobs` | [`DependencyProxyBlobConnection`](#dependencyproxyblobconnection) | Dependency Proxy blobs. (see [Connections](#connections)) | | <a id="groupdependencyproxyimagecount"></a>`dependencyProxyImageCount` | [`Int!`](#int) | Number of dependency proxy images cached in the group. | @@ -12367,7 +12709,7 @@ Returns [`Epic`](#epic). | <a id="groupepiciid"></a>`iid` | [`ID`](#id) | IID of the epic, e.g., "1". | | <a id="groupepiciidstartswith"></a>`iidStartsWith` | [`String`](#string) | Filter epics by IID for autocomplete. | | <a id="groupepiciids"></a>`iids` | [`[ID!]`](#id) | List of IIDs of epics, e.g., `[1, 2]`. | -| <a id="groupepicin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument. | +| <a id="groupepicin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="groupepicincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include epics from ancestor groups. | | <a id="groupepicincludedescendantgroups"></a>`includeDescendantGroups` | [`Boolean`](#boolean) | Include epics from descendant groups. | | <a id="groupepiclabelname"></a>`labelName` | [`[String!]`](#string) | Filter epics by labels. | @@ -12417,7 +12759,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupepicsiid"></a>`iid` | [`ID`](#id) | IID of the epic, e.g., "1". | | <a id="groupepicsiidstartswith"></a>`iidStartsWith` | [`String`](#string) | Filter epics by IID for autocomplete. | | <a id="groupepicsiids"></a>`iids` | [`[ID!]`](#id) | List of IIDs of epics, e.g., `[1, 2]`. | -| <a id="groupepicsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument. | +| <a id="groupepicsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="groupepicsincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include epics from ancestor groups. | | <a id="groupepicsincludedescendantgroups"></a>`includeDescendantGroups` | [`Boolean`](#boolean) | Include epics from descendant groups. | | <a id="groupepicslabelname"></a>`labelName` | [`[String!]`](#string) | Filter epics by labels. | @@ -12450,6 +12792,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupgroupmembersaccesslevels"></a>`accessLevels` | [`[AccessLevelEnum!]`](#accesslevelenum) | Filter members by the given access levels. | | <a id="groupgroupmembersrelations"></a>`relations` | [`[GroupMemberRelation!]`](#groupmemberrelation) | Filter members by the given member relations. | | <a id="groupgroupmemberssearch"></a>`search` | [`String`](#string) | Search query. | +| <a id="groupgroupmemberssort"></a>`sort` | [`MemberSort`](#membersort) | sort query. | ##### `Group.issues` @@ -12477,8 +12820,10 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupissuescrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. | | <a id="groupissuescrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. | | <a id="groupissuesepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. | +| <a id="groupissueshealthstatus"></a>`healthStatus` | [`HealthStatus`](#healthstatus) | Health status of the issue. | | <a id="groupissuesiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="groupissuesiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. | +| <a id="groupissuesin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="groupissuesincludearchived"></a>`includeArchived` | [`Boolean`](#boolean) | Return issues from archived projects. | | <a id="groupissuesincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. | | <a id="groupissuesincludesubgroups"></a>`includeSubgroups` | [`Boolean`](#boolean) | Include issues belonging to subgroups. | @@ -12653,6 +12998,19 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupmilestonestimeframe"></a>`timeframe` | [`Timeframe`](#timeframe) | List items overlapping the given timeframe. | | <a id="groupmilestonestitle"></a>`title` | [`String`](#string) | Title of the milestone. | +##### `Group.organizationStateCounts` + +Counts of organizations by status for the group. + +Returns [`OrganizationStateCounts`](#organizationstatecounts). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="grouporganizationstatecountssearch"></a>`search` | [`String`](#string) | Search term to find organizations with. | +| <a id="grouporganizationstatecountsstate"></a>`state` | [`CustomerRelationsOrganizationState`](#customerrelationsorganizationstate) | State of the organizations to search for. | + ##### `Group.organizations` Find organizations of this group. @@ -12669,11 +13027,12 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="grouporganizationsids"></a>`ids` | [`[CustomerRelationsOrganizationID!]`](#customerrelationsorganizationid) | Filter organizations by IDs. | | <a id="grouporganizationssearch"></a>`search` | [`String`](#string) | Search term used to find organizations with. | +| <a id="grouporganizationssort"></a>`sort` | [`OrganizationSort`](#organizationsort) | Criteria to sort organizations by. | | <a id="grouporganizationsstate"></a>`state` | [`CustomerRelationsOrganizationState`](#customerrelationsorganizationstate) | State of the organization to search for. | ##### `Group.packages` -Packages of the group. +Packages of the group. This field can only be resolved for one group in any single request. Returns [`PackageConnection`](#packageconnection). @@ -12727,7 +13086,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | | <a id="grouprunnersactive"></a>`active` **{warning-solid}** | [`Boolean`](#boolean) | **Deprecated** in 14.8. This was renamed. Use: `paused`. | -| <a id="grouprunnersmembership"></a>`membership` | [`RunnerMembershipFilter`](#runnermembershipfilter) | Control which runners to include in the results. | +| <a id="grouprunnersmembership"></a>`membership` | [`CiRunnerMembershipFilter`](#cirunnermembershipfilter) | Control which runners to include in the results. | | <a id="grouprunnerspaused"></a>`paused` | [`Boolean`](#boolean) | Filter runners by `paused` (true) or `active` (false) status. | | <a id="grouprunnerssearch"></a>`search` | [`String`](#string) | Filter by full token or partial text in description field. | | <a id="grouprunnerssort"></a>`sort` | [`CiRunnerSort`](#cirunnersort) | Sort order of results. | @@ -12841,8 +13200,10 @@ Returns [`VulnerabilitySeveritiesCount`](#vulnerabilityseveritiescount). | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="groupvulnerabilityseveritiescountclusteragentid"></a>`clusterAgentId` | [`[ClustersAgentID!]`](#clustersagentid) | Filter vulnerabilities by `cluster_agent_id`. Vulnerabilities with a `reportType` of `cluster_image_scanning` are only included with this filter. | | <a id="groupvulnerabilityseveritiescounthasissues"></a>`hasIssues` | [`Boolean`](#boolean) | Filter vulnerabilities that do or do not have issues. | | <a id="groupvulnerabilityseveritiescounthasresolution"></a>`hasResolution` | [`Boolean`](#boolean) | Filter vulnerabilities that do or do not have a resolution. | +| <a id="groupvulnerabilityseveritiescountimage"></a>`image` | [`[String!]`](#string) | Filter vulnerabilities by location image. When this filter is present, the response only matches entries for a `reportType` that includes `container_scanning`, `cluster_image_scanning`. | | <a id="groupvulnerabilityseveritiescountprojectid"></a>`projectId` | [`[ID!]`](#id) | Filter vulnerabilities by project. | | <a id="groupvulnerabilityseveritiescountreporttype"></a>`reportType` | [`[VulnerabilityReportType!]`](#vulnerabilityreporttype) | Filter vulnerabilities by report type. | | <a id="groupvulnerabilityseveritiescountscanner"></a>`scanner` | [`[String!]`](#string) | Filter vulnerabilities by scanner. | @@ -13090,7 +13451,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="instancesecuritydashboardprojectssearch"></a>`search` | [`String`](#string) | Search query for project name, path, or description. | +| <a id="instancesecuritydashboardprojectssearch"></a>`search` | [`String`](#string) | Search query, which can be for the project name, a path, or a description. | ##### `InstanceSecurityDashboard.vulnerabilitySeveritiesCount` @@ -13102,8 +13463,10 @@ Returns [`VulnerabilitySeveritiesCount`](#vulnerabilityseveritiescount). | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="instancesecuritydashboardvulnerabilityseveritiescountclusteragentid"></a>`clusterAgentId` | [`[ClustersAgentID!]`](#clustersagentid) | Filter vulnerabilities by `cluster_agent_id`. Vulnerabilities with a `reportType` of `cluster_image_scanning` are only included with this filter. | | <a id="instancesecuritydashboardvulnerabilityseveritiescounthasissues"></a>`hasIssues` | [`Boolean`](#boolean) | Filter vulnerabilities that do or do not have issues. | | <a id="instancesecuritydashboardvulnerabilityseveritiescounthasresolution"></a>`hasResolution` | [`Boolean`](#boolean) | Filter vulnerabilities that do or do not have a resolution. | +| <a id="instancesecuritydashboardvulnerabilityseveritiescountimage"></a>`image` | [`[String!]`](#string) | Filter vulnerabilities by location image. When this filter is present, the response only matches entries for a `reportType` that includes `container_scanning`, `cluster_image_scanning`. | | <a id="instancesecuritydashboardvulnerabilityseveritiescountprojectid"></a>`projectId` | [`[ID!]`](#id) | Filter vulnerabilities by project. | | <a id="instancesecuritydashboardvulnerabilityseveritiescountreporttype"></a>`reportType` | [`[VulnerabilityReportType!]`](#vulnerabilityreporttype) | Filter vulnerabilities by report type. | | <a id="instancesecuritydashboardvulnerabilityseveritiescountscanner"></a>`scanner` | [`[String!]`](#string) | Filter vulnerabilities by scanner. | @@ -13155,6 +13518,7 @@ Describes an issuable resource link for incident issues. | <a id="issueepic"></a>`epic` | [`Epic`](#epic) | Epic to which this issue belongs. | | <a id="issueescalationpolicy"></a>`escalationPolicy` | [`EscalationPolicyType`](#escalationpolicytype) | Escalation policy associated with the issue. Available for issues which support escalation. | | <a id="issueescalationstatus"></a>`escalationStatus` | [`IssueEscalationStatus`](#issueescalationstatus) | Escalation status of the issue. | +| <a id="issuehasepic"></a>`hasEpic` | [`Boolean!`](#boolean) | Indicates if the issue belongs to an epic. Can return true and not show an associated epic when the user has no access to the epic. | | <a id="issuehealthstatus"></a>`healthStatus` | [`HealthStatus`](#healthstatus) | Current health status. | | <a id="issuehidden"></a>`hidden` | [`Boolean`](#boolean) | Indicates the issue is hidden because the author has been banned. Will always return `null` if `ban_user_feature_flag` feature flag is disabled. | | <a id="issuehumantimeestimate"></a>`humanTimeEstimate` | [`String`](#string) | Human-readable time estimate of the issue. | @@ -13501,6 +13865,19 @@ Maven metadata. | <a id="mavenmetadatapath"></a>`path` | [`String!`](#string) | Path of the Maven package. | | <a id="mavenmetadataupdatedat"></a>`updatedAt` | [`Time!`](#time) | Date of most recent update. | +### `MergeAccessLevel` + +Represents the merge access level of a branch protection. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergeaccesslevelaccesslevel"></a>`accessLevel` | [`Int!`](#int) | GitLab::Access level. | +| <a id="mergeaccesslevelaccessleveldescription"></a>`accessLevelDescription` | [`String!`](#string) | Human readable representation for this access level. | +| <a id="mergeaccesslevelgroup"></a>`group` | [`Group`](#group) | Group associated with this access level. | +| <a id="mergeaccessleveluser"></a>`user` | [`UserCore`](#usercore) | User associated with this access level. | + ### `MergeRequest` #### Fields @@ -13579,6 +13956,7 @@ Maven metadata. | <a id="mergerequestsquashonmerge"></a>`squashOnMerge` | [`Boolean!`](#boolean) | Indicates if squash on merge is enabled. | | <a id="mergerequeststate"></a>`state` | [`MergeRequestState!`](#mergerequeststate) | State of the merge request. | | <a id="mergerequestsubscribed"></a>`subscribed` | [`Boolean!`](#boolean) | Indicates if the currently logged in user is subscribed to this merge request. | +| <a id="mergerequestsuggestedreviewers"></a>`suggestedReviewers` **{warning-solid}** | [`SuggestedReviewersType`](#suggestedreviewerstype) | **Introduced** in 15.4. This feature is in Alpha. It can be changed or removed at any time. Suggested reviewers for merge request. Returns `null` if `suggested_reviewers` feature flag is disabled. This flag is disabled by default and only available on GitLab.com because the feature is experimental and is subject to change without notice. | | <a id="mergerequesttargetbranch"></a>`targetBranch` | [`String!`](#string) | Target branch of the merge request. | | <a id="mergerequesttargetbranchexists"></a>`targetBranchExists` | [`Boolean!`](#boolean) | Indicates if the target branch of the merge request exists. | | <a id="mergerequesttargetproject"></a>`targetProject` | [`Project!`](#project) | Target project of the merge request. | @@ -14953,6 +15331,18 @@ Active period time range for on-call rotation. | <a id="oncallrotationactiveperiodtypeendtime"></a>`endTime` | [`String`](#string) | End of the rotation active period. | | <a id="oncallrotationactiveperiodtypestarttime"></a>`startTime` | [`String`](#string) | Start of the rotation active period. | +### `OrganizationStateCounts` + +Represents the total number of organizations for the represented states. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="organizationstatecountsactive"></a>`active` | [`Int`](#int) | Number of organizations with state `ACTIVE`. | +| <a id="organizationstatecountsall"></a>`all` | [`Int`](#int) | Number of organizations with state `ALL`. | +| <a id="organizationstatecountsinactive"></a>`inactive` | [`Int`](#int) | Number of organizations with state `INACTIVE`. | + ### `Package` Represents a package with pipelines in the Package Registry. @@ -15047,6 +15437,7 @@ Represents a package details in the Package Registry. | <a id="packagedetailstypecreatedat"></a>`createdAt` | [`Time!`](#time) | Date of creation. | | <a id="packagedetailstypedependencylinks"></a>`dependencyLinks` | [`PackageDependencyLinkConnection`](#packagedependencylinkconnection) | Dependency link. (see [Connections](#connections)) | | <a id="packagedetailstypeid"></a>`id` | [`PackagesPackageID!`](#packagespackageid) | ID of the package. | +| <a id="packagedetailstypelastdownloadedat"></a>`lastDownloadedAt` | [`Time`](#time) | Last time that a file of this package was downloaded. | | <a id="packagedetailstypemavenurl"></a>`mavenUrl` | [`String`](#string) | Url of the Maven project endpoint. | | <a id="packagedetailstypemetadata"></a>`metadata` | [`PackageMetadata`](#packagemetadata) | Package metadata. | | <a id="packagedetailstypename"></a>`name` | [`String!`](#string) | Name of the package. | @@ -15438,8 +15829,9 @@ Represents vulnerability finding of a security report on the pipeline. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="pipelinesecurityreportfindingassets"></a>`assets` | [`[AssetType!]`](#assettype) | List of assets associated with the vulnerability. | -| <a id="pipelinesecurityreportfindingconfidence"></a>`confidence` | [`String`](#string) | Type of the security report that found the vulnerability. | +| <a id="pipelinesecurityreportfindingconfidence"></a>`confidence` **{warning-solid}** | [`String`](#string) | **Deprecated** in 15.4. This field will be removed from the Finding domain model. | | <a id="pipelinesecurityreportfindingdescription"></a>`description` | [`String`](#string) | Description of the vulnerability finding. | +| <a id="pipelinesecurityreportfindingdescriptionhtml"></a>`descriptionHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `description`. | | <a id="pipelinesecurityreportfindingevidence"></a>`evidence` | [`VulnerabilityEvidence`](#vulnerabilityevidence) | Evidence for the vulnerability. | | <a id="pipelinesecurityreportfindingfalsepositive"></a>`falsePositive` | [`Boolean`](#boolean) | Indicates whether the vulnerability is a false positive. | | <a id="pipelinesecurityreportfindingidentifiers"></a>`identifiers` | [`[VulnerabilityIdentifier!]!`](#vulnerabilityidentifier) | Identifiers of the vulnerability finding. | @@ -15469,6 +15861,7 @@ Represents vulnerability finding of a security report on the pipeline. | <a id="projectarchived"></a>`archived` | [`Boolean`](#boolean) | Indicates the archived status of the project. | | <a id="projectautoclosereferencedissues"></a>`autocloseReferencedIssues` | [`Boolean`](#boolean) | Indicates if issues referenced by merge requests and commits within the default branch are closed automatically. | | <a id="projectavatarurl"></a>`avatarUrl` | [`String`](#string) | URL to avatar image file of the project. | +| <a id="projectbranchrules"></a>`branchRules` | [`BranchRuleConnection`](#branchruleconnection) | Branch rules configured for the project. (see [Connections](#connections)) | | <a id="projectcicdsettings"></a>`ciCdSettings` | [`ProjectCiCdSetting`](#projectcicdsetting) | CI/CD settings for the project. | | <a id="projectciconfigpathordefault"></a>`ciConfigPathOrDefault` | [`String!`](#string) | Path of the CI configuration file. | | <a id="projectcijobtokenscope"></a>`ciJobTokenScope` | [`CiJobTokenScopeType`](#cijobtokenscopetype) | The CI Job Tokens scope of access. | @@ -15671,6 +16064,22 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="projectboardsid"></a>`id` | [`BoardID`](#boardid) | Find a board by its ID. | +##### `Project.ciConfigVariables` + +CI/CD config variable. + +WARNING: +**Introduced** in 15.3. +This feature is in Alpha. It can be changed or removed at any time. + +Returns [`[CiConfigVariable!]`](#ciconfigvariable). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectciconfigvariablessha"></a>`sha` | [`String!`](#string) | Sha. | + ##### `Project.ciTemplate` Find a single CI/CD template by name. @@ -15787,6 +16196,18 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="projectdastsitevalidationsnormalizedtargeturls"></a>`normalizedTargetUrls` | [`[String!]`](#string) | Normalized URL of the target to be scanned. | | <a id="projectdastsitevalidationsstatus"></a>`status` | [`DastSiteValidationStatusEnum`](#dastsitevalidationstatusenum) | Status of the site validation. | +##### `Project.deployment` + +Details of the deployment of the project. + +Returns [`DeploymentDetails`](#deploymentdetails). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectdeploymentiid"></a>`iid` | [`ID!`](#id) | Project-level internal ID of the Deployment. | + ##### `Project.environment` A single environment of the project. @@ -15931,8 +16352,10 @@ Returns [`Issue`](#issue). | <a id="projectissuecrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. | | <a id="projectissuecrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. | | <a id="projectissueepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. | +| <a id="projectissuehealthstatus"></a>`healthStatus` | [`HealthStatus`](#healthstatus) | Health status of the issue. | | <a id="projectissueiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="projectissueiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. | +| <a id="projectissuein"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="projectissueincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. | | <a id="projectissueiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. | | <a id="projectissueiterationwildcardid"></a>`iterationWildcardId` | [`IterationWildcardId`](#iterationwildcardid) | Filter by iteration ID wildcard. | @@ -15974,6 +16397,7 @@ Returns [`IssueStatusCountsType`](#issuestatuscountstype). | <a id="projectissuestatuscountscrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. | | <a id="projectissuestatuscountsiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="projectissuestatuscountsiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. | +| <a id="projectissuestatuscountsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="projectissuestatuscountslabelname"></a>`labelName` | [`[String]`](#string) | Labels applied to this issue. | | <a id="projectissuestatuscountsmilestonetitle"></a>`milestoneTitle` | [`[String]`](#string) | Milestone applied to this issue. | | <a id="projectissuestatuscountsmilestonewildcardid"></a>`milestoneWildcardId` | [`MilestoneWildcardId`](#milestonewildcardid) | Filter issues by milestone ID wildcard. | @@ -16012,8 +16436,10 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="projectissuescrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. | | <a id="projectissuescrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. | | <a id="projectissuesepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. | +| <a id="projectissueshealthstatus"></a>`healthStatus` | [`HealthStatus`](#healthstatus) | Health status of the issue. | | <a id="projectissuesiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="projectissuesiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. | +| <a id="projectissuesin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="projectissuesincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. | | <a id="projectissuesiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. | | <a id="projectissuesiterationwildcardid"></a>`iterationWildcardId` | [`IterationWildcardId`](#iterationwildcardid) | Filter by iteration ID wildcard. | @@ -16080,6 +16506,18 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="projectiterationstimeframe"></a>`timeframe` | [`Timeframe`](#timeframe) | List items overlapping the given timeframe. | | <a id="projectiterationstitle"></a>`title` **{warning-solid}** | [`String`](#string) | **Deprecated** in 15.4. The argument will be removed in 15.4. Please use `search` and `in` fields instead. | +##### `Project.job` + +One job belonging to the project, selected by ID. + +Returns [`CiJob`](#cijob). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectjobid"></a>`id` | [`JobID!`](#jobid) | ID of the job. | + ##### `Project.jobs` Jobs of a project. This field can only be resolved for one project in any single request. @@ -16301,6 +16739,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="projectprojectmembersrelations"></a>`relations` | [`[ProjectMemberRelation!]`](#projectmemberrelation) | Filter members by the given member relations. | | <a id="projectprojectmemberssearch"></a>`search` | [`String`](#string) | Search query. | +| <a id="projectprojectmemberssort"></a>`sort` | [`MemberSort`](#membersort) | sort query. | ##### `Project.release` @@ -16546,8 +16985,10 @@ Returns [`VulnerabilitySeveritiesCount`](#vulnerabilityseveritiescount). | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="projectvulnerabilityseveritiescountclusteragentid"></a>`clusterAgentId` | [`[ClustersAgentID!]`](#clustersagentid) | Filter vulnerabilities by `cluster_agent_id`. Vulnerabilities with a `reportType` of `cluster_image_scanning` are only included with this filter. | | <a id="projectvulnerabilityseveritiescounthasissues"></a>`hasIssues` | [`Boolean`](#boolean) | Filter vulnerabilities that do or do not have issues. | | <a id="projectvulnerabilityseveritiescounthasresolution"></a>`hasResolution` | [`Boolean`](#boolean) | Filter vulnerabilities that do or do not have a resolution. | +| <a id="projectvulnerabilityseveritiescountimage"></a>`image` | [`[String!]`](#string) | Filter vulnerabilities by location image. When this filter is present, the response only matches entries for a `reportType` that includes `container_scanning`, `cluster_image_scanning`. | | <a id="projectvulnerabilityseveritiescountprojectid"></a>`projectId` | [`[ID!]`](#id) | Filter vulnerabilities by project. | | <a id="projectvulnerabilityseveritiescountreporttype"></a>`reportType` | [`[VulnerabilityReportType!]`](#vulnerabilityreporttype) | Filter vulnerabilities by report type. | | <a id="projectvulnerabilityseveritiescountscanner"></a>`scanner` | [`[String!]`](#string) | Filter vulnerabilities by scanner. | @@ -16591,6 +17032,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="projectworkitemsiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="projectworkitemsiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of work items. For example, `["1", "2"]`. | +| <a id="projectworkitemsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="projectworkitemssearch"></a>`search` | [`String`](#string) | Search query for title or description. | | <a id="projectworkitemssort"></a>`sort` | [`WorkItemSort`](#workitemsort) | Sort work items by this criteria. | | <a id="projectworkitemsstate"></a>`state` | [`IssuableState`](#issuablestate) | Current state of this work item. | @@ -16743,6 +17185,19 @@ The alert condition for Prometheus. | <a id="prometheusalerthumanizedtext"></a>`humanizedText` | [`String!`](#string) | Human-readable text of the alert condition. | | <a id="prometheusalertid"></a>`id` | [`ID!`](#id) | ID of the alert condition. | +### `PushAccessLevel` + +Represents the push access level of a branch protection. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="pushaccesslevelaccesslevel"></a>`accessLevel` | [`Int!`](#int) | GitLab::Access level. | +| <a id="pushaccesslevelaccessleveldescription"></a>`accessLevelDescription` | [`String!`](#string) | Human readable representation for this access level. | +| <a id="pushaccesslevelgroup"></a>`group` | [`Group`](#group) | Group associated with this access level. | +| <a id="pushaccessleveluser"></a>`user` | [`UserCore`](#usercore) | User associated with this access level. | + ### `PushRules` Represents rules that commit pushes must follow. @@ -17629,6 +18084,18 @@ Represents an entry from the future subscriptions. | <a id="subscriptionfutureentrytype"></a>`type` | [`String!`](#string) | Type of license the subscription will yield. | | <a id="subscriptionfutureentryusersinlicensecount"></a>`usersInLicenseCount` | [`Int`](#int) | Number of paid user seats. | +### `SuggestedReviewersType` + +Represents a Suggested Reviewers result set. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="suggestedreviewerstypereviewers"></a>`reviewers` | [`[String!]!`](#string) | List of reviewers. | +| <a id="suggestedreviewerstypetopn"></a>`topN` | [`Int`](#int) | Number of reviewers returned. | +| <a id="suggestedreviewerstypeversion"></a>`version` | [`String`](#string) | Suggested reviewer version. | + ### `TaskCompletionStatus` Completion status of tasks. @@ -18785,6 +19252,7 @@ Represents a vulnerability scanner. | <a id="vulnerabilityscannerid"></a>`id` | [`ID`](#id) | ID of the scanner. | | <a id="vulnerabilityscannername"></a>`name` | [`String`](#string) | Name of the vulnerability scanner. | | <a id="vulnerabilityscannerreporttype"></a>`reportType` | [`VulnerabilityReportType`](#vulnerabilityreporttype) | Type of the vulnerability report. | +| <a id="vulnerabilityscannerreporttypehumanized"></a>`reportTypeHumanized` | [`String`](#string) | Humanized type of the vulnerability report. | | <a id="vulnerabilityscannervendor"></a>`vendor` | [`String`](#string) | Vendor of the vulnerability scanner. | ### `VulnerabilitySeveritiesCount` @@ -18918,6 +19386,9 @@ Represents a description widget. | ---- | ---- | ----------- | | <a id="workitemwidgetdescriptiondescription"></a>`description` | [`String`](#string) | Description of the work item. | | <a id="workitemwidgetdescriptiondescriptionhtml"></a>`descriptionHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `description`. | +| <a id="workitemwidgetdescriptionedited"></a>`edited` | [`Boolean!`](#boolean) | Whether the description has been edited since the work item was created. | +| <a id="workitemwidgetdescriptionlasteditedat"></a>`lastEditedAt` | [`Time`](#time) | Timestamp of when the work item's description was last edited. | +| <a id="workitemwidgetdescriptionlasteditedby"></a>`lastEditedBy` | [`UserCore`](#usercore) | User that made the last edit to the work item's description. | | <a id="workitemwidgetdescriptiontype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. | ### `WorkItemWidgetHierarchy` @@ -18932,6 +19403,17 @@ Represents a hierarchy widget. | <a id="workitemwidgethierarchyparent"></a>`parent` | [`WorkItem`](#workitem) | Parent work item. | | <a id="workitemwidgethierarchytype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. | +### `WorkItemWidgetIteration` + +Represents an iteration widget. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="workitemwidgetiterationiteration"></a>`iteration` | [`Iteration`](#iteration) | Iteration of the work item. | +| <a id="workitemwidgetiterationtype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. | + ### `WorkItemWidgetLabels` Represents the labels widget. @@ -19207,6 +19689,15 @@ Values for YAML processor result. | <a id="cirunneraccesslevelnot_protected"></a>`NOT_PROTECTED` | A runner that is not protected. | | <a id="cirunneraccesslevelref_protected"></a>`REF_PROTECTED` | A runner that is ref protected. | +### `CiRunnerMembershipFilter` + +Values for filtering runners in namespaces. The previous type name `RunnerMembershipFilter` was deprecated in 15.4. + +| Value | Description | +| ----- | ----------- | +| <a id="cirunnermembershipfilterdescendants"></a>`DESCENDANTS` | Include runners that have either a direct or inherited relationship. These runners can be specific to a project or a group. | +| <a id="cirunnermembershipfilterdirect"></a>`DIRECT` | Include runners that have a direct relationship. | + ### `CiRunnerSort` Values for sorting runners. @@ -19339,18 +19830,18 @@ Values for sorting contacts. | ----- | ----------- | | <a id="contactsortcreated_asc"></a>`CREATED_ASC` | Created at ascending order. | | <a id="contactsortcreated_desc"></a>`CREATED_DESC` | Created at descending order. | -| <a id="contactsortdescription_asc"></a>`DESCRIPTION_ASC` | Description by ascending order. | -| <a id="contactsortdescription_desc"></a>`DESCRIPTION_DESC` | Description by descending order. | -| <a id="contactsortemail_asc"></a>`EMAIL_ASC` | Email by ascending order. | -| <a id="contactsortemail_desc"></a>`EMAIL_DESC` | Email by descending order. | -| <a id="contactsortfirst_name_asc"></a>`FIRST_NAME_ASC` | First name by ascending order. | -| <a id="contactsortfirst_name_desc"></a>`FIRST_NAME_DESC` | First name by descending order. | -| <a id="contactsortlast_name_asc"></a>`LAST_NAME_ASC` | Last name by ascending order. | -| <a id="contactsortlast_name_desc"></a>`LAST_NAME_DESC` | Last name by descending order. | -| <a id="contactsortorganization_asc"></a>`ORGANIZATION_ASC` | Organization by ascending order. | -| <a id="contactsortorganization_desc"></a>`ORGANIZATION_DESC` | Organization by descending order. | -| <a id="contactsortphone_asc"></a>`PHONE_ASC` | Phone by ascending order. | -| <a id="contactsortphone_desc"></a>`PHONE_DESC` | Phone by descending order. | +| <a id="contactsortdescription_asc"></a>`DESCRIPTION_ASC` | Description in ascending order. | +| <a id="contactsortdescription_desc"></a>`DESCRIPTION_DESC` | Description in descending order. | +| <a id="contactsortemail_asc"></a>`EMAIL_ASC` | Email in ascending order. | +| <a id="contactsortemail_desc"></a>`EMAIL_DESC` | Email in descending order. | +| <a id="contactsortfirst_name_asc"></a>`FIRST_NAME_ASC` | First name in ascending order. | +| <a id="contactsortfirst_name_desc"></a>`FIRST_NAME_DESC` | First name in descending order. | +| <a id="contactsortlast_name_asc"></a>`LAST_NAME_ASC` | Last name in ascending order. | +| <a id="contactsortlast_name_desc"></a>`LAST_NAME_DESC` | Last name in descending order. | +| <a id="contactsortorganization_asc"></a>`ORGANIZATION_ASC` | Organization in ascending order. | +| <a id="contactsortorganization_desc"></a>`ORGANIZATION_DESC` | Organization in descending order. | +| <a id="contactsortphone_asc"></a>`PHONE_ASC` | Phone in ascending order. | +| <a id="contactsortphone_desc"></a>`PHONE_DESC` | Phone in descending order. | | <a id="contactsortupdated_asc"></a>`UPDATED_ASC` | Updated at ascending order. | | <a id="contactsortupdated_desc"></a>`UPDATED_DESC` | Updated at descending order. | | <a id="contactsortcreated_asc"></a>`created_asc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `CREATED_ASC`. | @@ -19447,8 +19938,9 @@ Values for sorting tags. | Value | Description | | ----- | ----------- | -| <a id="customerrelationsorganizationstateactive"></a>`active` | Active organization. | -| <a id="customerrelationsorganizationstateinactive"></a>`inactive` | Inactive organization. | +| <a id="customerrelationsorganizationstateactive"></a>`active` | Active organizations. | +| <a id="customerrelationsorganizationstateall"></a>`all` | All available organizations. | +| <a id="customerrelationsorganizationstateinactive"></a>`inactive` | Inactive organizations. | ### `DastProfileCadenceUnit` @@ -19552,6 +20044,20 @@ Weight of the data visualization palette. | <a id="dependencyproxymanifeststatuspending_destruction"></a>`PENDING_DESTRUCTION` | Dependency proxy manifest has a status of pending_destruction. | | <a id="dependencyproxymanifeststatusprocessing"></a>`PROCESSING` | Dependency proxy manifest has a status of processing. | +### `DeploymentStatus` + +All deployment statuses. + +| Value | Description | +| ----- | ----------- | +| <a id="deploymentstatusblocked"></a>`BLOCKED` | A deployment that is blocked. | +| <a id="deploymentstatuscanceled"></a>`CANCELED` | A deployment that is canceled. | +| <a id="deploymentstatuscreated"></a>`CREATED` | A deployment that is created. | +| <a id="deploymentstatusfailed"></a>`FAILED` | A deployment that is failed. | +| <a id="deploymentstatusrunning"></a>`RUNNING` | A deployment that is running. | +| <a id="deploymentstatusskipped"></a>`SKIPPED` | A deployment that is skipped. | +| <a id="deploymentstatussuccess"></a>`SUCCESS` | A deployment that is success. | + ### `DeploymentTier` All environment deployment tiers. @@ -19595,6 +20101,7 @@ Detailed representation of whether a GitLab merge request can be merged. | <a id="detailedmergestatusbroken_status"></a>`BROKEN_STATUS` | Can not merge the source into the target branch, potential conflict. | | <a id="detailedmergestatuschecking"></a>`CHECKING` | Currently checking for mergeability. | | <a id="detailedmergestatusci_must_pass"></a>`CI_MUST_PASS` | Pipeline must succeed before merging. | +| <a id="detailedmergestatusci_still_running"></a>`CI_STILL_RUNNING` | Pipeline is still running. | | <a id="detailedmergestatusdiscussions_not_resolved"></a>`DISCUSSIONS_NOT_RESOLVED` | Discussions must be resolved before merging. | | <a id="detailedmergestatusdraft_status"></a>`DRAFT_STATUS` | Merge request must not be draft before merging. | | <a id="detailedmergestatusmergeable"></a>`MERGEABLE` | Branch can be merged. | @@ -19985,6 +20492,25 @@ Possible identifier types for a measurement. | <a id="measurementidentifierprojects"></a>`PROJECTS` | Project count. | | <a id="measurementidentifierusers"></a>`USERS` | User count. | +### `MemberSort` + +Values for sorting members. + +| Value | Description | +| ----- | ----------- | +| <a id="membersortaccess_level_asc"></a>`ACCESS_LEVEL_ASC` | Access level ascending order. | +| <a id="membersortaccess_level_desc"></a>`ACCESS_LEVEL_DESC` | Access level descending order. | +| <a id="membersortcreated_asc"></a>`CREATED_ASC` | Created at ascending order. | +| <a id="membersortcreated_desc"></a>`CREATED_DESC` | Created at descending order. | +| <a id="membersortupdated_asc"></a>`UPDATED_ASC` | Updated at ascending order. | +| <a id="membersortupdated_desc"></a>`UPDATED_DESC` | Updated at descending order. | +| <a id="membersortuser_full_name_asc"></a>`USER_FULL_NAME_ASC` | User's full name ascending order. | +| <a id="membersortuser_full_name_desc"></a>`USER_FULL_NAME_DESC` | User's full name descending order. | +| <a id="membersortcreated_asc"></a>`created_asc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `CREATED_ASC`. | +| <a id="membersortcreated_desc"></a>`created_desc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `CREATED_DESC`. | +| <a id="membersortupdated_asc"></a>`updated_asc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `UPDATED_ASC`. | +| <a id="membersortupdated_desc"></a>`updated_desc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `UPDATED_DESC`. | + ### `MergeRequestNewState` New state to apply to a merge request. @@ -20000,7 +20526,6 @@ State of a review of a GitLab merge request. | Value | Description | | ----- | ----------- | -| <a id="mergerequestreviewstateattention_requested"></a>`ATTENTION_REQUESTED` | The merge request is attention_requested. | | <a id="mergerequestreviewstatereviewed"></a>`REVIEWED` | The merge request is reviewed. | | <a id="mergerequestreviewstateunreviewed"></a>`UNREVIEWED` | The merge request is unreviewed. | @@ -20166,6 +20691,27 @@ Rotation length unit of an on-call rotation. | <a id="oncallrotationunitenumhours"></a>`HOURS` | Hours. | | <a id="oncallrotationunitenumweeks"></a>`WEEKS` | Weeks. | +### `OrganizationSort` + +Values for sorting organizations. + +| Value | Description | +| ----- | ----------- | +| <a id="organizationsortcreated_asc"></a>`CREATED_ASC` | Created at ascending order. | +| <a id="organizationsortcreated_desc"></a>`CREATED_DESC` | Created at descending order. | +| <a id="organizationsortdefault_rate_asc"></a>`DEFAULT_RATE_ASC` | Default Rate in ascending order. | +| <a id="organizationsortdefault_rate_desc"></a>`DEFAULT_RATE_DESC` | Default Rate in descending order. | +| <a id="organizationsortdescription_asc"></a>`DESCRIPTION_ASC` | Description in ascending order. | +| <a id="organizationsortdescription_desc"></a>`DESCRIPTION_DESC` | Description in descending order. | +| <a id="organizationsortname_asc"></a>`NAME_ASC` | Name in ascending order. | +| <a id="organizationsortname_desc"></a>`NAME_DESC` | Name in descending order. | +| <a id="organizationsortupdated_asc"></a>`UPDATED_ASC` | Updated at ascending order. | +| <a id="organizationsortupdated_desc"></a>`UPDATED_DESC` | Updated at descending order. | +| <a id="organizationsortcreated_asc"></a>`created_asc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `CREATED_ASC`. | +| <a id="organizationsortcreated_desc"></a>`created_desc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `CREATED_DESC`. | +| <a id="organizationsortupdated_asc"></a>`updated_asc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `UPDATED_ASC`. | +| <a id="organizationsortupdated_desc"></a>`updated_desc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `UPDATED_DESC`. | + ### `PackageDependencyType` | Value | Description | @@ -20231,6 +20777,7 @@ Values for sorting package. | <a id="packagetypeenumnpm"></a>`NPM` | Packages from the npm package manager. | | <a id="packagetypeenumnuget"></a>`NUGET` | Packages from the Nuget package manager. | | <a id="packagetypeenumpypi"></a>`PYPI` | Packages from the PyPI package manager. | +| <a id="packagetypeenumrpm"></a>`RPM` | Packages from the Rpm package manager. | | <a id="packagetypeenumrubygems"></a>`RUBYGEMS` | Packages from the Rubygems package manager. | | <a id="packagetypeenumterraform_module"></a>`TERRAFORM_MODULE` | Packages from the Terraform Module package manager. | @@ -20377,15 +20924,6 @@ Status of a requirement based on last test report. | <a id="requirementstatusfiltermissing"></a>`MISSING` | Requirements without any test report. | | <a id="requirementstatusfilterpassed"></a>`PASSED` | Passed test report. | -### `RunnerMembershipFilter` - -Values for filtering runners in namespaces. - -| Value | Description | -| ----- | ----------- | -| <a id="runnermembershipfilterdescendants"></a>`DESCENDANTS` | Include runners that have either a direct relationship or a relationship with descendants. These can be project runners or group runners (in the case where group is queried). | -| <a id="runnermembershipfilterdirect"></a>`DIRECT` | Include runners that have a direct relationship. | - ### `SastUiComponentSize` Size of UI component in SAST configuration page. @@ -20547,6 +21085,15 @@ Common sort values. | <a id="sortupdated_asc"></a>`updated_asc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `UPDATED_ASC`. | | <a id="sortupdated_desc"></a>`updated_desc` **{warning-solid}** | **Deprecated** in 13.5. This was renamed. Use: `UPDATED_DESC`. | +### `SortDirectionEnum` + +Values for sort direction. + +| Value | Description | +| ----- | ----------- | +| <a id="sortdirectionenumasc"></a>`ASC` | Ascending order. | +| <a id="sortdirectionenumdesc"></a>`DESC` | Descending order. | + ### `TestCaseStatus` | Value | Description | @@ -20644,8 +21191,6 @@ Name of the feature that the callout is for. | Value | Description | | ----- | ----------- | | <a id="usercalloutfeaturenameenumactive_user_count_threshold"></a>`ACTIVE_USER_COUNT_THRESHOLD` | Callout feature name for active_user_count_threshold. | -| <a id="usercalloutfeaturenameenumattention_requests_side_nav"></a>`ATTENTION_REQUESTS_SIDE_NAV` | Callout feature name for attention_requests_side_nav. | -| <a id="usercalloutfeaturenameenumattention_requests_top_nav"></a>`ATTENTION_REQUESTS_TOP_NAV` | Callout feature name for attention_requests_top_nav. | | <a id="usercalloutfeaturenameenumbuy_pipeline_minutes_notification_dot"></a>`BUY_PIPELINE_MINUTES_NOTIFICATION_DOT` | Callout feature name for buy_pipeline_minutes_notification_dot. | | <a id="usercalloutfeaturenameenumcanary_deployment"></a>`CANARY_DEPLOYMENT` | Callout feature name for canary_deployment. | | <a id="usercalloutfeaturenameenumci_deprecation_warning_for_types_keyword"></a>`CI_DEPRECATION_WARNING_FOR_TYPES_KEYWORD` | Callout feature name for ci_deprecation_warning_for_types_keyword. | @@ -20658,6 +21203,7 @@ Name of the feature that the callout is for. | <a id="usercalloutfeaturenameenumgeo_migrate_hashed_storage"></a>`GEO_MIGRATE_HASHED_STORAGE` | Callout feature name for geo_migrate_hashed_storage. | | <a id="usercalloutfeaturenameenumgke_cluster_integration"></a>`GKE_CLUSTER_INTEGRATION` | Callout feature name for gke_cluster_integration. | | <a id="usercalloutfeaturenameenumgold_trial_billings"></a>`GOLD_TRIAL_BILLINGS` | Callout feature name for gold_trial_billings. | +| <a id="usercalloutfeaturenameenummerge_request_settings_moved_callout"></a>`MERGE_REQUEST_SETTINGS_MOVED_CALLOUT` | Callout feature name for merge_request_settings_moved_callout. | | <a id="usercalloutfeaturenameenummr_experience_survey"></a>`MR_EXPERIENCE_SURVEY` | Callout feature name for mr_experience_survey. | | <a id="usercalloutfeaturenameenumnamespace_storage_limit_banner_alert_threshold"></a>`NAMESPACE_STORAGE_LIMIT_BANNER_ALERT_THRESHOLD` | Callout feature name for namespace_storage_limit_banner_alert_threshold. | | <a id="usercalloutfeaturenameenumnamespace_storage_limit_banner_error_threshold"></a>`NAMESPACE_STORAGE_LIMIT_BANNER_ERROR_THRESHOLD` | Callout feature name for namespace_storage_limit_banner_error_threshold. | @@ -20899,6 +21445,7 @@ Type of a work item widget. | <a id="workitemwidgettypeassignees"></a>`ASSIGNEES` | Assignees widget. | | <a id="workitemwidgettypedescription"></a>`DESCRIPTION` | Description widget. | | <a id="workitemwidgettypehierarchy"></a>`HIERARCHY` | Hierarchy widget. | +| <a id="workitemwidgettypeiteration"></a>`ITERATION` | Iteration widget. | | <a id="workitemwidgettypelabels"></a>`LABELS` | Labels widget. | | <a id="workitemwidgettypestart_and_due_date"></a>`START_AND_DUE_DATE` | Start And Due Date widget. | | <a id="workitemwidgettypeverification_status"></a>`VERIFICATION_STATUS` | Verification Status widget. | @@ -20983,6 +21530,12 @@ A `CiBuildID` is a global ID. It is encoded as a string. An example `CiBuildID` is: `"gid://gitlab/Ci::Build/1"`. +### `CiJobArtifactID` + +A `CiJobArtifactID` is a global ID. It is encoded as a string. + +An example `CiJobArtifactID` is: `"gid://gitlab/Ci::JobArtifact/1"`. + ### `CiPipelineID` A `CiPipelineID` is a global ID. It is encoded as a string. @@ -22155,6 +22708,7 @@ Implementations: - [`WorkItemWidgetAssignees`](#workitemwidgetassignees) - [`WorkItemWidgetDescription`](#workitemwidgetdescription) - [`WorkItemWidgetHierarchy`](#workitemwidgethierarchy) +- [`WorkItemWidgetIteration`](#workitemwidgetiteration) - [`WorkItemWidgetLabels`](#workitemwidgetlabels) - [`WorkItemWidgetStartAndDueDate`](#workitemwidgetstartandduedate) - [`WorkItemWidgetVerificationStatus`](#workitemwidgetverificationstatus) @@ -22301,6 +22855,17 @@ Input type for DastSiteProfile authentication. | <a id="dastsiteprofileauthinputusername"></a>`username` | [`String`](#string) | Username to authenticate with on the target. | | <a id="dastsiteprofileauthinputusernamefield"></a>`usernameField` | [`String`](#string) | Name of username field at the sign-in HTML form. | +### `DeploymentsOrderByInput` + +Values for ordering deployments by a specific field. + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="deploymentsorderbyinputcreatedat"></a>`createdAt` | [`SortDirectionEnum`](#sortdirectionenum) | Order by Created time. | +| <a id="deploymentsorderbyinputfinishedat"></a>`finishedAt` | [`SortDirectionEnum`](#sortdirectionenum) | Order by Finished time. | + ### `DiffImagePositionInput` #### Arguments @@ -22697,6 +23262,14 @@ A time-frame defined as a closed inclusive range of two dates. | <a id="workitemwidgethierarchyupdateinputchildrenids"></a>`childrenIds` | [`[WorkItemID!]`](#workitemid) | Global IDs of children work items. | | <a id="workitemwidgethierarchyupdateinputparentid"></a>`parentId` | [`WorkItemID`](#workitemid) | Global ID of the parent work item. Use `null` to remove the association. | +### `WorkItemWidgetIterationInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="workitemwidgetiterationinputiterationid"></a>`iterationId` | [`IterationID`](#iterationid) | Iteration to assign to the work item. | + ### `WorkItemWidgetStartAndDueDateUpdateInput` #### Arguments diff --git a/doc/api/graphql/users_example.md b/doc/api/graphql/users_example.md index 60ca7090aac..1c366587e5f 100644 --- a/doc/api/graphql/users_example.md +++ b/doc/api/graphql/users_example.md @@ -1,6 +1,6 @@ --- -stage: Ecosystem -group: Integrations +stage: Manage +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/group_protected_environments.md b/doc/api/group_protected_environments.md index 0f1527f8968..3f1d932e0c8 100644 --- a/doc/api/group_protected_environments.md +++ b/doc/api/group_protected_environments.md @@ -15,7 +15,7 @@ Read more about [group-level protected environments](../ci/environments/protecte ## Valid access levels -The access levels are defined in the `ProtectedEnvironment::DeployAccessLevel::ALLOWED_ACCESS_LEVELS` method. +The access levels are defined in the `ProtectedEnvironments::DeployAccessLevel::ALLOWED_ACCESS_LEVELS` method. Currently, these levels are recognized: ```plaintext @@ -48,13 +48,14 @@ Example response: "name":"production", "deploy_access_levels":[ { + "id": 12, "access_level": 40, "access_level_description": "Maintainers", "user_id": null, "group_id": null } ], - "required_approval_count": 0 + "required_approval_count": 0 } ] ``` @@ -83,6 +84,7 @@ Example response: "name":"production", "deploy_access_levels":[ { + "id": 12, "access_level":40, "access_level_description":"Maintainers", "user_id":null, @@ -123,13 +125,182 @@ Example response: "name":"production", "deploy_access_levels":[ { + "id": 12, "access_level": 40, "access_level_description": "protected-access-group", "user_id": null, "group_id": 9899826 } ], - "required_approval_count": 0 + "required_approval_count": 0 +} +``` + +## Update an environment + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/351854) in GitLab 15.4. + +Updates a single environment. + +```plaintext +PUT /groups/:id/protected_environments/:name +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) maintained by the authenticated user. | +| `name` | string | yes | The deployment tier of the protected environment. One of `production`, `staging`, `testing`, `development`, or `other`. Read more about [deployment tiers](../ci/environments/index.md#deployment-tier-of-environments).| +| `deploy_access_levels` | array | no | Array of access levels allowed to deploy, with each described by a hash. One of `user_id`, `group_id` or `access_level`. They take the form of `{user_id: integer}`, `{group_id: integer}` or `{access_level: integer}` respectively. | +| `required_approval_count` | integer | no | The number of approvals required to deploy to this environment. | +| `approval_rules` | array | no | Array of access levels allowed to approve, with each described by a hash. One of `user_id`, `group_id` or `access_level`. They take the form of `{user_id: integer}`, `{group_id: integer}` or `{access_level: integer}` respectively. You can also specify the number of required approvals from the specified entity with `required_approvals` field. See [Multiple approval rules](../ci/environments/deployment_approvals.md#multiple-approval-rules) for more information. | + +To update: + +- **`user_id`**: Ensure the updated user belongs to the given group with the Maintainer role (or above). You must also pass the `id` of either a `deploy_access_level` or `approval_rule` in the respective hash. +- **`group_id`**: Ensure the updated group is a sub-group of the group this protected environment belongs to. You must also pass the `id` of either a `deploy_access_level` or `approval_rule` in the respective hash. + +To delete: + +- You must pass `_destroy` set to `true`. See the following examples. + +### Example: Create a `deploy_access_level` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"deploy_access_levels": [{"group_id": 9899829, access_level: 40}], "required_approval_count": 1}' \ + --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/groups/22034114/protected_environments/production" +``` + +Example response: + +```json +{ + "name": "production", + "deploy_access_levels": [ + { + "id": 12, + "access_level": 40, + "access_level_description": "protected-access-group", + "user_id": null, + "group_id": 9899829, + "group_inheritance_type": 1 + } + ], + "required_approval_count": 0 +} +``` + +### Example: Update a `deploy_access_level` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"deploy_access_levels": [{"id": 12, "group_id": 22034120}], "required_approval_count": 2}' \ + --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/22034114/protected_environments/production" +``` + +```json +{ + "name": "production", + "deploy_access_levels": [ + { + "id": 12, + "access_level": 40, + "access_level_description": "protected-access-group", + "user_id": null, + "group_id": 22034120, + "group_inheritance_type": 0 + } + ], + "required_approval_count": 2 +} +``` + +### Example: Delete a `deploy_access_level` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"deploy_access_levels": [{"id": 12, "_destroy": true}], "required_approval_count": 0}' \ + --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/22034114/protected_environments/production" +``` + +Example response: + +```json +{ + "name": "production", + "deploy_access_levels": [], + "required_approval_count": 0 +} +``` + +### Example: Create an `approval_rule` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"approval_rules": [{"group_id": 134, "required_approvals": 1}]}' \ + --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/groups/22034114/protected_environments/production" +``` + +Example response: + +```json +{ + "name": "production", + "approval_rules": [ + { + "id": 38, + "user_id": null, + "group_id": 134, + "access_level": null, + "access_level_description": "qa-group", + "required_approvals": 1, + "group_inheritance_type": 0 + } + ] +} +``` + +### Example: Update an `approval_rule` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"approval_rules": [{"id": 38, "group_id": 135, "required_approvals": 2}]}' \ + --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/22034114/protected_environments/production" +``` + +```json +{ + "name": "production", + "approval_rules": [ + { + "id": 38, + "user_id": null, + "group_id": 135, + "access_level": null, + "access_level_description": "security-group", + "required_approvals": 2, + "group_inheritance_type": 0 + } + ] +} +``` + +### Example: Delete an `approval_rule` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"approval_rules": [{"id": 38, "_destroy": true}]}' \ + --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/22034114/protected_environments/production" +``` + +Example response: + +```json +{ + "name": "production", + "approval_rules": [] } ``` diff --git a/doc/api/group_repository_storage_moves.md b/doc/api/group_repository_storage_moves.md index 1b3940479bb..f1ad7f51ea0 100644 --- a/doc/api/group_repository_storage_moves.md +++ b/doc/api/group_repository_storage_moves.md @@ -194,6 +194,11 @@ Example response: ## Schedule a repository storage move for a group +Schedules a repository storage move for a group. This endpoint: + +- Moves only group Wiki repositories. +- Doesn't move repositories for projects in a group. To schedule project moves, use the [Project repository storage moves](project_repository_storage_moves.md) API. + ```plaintext POST /groups/:group_id/repository_storage_moves ``` diff --git a/doc/api/groups.md b/doc/api/groups.md index 9cbd12e664c..d3c0d659d08 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -62,7 +62,8 @@ GET /groups "full_path": "foo-bar", "file_template_project_id": 1, "parent_id": null, - "created_at": "2020-01-15T12:36:29.590Z" + "created_at": "2020-01-15T12:36:29.590Z", + "ip_restriction_ranges": null } ] ``` @@ -684,7 +685,8 @@ Example response: } ] } - ] + ], + "ip_restriction_ranges": null } ``` @@ -874,6 +876,50 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/groups/4/projects/56" ``` +## Get groups to which a user can transfer a group + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371117) in GitLab 15.4 + +Retrieve a list of groups to which the user can transfer a group. + +```plaintext +GET /groups/:id/transfer_locations +``` + +| Attribute | Type | Required | Description | +|-------------|----------------|------------------------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the group to be transferred](index.md#namespaced-path-encoding). | +| `search` | string | **{dotted-circle}** No | The group names to search for. | + +Example request: + +```shell +curl --request GET "https://gitlab.example.com/api/v4/groups/1/transfer_locations" +``` + +Example response: + +```json +[ + { + "id": 27, + "web_url": "https://gitlab.example.com/groups/gitlab", + "name": "GitLab", + "avatar_url": null, + "full_name": "GitLab", + "full_path": "GitLab" + }, + { + "id": 31, + "web_url": "https://gitlab.example.com/groups/foobar", + "name": "FooBar", + "avatar_url": null, + "full_name": "FooBar", + "full_path": "FooBar" + } +] +``` + ## Transfer a group to a new parent group / Turn a subgroup to a top-level group > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23831) in GitLab 14.6. @@ -882,7 +928,7 @@ Transfer a group to a new parent group or turn a subgroup to a top-level group. - With the Owner role for the group to transfer. - With permission to [create a subgroup](../user/group/subgroups/index.md#create-a-subgroup) in the new parent group if transferring a group. -- With [permission to create a top-level group](../administration/user_settings.md#prevent-users-from-creating-top-level-groups) if turning a subgroup into a top-level group. +- With [permission to create a top-level group](../administration/user_settings.md) if turning a subgroup into a top-level group. ```plaintext POST /groups/:id/transfer @@ -905,7 +951,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ > `unique_project_download_limit`, `unique_project_download_limit_interval_in_seconds`, and `unique_project_download_limit_allowlist` [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92970) in GitLab 15.3 [with a flag](../administration/feature_flags.md) named `limit_unique_project_downloads_per_namespace_user`. Disabled by default. FLAG: -On self-managed GitLab, by default `unique_project_download_limit`, `unique_project_download_limit_interval_in_seconds`, and `unique_project_download_limit_allowlist` are not available. +On self-managed GitLab, by default `unique_project_download_limit`, `unique_project_download_limit_interval_in_seconds`, `unique_project_download_limit_allowlist` and `auto_ban_user_on_excessive_projects_download` are not available. To make them available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `limit_unique_project_downloads_per_namespace_user`. @@ -944,6 +990,8 @@ PUT /groups/:id | `unique_project_download_limit` **(ULTIMATE)** | integer | no | Maximum number of unique projects a user can download in the specified time period before they are banned. Available only on top-level groups. Default: 0, Maximum: 10,000. | | `unique_project_download_limit_interval_in_seconds` **(ULTIMATE)** | integer | no | Time period during which a user can download a maximum amount of projects before they are banned. Available only on top-level groups. Default: 0, Maximum: 864,000 seconds (10 days). | | `unique_project_download_limit_allowlist` **(ULTIMATE)** | array of strings | no | List of usernames excluded from the unique project download limit. Available only on top-level groups. Default: `[]`, Maximum: 100 usernames. | +| `auto_ban_user_on_excessive_projects_download` **(ULTIMATE)** | boolean | no | When enabled, users are automatically banned from the group when they download more than the maximum number of unique projects specified by `unique_project_download_limit` and `unique_project_download_limit_interval_in_seconds`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/94159) in GitLab 15.4. | +| `ip_restriction_ranges` **(PREMIUM)** | string | no | Comma-separated list of IP addresses or subnet masks to restrict group access. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/351493) in GitLab 15.4. | NOTE: The `projects` and `shared_projects` attributes in the response are deprecated and [scheduled for removal in API v5](https://gitlab.com/gitlab-org/gitlab/-/issues/213797). @@ -1019,7 +1067,8 @@ Example response: "shared_with_groups": [], "request_access_enabled": false } - ] + ], + "ip_restriction_ranges": null } ``` @@ -1064,14 +1113,32 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab --form "avatar=@/tmp/example.png" ``` +### Remove a group avatar + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96421) in GitLab 15.4. + +To remove a group avatar, use a blank value for the `avatar` attribute. + +Example request: + +```shell +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/22" \ + --data "avatar=" +``` + ## Remove group +> - Immediately deleting subgroups was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360008) in GitLab 15.3 [with a flag](../administration/feature_flags.md) named `immediate_delete_subgroup_api`. Disabled by default. +> - Immediately deleting subgroups was [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/368276) in GitLab 15.4. +> - Immediately deleting subgroups was [enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/368276) by default in GitLab 15.4. + Only available to group owners and administrators. -This endpoint either: +This endpoint: -- Removes group, and queues a background job to delete all projects in the group as well. -- Since [GitLab 12.8](https://gitlab.com/gitlab-org/gitlab/-/issues/33257), on [Premium](https://about.gitlab.com/pricing/) or higher tiers, marks a group for deletion. The deletion happens 7 days later by default, but this can be changed in the [instance settings](../user/admin_area/settings/visibility_and_access_controls.md#deletion-protection). +- On Premium and higher tiers, marks the group for deletion. The deletion happens 7 days later by default, but you can change the retention period in the [instance settings](../user/admin_area/settings/visibility_and_access_controls.md#deletion-protection). +- On Free tier, removes the group immediately and queues a background job to delete all projects in the group. +- Deletes a subgroup immediately if the subgroup is marked for deletion (GitLab 15.4 and later). The endpoint does not immediately delete top-level groups. ```plaintext DELETE /groups/:id @@ -1079,9 +1146,11 @@ DELETE /groups/:id Parameters: -| Attribute | Type | Required | Description | -| --------------- | -------------- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | +| Attribute | Type | Required | Description | +|----------------------|------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | +| `permanently_remove` **(PREMIUM)** | boolean/string | no | Immediately deletes a subgroup if it is marked for deletion. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368276) in GitLab 15.4 | +| `full_path` **(PREMIUM)** | string | no | Full path of subgroup to use with `permanently_remove`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368276) in GitLab 15.4. To find the subgroup path, see the [group details](groups.md#details-of-a-group) | The response is `202 Accepted` if the user has authorization. @@ -1232,6 +1301,7 @@ GET /groups/:id/hooks/:hook_id "url": "http://example.com/hook", "group_id": 3, "push_events": true, + "push_events_branch_filter": "", "issues_events": true, "confidential_issues_events": true, "merge_requests_events": true, @@ -1258,10 +1328,11 @@ POST /groups/:id/hooks ``` | Attribute | Type | Required | Description | -| -----------------------------| -------------- | ---------| ----------- | +| -----------------------------| -------------- |----------| ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | | `url` | string | yes | The hook URL | | `push_events` | boolean | no | Trigger hook on push events | +| `push_events_branch_filter` | string | No | Trigger hook on push events for matching branches only. | | `issues_events` | boolean | no | Trigger hook on issues events | | `confidential_issues_events` | boolean | no | Trigger hook on confidential issues events | | `merge_requests_events` | boolean | no | Trigger hook on merge requests events | @@ -1291,6 +1362,7 @@ PUT /groups/:id/hooks/:hook_id | `hook_id` | integer | yes | The ID of the group hook | | `url` | string | yes | The hook URL | | `push_events` | boolean | no | Trigger hook on push events | +| `push_events_branch_filter` | string | No | Trigger hook on push events for matching branches only. | | `issues_events` | boolean | no | Trigger hook on issues events | | `confidential_issues_events` | boolean | no | Trigger hook on confidential issues events | | `merge_requests_events` | boolean | no | Trigger hook on merge requests events | @@ -1369,7 +1441,7 @@ POST /groups/:id/ldap_group_links | `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | | `cn` | string | no | The CN of an LDAP group | | `filter` | string | no | The LDAP filter for the group | -| `group_access` | integer | yes | Minimum [access level](members.md#valid-access-levels) for members of the LDAP group | +| `group_access` | integer | yes | [Access level](members.md#valid-access-levels) for members of the LDAP group | | `provider` | string | yes | LDAP provider for the LDAP group link | NOTE: @@ -1420,7 +1492,8 @@ To delete the LDAP group link, provide either a `cn` or a `filter`, but not both ## SAML Group Links **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/290367) in GitLab 15.3. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/290367) in GitLab 15.3.0. +> - `access_level` type [changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95607) from `string` to `integer` in GitLab 15.3.3. List, get, add, and delete SAML group links. @@ -1438,13 +1511,12 @@ Supported attributes: |:----------|:---------------|:---------|:-------------------------------------------------------------------------| | `id` | integer/string | yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | -If successful, returns [`200`](index.md#status-codes) and the following -response attributes: +If successful, returns [`200`](index.md#status-codes) and the following response attributes: -| Attribute | Type | Description | -|:-------------------|:-------|:-------------------------------------------------------------------------------------| -| `[].name` | string | Name of the SAML group | -| `[].access_level` | integer | Minimum [access level](members.md#valid-access-levels) for members of the SAML group | +| Attribute | Type | Description | +|:-------------------|:--------|:-----------------------------------------------------------------------------| +| `[].name` | string | Name of the SAML group | +| `[].access_level` | integer | [Access level](members.md#valid-access-levels) for members of the SAML group. The attribute had a string type from GitLab 15.3.0 to GitLab 15.3.3 | Example request: @@ -1482,13 +1554,12 @@ Supported attributes: | `id` | integer/string | yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | | `saml_group_name` | string | yes | Name of an SAML group | -If successful, returns [`200`](index.md#status-codes) and the following -response attributes: +If successful, returns [`200`](index.md#status-codes) and the following response attributes: -| Attribute | Type | Description | -|:---------------|:-------|:-------------------------------------------------------------------------------------| -| `name` | string | Name of the SAML group | -| `access_level` | integer | Minimum [access level](members.md#valid-access-levels) for members of the SAML group | +| Attribute | Type | Description | +|:---------------|:--------|:-----------------------------------------------------------------------------| +| `name` | string | Name of the SAML group | +| `access_level` | integer | [Access level](members.md#valid-access-levels) for members of the SAML group. The attribute had a string type from GitLab 15.3.0 to GitLab 15.3.3 | Example request: @@ -1515,19 +1586,18 @@ POST /groups/:id/saml_group_links Supported attributes: -| Attribute | Type | Required | Description | -|:-------------------|:---------------|:---------|:-------------------------------------------------------------------------------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | -| `saml_group_name` | string | yes | Name of a SAML group | -| `access_level` | integer | yes | Minimum [access level](members.md#valid-access-levels) for members of the SAML group | +| Attribute | Type | Required | Description | +|:-------------------|:---------------|:---------|:-----------------------------------------------------------------------------| +| `id` | integer or string | yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | +| `saml_group_name` | string | yes | Name of a SAML group | +| `access_level` | integer | yes | [Access level](members.md#valid-access-levels) for members of the SAML group | -If successful, returns [`201`](index.md#status-codes) and the following -response attributes: +If successful, returns [`201`](index.md#status-codes) and the following response attributes: -| Attribute | Type | Description | -|:---------------|:-------|:-------------------------------------------------------------------------------------| -| `name` | string | Name of the SAML group | -| `access_level` | integer | Minimum [access level](members.md#valid-access-levels) for members of the SAML group | +| Attribute | Type | Description | +|:---------------|:--------|:-----------------------------------------------------------------------------| +| `name` | string | Name of the SAML group | +| `access_level` | integer | [Access level](members.md#valid-access-levels) for members of the for members of the SAML group. The attribute had a string type from GitLab 15.3.0 to GitLab 15.3.3 | Example request: @@ -1557,7 +1627,7 @@ Supported attributes: | Attribute | Type | Required | Description | |:-------------------|:---------------|:---------|:-------------------------------------------------------------------------| | `id` | integer/string | yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | -| `saml_group_name` | string | yes | Name of an SAML group | +| `saml_group_name` | string | yes | Name of a SAML group | If successful, returns [`204`](index.md#status-codes) status code without any response body. diff --git a/doc/api/integrations.md b/doc/api/integrations.md index 7912f3f6bf7..b7e110a7eef 100644 --- a/doc/api/integrations.md +++ b/doc/api/integrations.md @@ -691,6 +691,36 @@ Get Confluence integration settings for a project. GET /projects/:id/integrations/confluence ``` +## Shimo integration + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343386) in GitLab 14.5 [with a flag](../administration/feature_flags.md) named `shimo_integration`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/343386) in GitLab 15.4. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/343386) in GitLab 15.4. [Feature flag `shimo_integration`](https://gitlab.com/gitlab-org/gitlab/-/issues/345356) removed. + +Replaces the link to the internal wiki with a link to a Shimo Workspace. + +### Create/Edit Shimo integration + +Set Shimo integration for a project. + +```plaintext +PUT /projects/:id/integrations/shimo +``` + +Parameters: + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `external_wiki_url` | string | true | Shimo Workspace URL | + +### Disable Shimo integration + +Disable the Shimo integration for a project. Integration settings are preserved. + +```plaintext +DELETE /projects/:id/integrations/shimo +``` + ## External wiki Replaces the link to the internal wiki with a link to an external wiki. diff --git a/doc/api/issues.md b/doc/api/issues.md index e2c9fbd878d..6e8aaa3d489 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -294,7 +294,7 @@ GET /groups/:id/issues?state=opened | `created_before` | datetime | no | Return issues created on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | | `due_date` | string | no | Return issues that have no due date, are overdue, or whose due date is this week, this month, or between two weeks ago and next month. Accepts: `0` (no due date), `any`, `today`, `tomorrow`, `overdue`, `week`, `month`, `next_month_and_previous_two_weeks`. | | `epic_id` **(PREMIUM)** | integer | no | Return issues associated with the given epic ID. `None` returns issues that are not associated with an epic. `Any` returns issues that are associated with an epic. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/46887) in GitLab 13.6)_ -| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user | | `iids[]` | integer array | no | Return only the issues having the given `iid` | | `issue_type` | string | no | Filter to a given type of issue. One of `issue`, `incident`, or `test_case`. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/260375) in GitLab 13.12)_ | | `iteration_id` **(PREMIUM)** | integer | no | Return issues assigned to the given iteration ID. `None` returns issues that do not belong to an iteration. `Any` returns issues that belong to an iteration. Mutually exclusive with `iteration_title`. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118742) in GitLab 13.6)_ | @@ -498,7 +498,7 @@ GET /projects/:id/issues?state=opened | `created_before` | datetime | no | Return issues created on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | | `due_date` | string | no | Return issues that have no due date, are overdue, or whose due date is this week, this month, or between two weeks ago and next month. Accepts: `0` (no due date), `any`, `today`, `tomorrow`, `overdue`, `week`, `month`, `next_month_and_previous_two_weeks`. | | `epic_id` **(PREMIUM)** | integer | no | Return issues associated with the given epic ID. `None` returns issues that are not associated with an epic. `Any` returns issues that are associated with an epic. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/46887) in GitLab 13.6)_ -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `iids[]` | integer array | no | Return only the issues having the given `iid` | | `issue_type` | string | no | Filter to a given type of issue. One of `issue`, `incident`, or `test_case`. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/260375) in GitLab 13.12)_ | | `iteration_id` **(PREMIUM)** | integer | no | Return issues assigned to the given iteration ID. `None` returns issues that do not belong to an iteration. `Any` returns issues that belong to an iteration. Mutually exclusive with `iteration_title`. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118742) in GitLab 13.6)_ | @@ -845,7 +845,7 @@ GET /projects/:id/issues/:issue_iid | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -1011,7 +1011,7 @@ POST /projects/:id/issues | `due_date` | string | no | The due date. Date time string in the format `YYYY-MM-DD`, for example `2016-03-11` | | `epic_id` **(PREMIUM)** | integer | no | ID of the epic to add the issue to. Valid values are greater than or equal to 0. | | `epic_iid` **(PREMIUM)** | integer | no | IID of the epic to add the issue to. Valid values are greater than or equal to 0. (deprecated, [scheduled for removal](https://gitlab.com/gitlab-org/gitlab/-/issues/35157) in API version 5) | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `iid` | integer/string | no | The internal ID of the project's issue (requires administrator or project owner rights) | | `issue_type` | string | no | The type of issue. One of `issue`, `incident`, or `test_case`. Default is `issue`. | | `labels` | string | no | Comma-separated label names for an issue | @@ -1179,7 +1179,7 @@ PUT /projects/:id/issues/:issue_iid | `due_date` | string | no | The due date. Date time string in the format `YYYY-MM-DD`, for example `2016-03-11` | | `epic_id` **(PREMIUM)** | integer | no | ID of the epic to add the issue to. Valid values are greater than or equal to 0. | | `epic_iid` **(PREMIUM)** | integer | no | IID of the epic to add the issue to. Valid values are greater than or equal to 0. (deprecated, [scheduled for removal](https://gitlab.com/gitlab-org/gitlab/-/issues/35157) in API version 5) | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | | `issue_type` | string | no | Updates the type of issue. One of `issue`, `incident`, or `test_case`. | | `labels` | string | no | Comma-separated label names for an issue. Set to an empty string to unassign all labels. | @@ -1325,7 +1325,7 @@ DELETE /projects/:id/issues/:issue_iid | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -1344,10 +1344,10 @@ PUT /projects/:id/issues/:issue_iid/reorder | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of the project's issue | -| `move_after_id` | integer | no | The ID of a project's issue that should be placed after this issue | -| `move_before_id` | integer | no | The ID of a project's issue that should be placed before this issue | +| `move_after_id` | integer | no | The global ID of a project's issue that should be placed after this issue | +| `move_before_id` | integer | no | The global ID of a project's issue that should be placed before this issue | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/4/issues/85/reorder?move_after_id=51&move_before_id=92" @@ -1368,7 +1368,7 @@ POST /projects/:id/issues/:issue_iid/move | Attribute | Type | Required | Description | |-----------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | | `to_project_id` | integer | yes | The ID of the new project | @@ -1621,7 +1621,7 @@ POST /projects/:id/issues/:issue_iid/subscribe | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -1764,7 +1764,7 @@ POST /projects/:id/issues/:issue_iid/unsubscribe | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -1838,7 +1838,7 @@ POST /projects/:id/issues/:issue_iid/todo | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -1959,7 +1959,7 @@ Supported attributes: | Attribute | Type | Required | Description | | :---------- | :------------- | :------- | :---------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | | `body` | String | yes | The content of a note. Must contain `/promote` at the start of a new line. | @@ -2010,7 +2010,7 @@ POST /projects/:id/issues/:issue_iid/time_estimate | Attribute | Type | Required | Description | |-------------|---------|----------|------------------------------------------| | `duration` | string | yes | The duration in human format. e.g: 3h30m | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -2038,7 +2038,7 @@ POST /projects/:id/issues/:issue_iid/reset_time_estimate | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -2067,7 +2067,7 @@ POST /projects/:id/issues/:issue_iid/add_spent_time | Attribute | Type | Required | Description | |-------------|---------|----------|------------------------------------------| | `duration` | string | yes | The duration in human format. e.g: 3h30m | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | | `summary` | string | no | A summary of how the time was spent | @@ -2096,7 +2096,7 @@ POST /projects/:id/issues/:issue_iid/reset_spent_time | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -2125,7 +2125,7 @@ GET /projects/:id/issues/:issue_iid/time_stats | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -2156,7 +2156,7 @@ GET /projects/:id/issues/:issue_iid/related_merge_requests | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -2316,7 +2316,7 @@ GET /projects/:id/issues/:issue_iid/closed_by | Attribute | Type | Required | Description | | ----------- | ---------------| -------- | ---------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project issue | ```shell @@ -2393,7 +2393,7 @@ GET /projects/:id/issues/:issue_iid/participants | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -2437,7 +2437,7 @@ GET /projects/:id/issues/:issue_iid/user_agent_detail | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -2469,7 +2469,7 @@ POST /projects/:id/issues/:issue_iid/metric_images | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | | `file` | file | yes | The image file to be uploaded | | `url` | string | no | The URL to view more metric information | @@ -2503,7 +2503,7 @@ GET /projects/:id/issues/:issue_iid/metric_images | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | ```shell @@ -2541,7 +2541,7 @@ PUT /projects/:id/issues/:issue_iid/metric_images/:image_id | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | | `image_id` | integer | yes | The ID of the image | | `url` | string | no | The URL to view more metric information | @@ -2574,7 +2574,7 @@ DELETE /projects/:id/issues/:issue_iid/metric_images/:image_id | Attribute | Type | Required | Description | |-------------|---------|----------|--------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The global ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `issue_iid` | integer | yes | The internal ID of a project's issue | | `image_id` | integer | yes | The ID of the image | diff --git a/doc/api/job_artifacts.md b/doc/api/job_artifacts.md index 31da0638d23..d1bd40b91b9 100644 --- a/doc/api/job_artifacts.md +++ b/doc/api/job_artifacts.md @@ -70,7 +70,7 @@ is the same as [getting the job's artifacts](#get-job-artifacts), but by defining the job's name instead of its ID. NOTE: -If a pipeline is [parent of other child pipelines](../ci/pipelines/parent_child_pipelines.md), artifacts +If a pipeline is [parent of other child pipelines](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines), artifacts are searched in hierarchical order from parent to child. For example, if both parent and child pipelines have a job with the same name, the artifact from the parent pipeline is returned. @@ -175,7 +175,7 @@ The artifact file provides more detail than what is available in the [CSV export](../user/application_security/vulnerability_report/index.md#export-vulnerability-details). In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/201784) and later, artifacts -for [parent and child pipelines](../ci/pipelines/parent_child_pipelines.md) are searched in hierarchical +for [parent and child pipelines](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines) are searched in hierarchical order from parent to child. For example, if both parent and child pipelines have a job with the same name, the artifact from the parent pipeline is returned. diff --git a/doc/api/jobs.md b/doc/api/jobs.md index 647f8eafa62..1548045d7c2 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -303,7 +303,7 @@ Example of response ``` In GitLab 13.3 and later, this endpoint [returns data for any pipeline](pipelines.md#get-a-single-pipeline) -including [child pipelines](../ci/pipelines/parent_child_pipelines.md). +including [child pipelines](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines). In GitLab 13.5 and later, this endpoint does not return retried jobs in the response by default. Additionally, jobs are sorted by ID in descending order (newest first). @@ -368,6 +368,9 @@ Example of response "status": "pending", "tag": false, "web_url": "https://example.com/foo/bar/-/jobs/7", + "project": { + "ci_job_token_scope_enabled": false + }, "user": { "id": 1, "name": "Administrator", @@ -454,6 +457,9 @@ Example of response "failure_reason": "script_failure", "tag": false, "web_url": "https://example.com/foo/bar/-/jobs/8", + "project": { + "ci_job_token_scope_enabled": false + }, "user": { "id": 1, "name": "Administrator", @@ -611,6 +617,9 @@ Example of response "status": "failed", "tag": false, "web_url": "https://example.com/foo/bar/-/jobs/8", + "project": { + "ci_job_token_scope_enabled": false + }, "user": { "id": 1, "name": "Administrator", @@ -701,6 +710,9 @@ Example of response "status": "canceled", "tag": false, "web_url": "https://example.com/foo/bar/-/jobs/1", + "project": { + "ci_job_token_scope_enabled": false + }, "user": null } ``` @@ -751,6 +763,9 @@ Example of response "status": "pending", "tag": false, "web_url": "https://example.com/foo/bar/-/jobs/1", + "project": { + "ci_job_token_scope_enabled": false + }, "user": null } ``` @@ -806,6 +821,9 @@ Example of response "status": "failed", "tag": false, "web_url": "https://example.com/foo/bar/-/jobs/1", + "project": { + "ci_job_token_scope_enabled": false + }, "user": null } ``` @@ -816,7 +834,7 @@ You can't delete archived jobs with the API, but you can ## Run a job -Triggers a manual action to start a job. +For a job in manual status, trigger an action to start the job. ```plaintext POST /projects/:id/jobs/:job_id/play @@ -882,6 +900,9 @@ Example response: "status": "pending", "tag": false, "web_url": "https://example.com/foo/bar/-/jobs/1", + "project": { + "ci_job_token_scope_enabled": false + }, "user": null } ``` diff --git a/doc/api/members.md b/doc/api/members.md index a9817918d0b..aff601ba33f 100644 --- a/doc/api/members.md +++ b/doc/api/members.md @@ -47,6 +47,8 @@ GET /projects/:id/members | `id` | integer/string | yes | The ID or [URL-encoded path of the project or group](index.md#namespaced-path-encoding) owned by the authenticated user | | `query` | string | no | A query string to search for members | | `user_ids` | array of integers | no | Filter the results on the given user IDs | +| `skip_users` | array of integers | no | Filter skipped users out of the results | +| `show_seat_info` | boolean | no | Show seat information for users | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/:id/members" @@ -75,8 +77,7 @@ Example response: }, "expires_at": "2012-10-22T14:13:35Z", "access_level": 30, - "group_saml_identity": null, - "membership_state": "active" + "group_saml_identity": null }, { "id": 2, @@ -101,8 +102,7 @@ Example response: "extern_uid":"ABC-1234567890", "provider": "group_saml", "saml_provider_id": 10 - }, - "membership_state": "active" + } } ] ``` @@ -133,6 +133,7 @@ GET /projects/:id/members/all | `id` | integer/string | yes | The ID or [URL-encoded path of the project or group](index.md#namespaced-path-encoding) owned by the authenticated user | | `query` | string | no | A query string to search for members | | `user_ids` | array of integers | no | Filter the results on the given user IDs | +| `show_seat_info` | boolean | no | Show seat information for users | | `state` | string | no | Filter results by member state, one of `awaiting` or `active` **(PREMIUM)** | ```shell @@ -162,8 +163,7 @@ Example response: }, "expires_at": "2012-10-22T14:13:35Z", "access_level": 30, - "group_saml_identity": null, - "membership_state": "active" + "group_saml_identity": null }, { "id": 2, @@ -188,8 +188,7 @@ Example response: "extern_uid":"ABC-1234567890", "provider": "group_saml", "saml_provider_id": 10 - }, - "membership_state": "active" + } }, { "id": 3, @@ -209,8 +208,7 @@ Example response: }, "expires_at": "2012-11-22T14:13:35Z", "access_level": 30, - "group_saml_identity": null, - "membership_state": "active" + "group_saml_identity": null } ] ``` @@ -256,8 +254,7 @@ Example response: "web_url": "http://192.168.1.8:3000/root" }, "expires_at": null, - "group_saml_identity": null, - "membership_state": "active" + "group_saml_identity": null } ``` @@ -304,8 +301,7 @@ Example response: }, "email": "john@example.com", "expires_at": null, - "group_saml_identity": null, - "membership_state": "active" + "group_saml_identity": null } ``` @@ -335,7 +331,6 @@ GET /groups/:id/billable_members | `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user | | `search` | string | no | A query string to search for group members by name, username, or public email. | | `sort` | string | no | A query string containing parameters that specify the sort attribute and order. See supported values below. | -| `include_awaiting_members` | boolean | no | Determines if awaiting members are included. | The supported values for the `sort` attribute are: @@ -369,7 +364,6 @@ Example response: "web_url": "http://192.168.1.8:3000/root", "last_activity_on": "2021-01-27", "membership_type": "group_member", - "membership_state": "active", "removable": true, "created_at": "2021-01-03T12:16:02.000Z" }, @@ -383,7 +377,6 @@ Example response: "email": "john@example.com", "last_activity_on": "2021-01-25", "membership_type": "group_member", - "membership_state": "active", "removable": true, "created_at": "2021-01-04T18:46:42.000Z" }, @@ -396,7 +389,6 @@ Example response: "web_url": "http://192.168.1.8:3000/root", "last_activity_on": "2021-01-20", "membership_type": "group_invite", - "membership_state": "awaiting", "removable": false, "created_at": "2021-01-09T07:12:31.000Z" } diff --git a/doc/api/merge_request_approvals.md b/doc/api/merge_request_approvals.md index 1d99c323946..1fcce40a5b0 100644 --- a/doc/api/merge_request_approvals.md +++ b/doc/api/merge_request_approvals.md @@ -23,16 +23,17 @@ following endpoint: GET /projects/:id/approvals ``` -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | | --------- | ------- | -------- | ------------------- | -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | ```json { "approvals_before_merge": 2, "reset_approvals_on_push": true, + "selective_code_owner_removals": false, "disable_overriding_approvers_per_merge_request": false, "merge_requests_author_approval": true, "merge_requests_disable_committers_approval": false, @@ -51,22 +52,24 @@ endpoint: POST /projects/:id/approvals ``` -**Parameters:** +Supported attributes: -| Attribute | Type | Required | Description | -| ------------------------------------------------ | ------- | -------- | --------------------------------------------------------------------------------------------------- | -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `approvals_before_merge` | integer | no | How many approvals are required before an MR can be merged. Deprecated in 12.0 in favor of Approval Rules API. | -| `reset_approvals_on_push` | boolean | no | Reset approvals on a new push | -| `disable_overriding_approvers_per_merge_request` | boolean | no | Allow or prevent overriding approvers per MR | -| `merge_requests_author_approval` | boolean | no | Allow or prevent authors from self approving merge requests; `true` means authors can self approve | -| `merge_requests_disable_committers_approval` | boolean | no | Allow or prevent committers from self approving merge requests | -| `require_password_to_approve` | boolean | no | Require approver to enter a password to authenticate before adding the approval | +| Attribute | Type | Required | Description | +| ------------------------------------------------ | ------- | -------- | -- | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approvals_before_merge` | integer | **{dotted-circle}** No | How many approvals are required before a merge request can be merged. Deprecated in GitLab 12.0 in favor of Approval Rules API. | +| `disable_overriding_approvers_per_merge_request` | boolean | **{dotted-circle}** No | Allow or prevent overriding approvers per merge request. | +| `merge_requests_author_approval` | boolean | **{dotted-circle}** No | Allow or prevent authors from self approving merge requests; `true` means authors can self approve. | +| `merge_requests_disable_committers_approval` | boolean | **{dotted-circle}** No | Allow or prevent committers from self approving merge requests. | +| `require_password_to_approve` | boolean | **{dotted-circle}** No | Require approver to enter a password to authenticate before adding the approval. | +| `reset_approvals_on_push` | boolean | **{dotted-circle}** No | Reset approvals on a new push. | +| `selective_code_owner_removals` | boolean | **{dotted-circle}** No | Reset approvals from Code Owners if their files changed. Can be enabled only if `reset_approvals_on_push` is disabled. | ```json { "approvals_before_merge": 2, "reset_approvals_on_push": true, + "selective_code_owner_removals": false, "disable_overriding_approvers_per_merge_request": false, "merge_requests_author_approval": false, "merge_requests_disable_committers_approval": false, @@ -76,9 +79,7 @@ POST /projects/:id/approvals ### Get project-level rules -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11877) in GitLab 12.3. > - Moved to GitLab Premium in 13.9. -> - `protected_branches` property was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/460) in GitLab 12.7. > - Pagination support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31011) in GitLab 15.3 [with a flag](../administration/feature_flags.md) named `approval_rules_pagination`. Enabled by default. > - `applies_to_all_protected_branches` property was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335316) in GitLab 15.3. @@ -90,11 +91,11 @@ GET /projects/:id/approval_rules Use the `page` and `per_page` [pagination](index.md#offset-based-pagination) parameters to restrict the list of approval rules. -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |----------------------|---------|----------|-----------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | ```json [ @@ -191,12 +192,12 @@ You can request information about a single project approval rules using the foll GET /projects/:id/approval_rules/:approval_rule_id ``` -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |----------------------|---------|----------|-----------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `approval_rule_id` | integer | yes | The ID of a approval rule | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approval_rule_id` | integer | **{check-circle}** Yes | The ID of a approval rule. | ```json { @@ -282,7 +283,6 @@ GET /projects/:id/approval_rules/:approval_rule_id ### Create project-level rule -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11877) in GitLab 12.3. > - Moved to GitLab Premium in 13.9. > - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357300) the Vulnerability-Check feature in GitLab 15.0. > - `applies_to_all_protected_branches` property was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335316) in GitLab 15.3. @@ -293,19 +293,19 @@ You can create project approval rules using the following endpoint: POST /projects/:id/approval_rules ``` -**Parameters:** +Supported attributes: -| Attribute | Type | Required | Description | -|-------------------------------------|-------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `name` | string | yes | The name of the approval rule | -| `report_type` | string | no | The report type required when the rule type is `report_approver`. The supported report types are: `license_scanning` and `code_coverage`. | -| `approvals_required` | integer | yes | The number of required approvals for this rule | -| `rule_type` | string | no | The type of rule. `any_approver` is a pre-configured default rule with `approvals_required` at `0`. Other rules are `regular`. | -| `user_ids` | Array | no | The ids of users as approvers | -| `group_ids` | Array | no | The ids of groups as approvers | -| `protected_branch_ids` | Array | no | The IDs of protected branches to scope the rule by. To identify the ID, [use the API](protected_branches.md#list-protected-branches). | -| `applies_to_all_protected_branches` | boolean | no | Whether the rule is applied to all protected branches. If set to `true`, the value of `protected_branch_ids` is ignored. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335316) in GitLab 15.3. | +| Attribute | Type | Required | Description | +|-------------------------------------|-------------------|----------|------------ | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approvals_required` | integer | **{check-circle}** Yes | The number of required approvals for this rule. | +| `name` | string | **{check-circle}** Yes | The name of the approval rule. | +| `applies_to_all_protected_branches` | boolean | **{dotted-circle}** No | Whether the rule is applied to all protected branches. If set to `true`, the value of `protected_branch_ids` is ignored. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335316) in GitLab 15.3. | +| `group_ids` | Array | **{dotted-circle}** No | The IDs of groups as approvers. | +| `protected_branch_ids` | Array | **{dotted-circle}** No | The IDs of protected branches to scope the rule by. To identify the ID, [use the API](protected_branches.md#list-protected-branches). | +| `report_type` | string | **{dotted-circle}** No | The report type required when the rule type is `report_approver`. The supported report types are `license_scanning` and `code_coverage`. | +| `rule_type` | string | **{dotted-circle}** No | The type of rule. `any_approver` is a pre-configured default rule with `approvals_required` at `0`. Other rules are `regular`. | +| `user_ids` | Array | **{dotted-circle}** No | The IDs of users as approvers. | ```json { @@ -408,7 +408,6 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ ### Update project-level rule -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11877) in GitLab 12.3. > - Moved to GitLab Premium in 13.9. > - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357300) the Vulnerability-Check feature in GitLab 15.0. > - `applies_to_all_protected_branches` property was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335316) in GitLab 15.3. @@ -421,18 +420,18 @@ PUT /projects/:id/approval_rules/:approval_rule_id **Important:** Approvers and groups not in the `users`/`groups` parameters are **removed** -**Parameters:** +Supported attributes: -| Attribute | Type | Required | Description | -|-------------------------------------|-------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `approval_rule_id` | integer | yes | The ID of a approval rule | -| `name` | string | yes | The name of the approval rule | -| `approvals_required` | integer | yes | The number of required approvals for this rule | -| `user_ids` | Array | no | The ids of users as approvers | -| `group_ids` | Array | no | The ids of groups as approvers | -| `protected_branch_ids` | Array | no | The IDs of protected branches to scope the rule by. To identify the ID, [use the API](protected_branches.md#list-protected-branches). | -| `applies_to_all_protected_branches` | boolean | no | Whether the rule is applied to all protected branches. If set to `true`, the value of `protected_branch_ids` is ignored. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335316) in GitLab 15.3. | +| Attribute | Type | Required | Description | +|-------------------------------------|-------------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approvals_required` | integer | **{check-circle}** Yes | The number of required approvals for this rule. | +| `approval_rule_id` | integer | **{check-circle}** Yes | The ID of a approval rule. | +| `name` | string | **{check-circle}** Yes | The name of the approval rule. | +| `applies_to_all_protected_branches` | boolean | **{dotted-circle}** No | Whether the rule is applied to all protected branches. If set to `true`, the value of `protected_branch_ids` is ignored. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335316) in GitLab 15.3. | +| `group_ids` | Array | **{dotted-circle}** No | The IDs of groups as approvers. | +| `protected_branch_ids` | Array | **{dotted-circle}** No | The IDs of protected branches to scope the rule by. To identify the ID, [use the API](protected_branches.md#list-protected-branches). | +| `user_ids` | Array | **{dotted-circle}** No | The IDs of users as approvers. | ```json { @@ -518,8 +517,7 @@ PUT /projects/:id/approval_rules/:approval_rule_id ### Delete project-level rule -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11877) in GitLab 12.3. -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. You can delete project approval rules using the following endpoint: @@ -527,12 +525,12 @@ You can delete project approval rules using the following endpoint: DELETE /projects/:id/approval_rules/:approval_rule_id ``` -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |--------------------|-------------------|----------|------------------------------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `approval_rule_id` | integer | yes | The ID of a approval rule | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approval_rule_id` | integer | **{check-circle}** Yes | The ID of a approval rule. | ## Merge request-level MR approvals @@ -549,12 +547,12 @@ following endpoint: GET /projects/:id/merge_requests/:merge_request_iid/approvals ``` -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |---------------------|-------------------|----------|------------------------------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of MR | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of the merge request. | ```json { @@ -595,13 +593,13 @@ endpoint: POST /projects/:id/merge_requests/:merge_request_iid/approvals ``` -**Parameters:** +Supported attributes: -| Attribute | Type | Required | Description | -|----------------------|-------------------|----------|------------------------------------------------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of MR | -| `approvals_required` | integer | yes | Approvals required before MR can be merged. Deprecated in 12.0 in favor of Approval Rules API. | +| Attribute | Type | Required | Description | +|----------------------|-------------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approvals_required` | integer | **{check-circle}** Yes | Approvals required before MR can be merged. Deprecated in GitLab 12.0 in favor of Approval Rules API. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of the merge request. | ```json { @@ -622,8 +620,7 @@ POST /projects/:id/merge_requests/:merge_request_iid/approvals ### Get the approval state of merge requests -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13712) in GitLab 12.3. -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. You can request information about a merge request's approval state by using the following endpoint: @@ -637,12 +634,12 @@ are created for the merge request. If there are none, it is `false`. This includes additional information about the users who have already approved (`approved_by`) and whether a rule is already approved (`approved`). -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |---------------------|-------------------|----------|------------------------------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of MR | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of the merge request. | ```json { @@ -695,7 +692,6 @@ This includes additional information about the users who have already approved ### Get merge request level rules -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13712) in GitLab 12.3. > - Moved to GitLab Premium in 13.9. > - Pagination support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31011) in GitLab 15.3 [with a flag](../administration/feature_flags.md) named `approval_rules_pagination`. Enabled by default. @@ -707,12 +703,12 @@ GET /projects/:id/merge_requests/:merge_request_iid/approval_rules Use the `page` and `per_page` [pagination](index.md#offset-based-pagination) parameters to restrict the list of approval rules. -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |---------------------|---------|----------|---------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of MR | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of the merge request. | ```json [ @@ -784,13 +780,13 @@ You can request information about a single merge request approval rule using the GET /projects/:id/merge_requests/:merge_request_iid/approval_rules/:approval_rule_id ``` -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |---------------------|---------|----------|------------------------------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | -| `merge_request_iid` | integer | yes | The IID of a merge request. | -| `approval_rule_id` | integer | yes | The ID of an approval rule. | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approval_rule_id` | integer | **{check-circle}** Yes | The ID of an approval rule. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of a merge request. | ```json { @@ -852,8 +848,7 @@ GET /projects/:id/merge_requests/:merge_request_iid/approval_rules/:approval_rul ### Create merge request level rule -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11877) in GitLab 12.3. -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. You can create merge request approval rules using the following endpoint: @@ -861,17 +856,17 @@ You can create merge request approval rules using the following endpoint: POST /projects/:id/merge_requests/:merge_request_iid/approval_rules ``` -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |----------------------------|---------|----------|------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of MR | -| `name` | string | yes | The name of the approval rule | -| `approvals_required` | integer | yes | The number of required approvals for this rule | -| `approval_project_rule_id` | integer | no | The ID of a project-level approval rule | -| `user_ids` | Array | no | The ids of users as approvers | -| `group_ids` | Array | no | The ids of groups as approvers | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | +| `approvals_required` | integer | **{check-circle}** Yes | The number of required approvals for this rule. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of the merge request. | +| `name` | string | **{check-circle}** Yes | The name of the approval rule. | +| `approval_project_rule_id` | integer | **{dotted-circle}** No | The ID of a project-level approval rule. | +| `group_ids` | Array | **{dotted-circle}** No | The IDs of groups as approvers. | +| `user_ids` | Array | **{dotted-circle}** No | The IDs of users as approvers. | **Important:** When `approval_project_rule_id` is set, the `name`, `users` and `groups` of project-level rule are copied. The `approvals_required` specified @@ -937,8 +932,7 @@ is used. ### Update merge request level rule -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11877) in GitLab 12.3. -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. You can update merge request approval rules using the following endpoint: @@ -951,17 +945,17 @@ PUT /projects/:id/merge_requests/:merge_request_iid/approval_rules/:approval_rul **Important:** Updating a `report_approver` or `code_owner` rule is not allowed. These are system generated rules. -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |----------------------|---------|----------|------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | -| `merge_request_iid` | integer | yes | The IID of a merge request. | -| `approval_rule_id` | integer | yes | The ID of an approval rule. | -| `name` | string | yes | The name of the approval rule. | -| `approvals_required` | integer | yes | The number of required approvals for this rule. | -| `user_ids` | Array | no | The IDs of users as approvers. | -| `group_ids` | Array | no | The IDs of groups as approvers. | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approvals_required` | integer | **{check-circle}** Yes | The number of required approvals for this rule. | +| `approval_rule_id` | integer | **{check-circle}** Yes | The ID of an approval rule. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of a merge request. | +| `name` | string | **{check-circle}** Yes | The name of the approval rule. | +| `group_ids` | Array | **{dotted-circle}** No | The IDs of groups as approvers. | +| `user_ids` | Array | **{dotted-circle}** No | The IDs of users as approvers. | ```json { @@ -1023,8 +1017,7 @@ These are system generated rules. ### Delete merge request level rule -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11877) in GitLab 12.3. -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. You can delete merge request approval rules using the following endpoint: @@ -1035,13 +1028,13 @@ DELETE /projects/:id/merge_requests/:merge_request_iid/approval_rules/:approval_ **Important:** Deleting a `report_approver` or `code_owner` rule is not allowed. These are system generated rules. -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |---------------------|---------|----------|---------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of the merge request | -| `approval_rule_id` | integer | yes | The ID of an approval rule | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approval_rule_id` | integer | **{check-circle}** Yes | The ID of an approval rule. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of the merge request. | ## Approve merge request @@ -1054,14 +1047,14 @@ endpoint: POST /projects/:id/merge_requests/:merge_request_iid/approve ``` -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |---------------------|---------|----------|-------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of the merge request | -| `sha` | string | no | The `HEAD` of the merge request | -| `approval_password` | string | no | Current user's password. Required if [**Require user password to approve**](../user/project/merge_requests/approvals/settings.md#require-user-password-to-approve) is enabled in the project settings. | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `approval_password` | string | **{dotted-circle}** No | Current user's password. Required if [**Require user password to approve**](../user/project/merge_requests/approvals/settings.md#require-user-password-to-approve) is enabled in the project settings. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of the merge request. | +| `sha` | string | **{dotted-circle}** No | The `HEAD` of the merge request. | The `sha` parameter works in the same way as when [accepting a merge request](merge_requests.md#merge-a-merge-request): if it is passed, then it must @@ -1117,9 +1110,9 @@ endpoint: POST /projects/:id/merge_requests/:merge_request_iid/unapprove ``` -**Parameters:** +Supported attributes: | Attribute | Type | Required | Description | |---------------------|---------|----------|---------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of a merge request | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). | +| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of a merge request. | diff --git a/doc/api/merge_request_context_commits.md b/doc/api/merge_request_context_commits.md index 9984c5abb70..08020e5a613 100644 --- a/doc/api/merge_request_context_commits.md +++ b/doc/api/merge_request_context_commits.md @@ -17,10 +17,10 @@ GET /projects/:id/merge_requests/:merge_request_iid/context_commits Parameters: -| Attribute | Type | Required | Description | -|---------------------|---------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | -| `merge_request_iid` | integer | yes | The internal ID of the merge request | +| Attribute | Type | Required | Description | +|---------------------|---------|----------|-------------| +| `id` | integer | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```json [ @@ -51,18 +51,18 @@ POST /projects/:id/merge_requests/:merge_request_iid/context_commits Parameters: -| Attribute | Type | Required | Description | -|---------------------|---------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | -| `merge_request_iid` | integer | yes | The internal ID of the merge request | +| Attribute | Type | Required | Description | +|---------------------|---------|----------|-------------| +| `id` | integer | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```plaintext POST /projects/:id/merge_requests/ ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `commits` | string array | yes | The context commits' SHA | +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `commits` | string array | **{check-circle}** Yes | The context commits' SHA. | ```json [ @@ -92,8 +92,8 @@ DELETE /projects/:id/merge_requests/:merge_request_iid/context_commits Parameters: -| Attribute | Type | Required | Description | -|---------------------|--------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | -| `merge_request_iid` | integer | yes | The internal ID of the merge request | -| `commits` | string array | yes | The context commits' SHA | +| Attribute | Type | Required | Description | +|---------------------|--------------|----------|--------------| +| `commits` | string array | **{check-circle}** Yes | The context commits' SHA. | +| `id` | integer | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index a491756e5f0..82125aec366 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -6,8 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Merge requests API **(FREE)** -> - `reference` was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20354) in GitLab 12.10 in favour of `references`. -> - `reviewer_username` and `reviewer_id` were [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. > - `draft` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63473) as a replacement for `work_in_progress` in GitLab 14.0. > - `merge_user` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/349031) as an eventual replacement for `merged_by` in GitLab 14.7. @@ -38,40 +36,40 @@ GET /merge_requests?scope=assigned_to_me GET /merge_requests?search=foo&in=title ``` -Parameters: - -| Attribute | Type | Required | Description | -| ------------------------------- | -------------- | -------- | ---------------------------------------------------------------------------------------------------------------------- | -| `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, `locked`, or `merged`. | -| `order_by` | string | no | Return requests ordered by `created_at`, `title`, or `updated_at` fields. Default is `created_at`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331625) in GitLab 14.8.| -| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc`. | -| `milestone` | string | no | Return merge requests for a specific milestone. `None` returns merge requests with no milestone. `Any` returns merge requests that have an assigned milestone. | -| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request. | -| `labels` | string | no | Return merge requests matching a comma-separated list of labels. `None` lists all merge requests with no labels. `Any` lists all merge requests with at least one label. Predefined names are case-insensitive. | -| `with_labels_details` | boolean | no | If `true`, response returns more details for each label in labels field: `:name`, `:color`, `:description`, `:description_html`, `:text_color`. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21413) in GitLab 12.7. | -| `with_merge_status_recheck` | boolean | no | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. | -| `created_after` | datetime | no | Return merge requests created on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `created_before` | datetime | no | Return merge requests created on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `updated_after` | datetime | no | Return merge requests updated on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `updated_before` | datetime | no | Return merge requests updated on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `scope` | string | no | Return merge requests for the given scope: `created_by_me`, `assigned_to_me` or `all`. Defaults to `created_by_me`. | -| `author_id` | integer | no | Returns merge requests created by the given user `id`. Mutually exclusive with `author_username`. Combine with `scope=all` or `scope=assigned_to_me`. | -| `author_username` | string | no | Returns merge requests created by the given `username`. Mutually exclusive with `author_id`. [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13060) in GitLab 12.10. | -| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id`. `None` returns unassigned merge requests. `Any` returns merge requests with an assignee. | -| `approver_ids` **(PREMIUM)** | integer array | no | Returns merge requests which have specified all the users with the given `id`s as individual approvers. `None` returns merge requests without approvers. `Any` returns merge requests with an approver. | -| `approved_by_ids` **(PREMIUM)** | integer array | no | Returns merge requests which have been approved by all the users with the given `id`s (Max: 5). `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | -| `reviewer_id` | integer | no | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given user `id`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_username`. | -| `reviewer_username` | string | no | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given `username`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_id`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | -| `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. | -| `source_branch` | string | no | Return merge requests with the given source branch. | -| `target_branch` | string | no | Return merge requests with the given target branch. | -| `search` | string | no | Search merge requests against their `title` and `description`. | -| `in` | string | no | Modify the scope of the `search` attribute. `title`, `description`, or a string joining them with comma. Default is `title,description`. | -| `wip` | string | no | Filter merge requests against their `wip` status. `yes` to return *only* draft merge requests, `no` to return *non-draft* merge requests. | -| `not` | Hash | no | Return merge requests that do not match the parameters supplied. Accepts: `labels`, `milestone`, `author_id`, `author_username`, `assignee_id`, `assignee_username`, `reviewer_id`, `reviewer_username`, `my_reaction_emoji`. | -| `environment` | string | no | Returns merge requests deployed to the given environment. | -| `deployed_before` | datetime | no | Return merge requests deployed before the given date/time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `deployed_after` | datetime | no | Return merge requests deployed after the given date/time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | +Supported attributes: + +| Attribute | Type | Required | Description | +| ------------------------------- | -------------- | -------- | ----------- | +| `approved_by_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have been approved by all the users with the given `id`. Maximum of 5. `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | +| `approver_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have specified all the users with the given `id` as individual approvers. `None` returns merge requests without approvers. `Any` returns merge requests with an approver. | +| `assignee_id` | integer | **{dotted-circle}** No | Returns merge requests assigned to the given user `id`. `None` returns unassigned merge requests. `Any` returns merge requests with an assignee. | +| `author_id` | integer | **{dotted-circle}** No | Returns merge requests created by the given user `id`. Mutually exclusive with `author_username`. Combine with `scope=all` or `scope=assigned_to_me`. | +| `author_username` | string | **{dotted-circle}** No | Returns merge requests created by the given `username`. Mutually exclusive with `author_id`. | +| `created_after` | datetime | **{dotted-circle}** No | Return merge requests created on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `created_before` | datetime | **{dotted-circle}** No | Return merge requests created on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `deployed_after` | datetime | **{dotted-circle}** No | Return merge requests deployed after the given date/time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `deployed_before` | datetime | **{dotted-circle}** No | Return merge requests deployed before the given date/time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `environment` | string | **{dotted-circle}** No | Returns merge requests deployed to the given environment. | +| `in` | string | **{dotted-circle}** No | Modify the scope of the `search` attribute. `title`, `description`, or a string joining them with comma. Default is `title,description`. | +| `labels` | string | **{dotted-circle}** No | Return merge requests matching a comma-separated list of labels. `None` lists all merge requests with no labels. `Any` lists all merge requests with at least one label. Predefined names are case-insensitive. | +| `milestone` | string | **{dotted-circle}** No | Return merge requests for a specific milestone. `None` returns merge requests with no milestone. `Any` returns merge requests that have an assigned milestone. | +| `my_reaction_emoji` | string | **{dotted-circle}** No | Return merge requests reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. | +| `not` | Hash | **{dotted-circle}** No | Return merge requests that do not match the parameters supplied. Accepts: `labels`, `milestone`, `author_id`, `author_username`, `assignee_id`, `assignee_username`, `reviewer_id`, `reviewer_username`, `my_reaction_emoji`. | +| `order_by` | string | **{dotted-circle}** No | Return requests ordered by `created_at`, `title`, or `updated_at` fields. Default is `created_at`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331625) in GitLab 14.8.| +| `reviewer_id` | integer | **{dotted-circle}** No | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given user `id`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_username`. | +| `reviewer_username` | string | **{dotted-circle}** No | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given `username`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_id`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | +| `scope` | string | **{dotted-circle}** No | Return merge requests for the given scope: `created_by_me`, `assigned_to_me` or `all`. Defaults to `created_by_me`. | +| `search` | string | **{dotted-circle}** No | Search merge requests against their `title` and `description`. | +| `sort` | string | **{dotted-circle}** No | Return requests sorted in `asc` or `desc` order. Default is `desc`. | +| `source_branch` | string | **{dotted-circle}** No | Return merge requests with the given source branch. | +| `state` | string | **{dotted-circle}** No | Return all merge requests or just those that are `opened`, `closed`, `locked`, or `merged`. | +| `target_branch` | string | **{dotted-circle}** No | Return merge requests with the given target branch. | +| `updated_after` | datetime | **{dotted-circle}** No | Return merge requests updated on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `updated_before` | datetime | **{dotted-circle}** No | Return merge requests updated on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `view` | string | **{dotted-circle}** No | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request. | +| `with_labels_details` | boolean | **{dotted-circle}** No | If `true`, response returns more details for each label in labels field: `:name`, `:color`, `:description`, `:description_html`, `:text_color`. Default is `false`. | +| `with_merge_status_recheck` | boolean | **{dotted-circle}** No | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. | +| `wip` | string | **{dotted-circle}** No | Filter merge requests against their `wip` status. `yes` to return *only* draft merge requests, `no` to return *non-draft* merge requests. | ```json [ @@ -239,39 +237,39 @@ are the same. In the case of a merge request from a fork, `target_project_id` and `project_id` are the same and `source_project_id` is the fork project's ID. -Parameters: - -| Attribute | Type | Required | Description | -| ------------------------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------ | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `iids[]` | integer array | no | Return the request having the given `iid`. | -| `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, `locked`, or `merged`. | -| `order_by` | string | no | Return requests ordered by `created_at`, `title` or `updated_at` fields. Default is `created_at`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331625) in GitLab 14.8. | -| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc`. | -| `milestone` | string | no | Return merge requests for a specific milestone. `None` returns merge requests with no milestone. `Any` returns merge requests that have an assigned milestone. | -| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request. | -| `labels` | string | no | Return merge requests matching a comma-separated list of labels. `None` lists all merge requests with no labels. `Any` lists all merge requests with at least one label. Predefined names are case-insensitive. | -| `with_labels_details` | boolean | no | If `true`, response returns more details for each label in labels field: `:name`, `:color`, `:description`, `:description_html`, `:text_color`. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21413) in GitLab 12.7. | -| `with_merge_status_recheck` | boolean | no | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. | -| `created_after` | datetime | no | Return merge requests created on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `created_before` | datetime | no | Return merge requests created on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `updated_after` | datetime | no | Return merge requests updated on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `updated_before` | datetime | no | Return merge requests updated on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `scope` | string | no | Return merge requests for the given scope: `created_by_me`, `assigned_to_me`, or `all`. | -| `author_id` | integer | no | Returns merge requests created by the given user `id`. Mutually exclusive with `author_username`. | -| `author_username` | string | no | Returns merge requests created by the given `username`. Mutually exclusive with `author_id`. [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13060) in GitLab 12.10. | -| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id`. `None` returns unassigned merge requests. `Any` returns merge requests with an assignee. | -| `approver_ids` **(PREMIUM)** | integer array | no | Returns merge requests which have specified all the users with the given `id`s as individual approvers. `None` returns merge requests without approvers. `Any` returns merge requests with an approver. | -| `approved_by_ids` **(PREMIUM)** | integer array | no | Returns merge requests which have been approved by all the users with the given `id`s (Max: 5). `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | -| `reviewer_id` | integer | no | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given user `id`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_username`. | -| `reviewer_username` | string | no | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given `username`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_id`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | -| `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. | -| `source_branch` | string | no | Return merge requests with the given source branch. | -| `target_branch` | string | no | Return merge requests with the given target branch. | -| `search` | string | no | Search merge requests against their `title` and `description`. | -| `wip` | string | no | Filter merge requests against their `wip` status. `yes` to return *only* draft merge requests, `no` to return *non-draft* merge requests. | -| `not` | Hash | no | Return merge requests that do not match the parameters supplied. Accepts: `labels`, `milestone`, `author_id`, `author_username`, `assignee_id`, `assignee_username`, `reviewer_id`, `reviewer_username`, `my_reaction_emoji`. | -| `environment` | string | no | Returns merge requests deployed to the given environment. | +Supported attributes: + +| Attribute | Type | Required | Description | +| ------------------------------- | -------------- | -------- | ----------- | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `approved_by_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have been approved by all the users with the given `id`, with a maximum of 5. `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | +| `approver_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have specified all the users with the given `id` as individual approvers. `None` returns merge requests without approvers. `Any` returns merge requests with an approver. | +| `assignee_id` | integer | **{dotted-circle}** No | Returns merge requests assigned to the given user `id`. `None` returns unassigned merge requests. `Any` returns merge requests with an assignee. | +| `author_id` | integer | **{dotted-circle}** No | Returns merge requests created by the given user `id`. Mutually exclusive with `author_username`. | +| `author_username` | string | **{dotted-circle}** No | Returns merge requests created by the given `username`. Mutually exclusive with `author_id`. | +| `created_after` | datetime | **{dotted-circle}** No | Return merge requests created on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `created_before` | datetime | **{dotted-circle}** No | Return merge requests created on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `environment` | string | **{dotted-circle}** No | Returns merge requests deployed to the given environment. | +| `iids[]` | integer array | **{dotted-circle}** No | Return the request having the given `iid`. | +| `labels` | string | **{dotted-circle}** No | Return merge requests matching a comma-separated list of labels. `None` lists all merge requests with no labels. `Any` lists all merge requests with at least one label. Predefined names are case-insensitive. | +| `milestone` | string | **{dotted-circle}** No | Return merge requests for a specific milestone. `None` returns merge requests with no milestone. `Any` returns merge requests that have an assigned milestone. | +| `my_reaction_emoji` | string | **{dotted-circle}** No | Return merge requests reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. | +| `not` | Hash | **{dotted-circle}** No | Return merge requests that do not match the parameters supplied. Accepts: `labels`, `milestone`, `author_id`, `author_username`, `assignee_id`, `assignee_username`, `reviewer_id`, `reviewer_username`, `my_reaction_emoji`. | +| `order_by` | string | **{dotted-circle}** No | Return requests ordered by `created_at`, `title` or `updated_at` fields. Default is `created_at`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331625) in GitLab 14.8. | +| `reviewer_id` | integer | **{dotted-circle}** No | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given user `id`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_username`. | +| `reviewer_username` | string | **{dotted-circle}** No | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given `username`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_id`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | +| `scope` | string | **{dotted-circle}** No | Return merge requests for the given scope: `created_by_me`, `assigned_to_me`, or `all`. | +| `search` | string | **{dotted-circle}** No | Search merge requests against their `title` and `description`. | +| `sort` | string | **{dotted-circle}** No | Return requests sorted in `asc` or `desc` order. Default is `desc`. | +| `source_branch` | string | **{dotted-circle}** No | Return merge requests with the given source branch. | +| `state` | string | **{dotted-circle}** No | Return all merge requests or just those that are `opened`, `closed`, `locked`, or `merged`. | +| `target_branch` | string | **{dotted-circle}** No | Return merge requests with the given target branch. | +| `updated_after` | datetime | **{dotted-circle}** No | Return merge requests updated on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `updated_before` | datetime | **{dotted-circle}** No | Return merge requests updated on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `view` | string | **{dotted-circle}** No | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request. | +| `wip` | string | **{dotted-circle}** No | Filter merge requests against their `wip` status. `yes` to return *only* draft merge requests, `no` to return *non-draft* merge requests. | +| `with_labels_details` | boolean | **{dotted-circle}** No | If `true`, response returns more details for each label in labels field: `:name`, `:color`, `:description`, `:description_html`, `:text_color`. Default is `false`. | +| `with_merge_status_recheck` | boolean | **{dotted-circle}** No | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. | ```json [ @@ -427,38 +425,38 @@ GET /groups/:id/merge_requests?my_reaction_emoji=star `group_id` represents the ID of the group which contains the project where the MR resides. -Parameters: - -| Attribute | Type | Required | Description | -| ------------------------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------ | -| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, `locked`, or `merged`. | -| `order_by` | string | no | Return merge requests ordered by `created_at`, `title` or `updated_at` fields. Default is `created_at`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331625) in GitLab 14.8. | -| `sort` | string | no | Return merge requests sorted in `asc` or `desc` order. Default is `desc`. | -| `milestone` | string | no | Return merge requests for a specific milestone. `None` returns merge requests with no milestone. `Any` returns merge requests that have an assigned milestone. | -| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request. | -| `labels` | string | no | Return merge requests matching a comma-separated list of labels. `None` lists all merge requests with no labels. `Any` lists all merge requests with at least one label. Predefined names are case-insensitive. | -| `with_labels_details` | boolean | no | If `true`, response returns more details for each label in labels field: `:name`, `:color`, `:description`, `:description_html`, `:text_color`. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21413) in GitLab 12.7. | -| `with_merge_status_recheck` | boolean | no | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. | -| `created_after` | datetime | no | Return merge requests created on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | -| `created_before` | datetime | no | Return merge requests created on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | -| `updated_after` | datetime | no | Return merge requests updated on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | -| `updated_before` | datetime | no | Return merge requests updated on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | -| `scope` | string | no | Return merge requests for the given scope: `created_by_me`, `assigned_to_me` or `all`. | -| `author_id` | integer | no | Returns merge requests created by the given user `id`. Mutually exclusive with `author_username`. | -| `author_username` | string | no | Returns merge requests created by the given `username`. Mutually exclusive with `author_id`. [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13060) in GitLab 12.10. | -| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id`. `None` returns unassigned merge requests. `Any` returns merge requests with an assignee. | -| `approver_ids` **(PREMIUM)** | integer array | no | Returns merge requests which have specified all the users with the given `id`s as individual approvers. `None` returns merge requests without approvers. `Any` returns merge requests with an approver. | -| `approved_by_ids` **(PREMIUM)** | integer array | no | Returns merge requests which have been approved by all the users with the given `id`s (Max: 5). `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | -| `approved_by_usernames` **(PREMIUM)** | string array | no | Returns merge requests which have been approved by all the users with the given `username`s (Max: 5). `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | -| `reviewer_id` | integer | no | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given user `id`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_username`. | -| `reviewer_username` | string | no | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given `username`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_id`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | -| `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. | -| `source_branch` | string | no | Return merge requests with the given source branch. | -| `target_branch` | string | no | Return merge requests with the given target branch. | -| `search` | string | no | Search merge requests against their `title` and `description`. | -| `non_archived` | boolean | no | Return merge requests from non archived projects only. Default is true. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/23809) in GitLab 12.8)_. | -| `not` | Hash | no | Return merge requests that do not match the parameters supplied. Accepts: `labels`, `milestone`, `author_id`, `author_username`, `assignee_id`, `assignee_username`, `reviewer_id`, `reviewer_username`, `my_reaction_emoji`. | +Supported attributes: + +| Attribute | Type | Required | Description | +| ------------------------------- | -------------- | -------- | ----------- | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `approved_by_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have been approved by all the users with the given `id`, with a maximum of 5. `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | +| `approved_by_usernames` **(PREMIUM)** | string array | **{dotted-circle}** No | Returns merge requests which have been approved by all the users with the given `username`, with a maximum of 5. `None` returns merge requests with no approvals. `Any` returns merge requests with an approval. | +| `approver_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | Returns merge requests which have specified all the users with the given `id`s as individual approvers. `None` returns merge requests without approvers. `Any` returns merge requests with an approver. | +| `assignee_id` | integer | **{dotted-circle}** No | Returns merge requests assigned to the given user `id`. `None` returns unassigned merge requests. `Any` returns merge requests with an assignee. | +| `author_id` | integer | **{dotted-circle}** No | Returns merge requests created by the given user `id`. Mutually exclusive with `author_username`. | +| `author_username` | string | **{dotted-circle}** No | Returns merge requests created by the given `username`. Mutually exclusive with `author_id`. | +| `created_after` | datetime | **{dotted-circle}** No | Return merge requests created on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `created_before` | datetime | **{dotted-circle}** No | Return merge requests created on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `labels` | string | **{dotted-circle}** No | Return merge requests matching a comma-separated list of labels. `None` lists all merge requests with no labels. `Any` lists all merge requests with at least one label. Predefined names are case-insensitive. | +| `milestone` | string | **{dotted-circle}** No | Return merge requests for a specific milestone. `None` returns merge requests with no milestone. `Any` returns merge requests that have an assigned milestone. | +| `my_reaction_emoji` | string | **{dotted-circle}** No | Return merge requests reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. | +| `non_archived` | boolean | **{dotted-circle}** No | Return merge requests from non archived projects only. Default is `true`. | +| `not` | Hash | **{dotted-circle}** No | Return merge requests that do not match the parameters supplied. Accepts: `labels`, `milestone`, `author_id`, `author_username`, `assignee_id`, `assignee_username`, `reviewer_id`, `reviewer_username`, `my_reaction_emoji`. | +| `order_by` | string | **{dotted-circle}** No | Return merge requests ordered by `created_at`, `title` or `updated_at` fields. Default is `created_at`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331625) in GitLab 14.8. | +| `reviewer_id` | integer | **{dotted-circle}** No | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given user `id`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_username`. | +| `reviewer_username` | string | **{dotted-circle}** No | Returns merge requests which have the user as a [reviewer](../user/project/merge_requests/getting_started.md#reviewer) with the given `username`. `None` returns merge requests with no reviewers. `Any` returns merge requests with any reviewer. Mutually exclusive with `reviewer_id`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | +| `scope` | string | **{dotted-circle}** No | Return merge requests for the given scope: `created_by_me`, `assigned_to_me` or `all`. | +| `search` | string | **{dotted-circle}** No | Search merge requests against their `title` and `description`. | +| `source_branch` | string | **{dotted-circle}** No | Return merge requests with the given source branch. | +| `sort` | string | **{dotted-circle}** No | Return merge requests sorted in `asc` or `desc` order. Default is `desc`. | +| `state` | string | **{dotted-circle}** No | Return all merge requests or just those that are `opened`, `closed`, `locked`, or `merged`. | +| `target_branch` | string | **{dotted-circle}** No | Return merge requests with the given target branch. | +| `updated_after` | datetime | **{dotted-circle}** No | Return merge requests updated on or after the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `updated_before` | datetime | **{dotted-circle}** No | Return merge requests updated on or before the given time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `view` | string | **{dotted-circle}** No | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request. | +| `with_labels_details` | boolean | **{dotted-circle}** No | If `true`, response returns more details for each label in labels field: `:name`, `:color`, `:description`, `:description_html`, `:text_color`. Default is `false`. | +| `with_merge_status_recheck` | boolean | **{dotted-circle}** No | If `true`, this projection requests (but does not guarantee) that the `merge_status` field be recalculated asynchronously. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31890) in GitLab 13.0. | ```json [ @@ -608,15 +606,15 @@ it is capped at 1,000. In that case, the API returns the string GET /projects/:id/merge_requests/:merge_request_iid ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|----------------------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `render_html` | boolean | no | If `true` response includes rendered HTML for title and description. | -| `include_diverged_commits_count` | boolean | no | If `true` response includes the commits behind the target branch. | -| `include_rebase_in_progress` | boolean | no | If `true` response includes whether a rebase operation is in progress. | +| Attribute | Type | Required | Description | +|----------------------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | +| `include_diverged_commits_count` | boolean | **{dotted-circle}** No | If `true`, response includes the commits behind the target branch. | +| `include_rebase_in_progress` | boolean | **{dotted-circle}** No | If `true`, response includes whether a rebase operation is in progress. | +| `render_html` | boolean | **{dotted-circle}** No | If `true`, response includes rendered HTML for title and description. | ```json { @@ -797,12 +795,12 @@ Get a list of merge request participants. GET /projects/:id/merge_requests/:merge_request_iid/participants ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```json [ @@ -833,12 +831,12 @@ Get a list of merge request reviewers. GET /projects/:id/merge_requests/:merge_request_iid/reviewers ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```json [ @@ -851,14 +849,6 @@ Parameters: "avatar_url": "http://www.gravatar.com/avatar/c922747a93b40d1ea88262bf1aebee62?s=80&d=identicon", "web_url": "http://localhost/user1" }, - "updated_state_by": { - "id": 1, - "name": "John Doe1", - "username": "user1", - "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/c922747a93b40d1ea88262bf1aebee62?s=80&d=identicon", - "web_url": "http://localhost/user1" - }, "state": "unreviewed", "created_at": "2022-07-27T17:03:27.684Z" }, @@ -871,14 +861,6 @@ Parameters: "avatar_url": "http://www.gravatar.com/avatar/10fc7f102be8de7657fb4d80898bbfe3?s=80&d=identicon", "web_url": "http://localhost/user2" }, - "updated_state_by": { - "id": 1, - "name": "John Doe1", - "username": "user1", - "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/c922747a93b40d1ea88262bf1aebee62?s=80&d=identicon", - "web_url": "http://localhost/user1" - }, "state": "reviewed", "created_at": "2022-07-27T17:03:27.684Z" } @@ -893,12 +875,12 @@ Get a list of merge request commits. GET /projects/:id/merge_requests/:merge_request_iid/commits ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```json [ @@ -940,13 +922,13 @@ still apply. GET /projects/:id/merge_requests/:merge_request_iid/changes ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `access_raw_diffs` | boolean | no | Retrieve change diffs via Gitaly. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | +| `access_raw_diffs` | boolean | **{dotted-circle}** No | Retrieve change diffs via Gitaly. | ```json { @@ -1062,12 +1044,12 @@ Get a list of merge request pipelines. GET /projects/:id/merge_requests/:merge_request_iid/pipelines ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```json [ @@ -1082,8 +1064,6 @@ Parameters: ## Create MR Pipeline -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31722) in GitLab 12.3. - Create a new [pipeline for a merge request](../ci/pipelines/merge_request_pipelines.md). A pipeline created via this endpoint doesn't run a regular branch/tag pipeline. It requires `.gitlab-ci.yml` to be configured with `only: [merge_requests]` to create jobs. @@ -1098,12 +1078,12 @@ The new pipeline can be: POST /projects/:id/merge_requests/:merge_request_iid/pipelines ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```json { @@ -1152,24 +1132,24 @@ Creates a new merge request. POST /projects/:id/merge_requests ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | -| `source_branch` | string | yes | The source branch. | -| `target_branch` | string | yes | The target branch. | -| `title` | string | yes | Title of MR. | -| `assignee_id` | integer | no | Assignee user ID. | -| `assignee_ids` | integer array | no | The ID of the users to assign the MR to. Set to `0` or provide an empty value to unassign all assignees. | -| `reviewer_ids` | integer array | no | The ID of the users added as a reviewer to the MR. If set to `0` or left empty, no reviewers are added. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | -| `description` | string | no | Description of MR. Limited to 1,048,576 characters. | -| `target_project_id` | integer | no | The target project (numeric ID). | -| `labels` | string | no | Labels for MR as a comma-separated list. | -| `milestone_id` | integer | no | The global ID of a milestone. | -| `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging. | -| `allow_collaboration` | boolean | no | Allow commits from members who can merge to the target branch. | -| `allow_maintainer_to_push` | boolean | no | Alias of `allow_collaboration`. | -| `approvals_before_merge` **(PREMIUM)** | integer | no | Number of approvals required before this can be merged (see below). | -| `squash` | boolean | no | Squash commits into a single commit when merging. | +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `source_branch` | string | **{check-circle}** Yes | The source branch. | +| `target_branch` | string | **{check-circle}** Yes | The target branch. | +| `title` | string | **{check-circle}** Yes | Title of MR. | +| `allow_collaboration` | boolean | **{dotted-circle}** No | Allow commits from members who can merge to the target branch. | +| `allow_maintainer_to_push` | boolean | **{dotted-circle}** No | Alias of `allow_collaboration`. | +| `approvals_before_merge` **(PREMIUM)** | integer | **{dotted-circle}** No | Number of approvals required before this can be merged (see below). | +| `assignee_id` | integer | **{dotted-circle}** No | Assignee user ID. | +| `assignee_ids` | integer array | **{dotted-circle}** No | The ID of the users to assign the MR to. Set to `0` or provide an empty value to unassign all assignees. | +| `description` | string | **{dotted-circle}** No | Description of the merge request. Limited to 1,048,576 characters. | +| `labels` | string | **{dotted-circle}** No | Labels for the merge request, as a comma-separated list. | +| `milestone_id` | integer | **{dotted-circle}** No | The global ID of a milestone. | +| `remove_source_branch` | boolean | **{dotted-circle}** No | Flag indicating if a merge request should remove the source branch when merging. | +| `reviewer_ids` | integer array | **{dotted-circle}** No | The ID of the users added as a reviewer to the merge request. If set to `0` or left empty, no reviewers are added. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | +| `squash` | boolean | **{dotted-circle}** No | Squash commits into a single commit when merging. | +| `target_project_id` | integer | **{dotted-circle}** No | Numeric ID of the target project. | If `approvals_before_merge` is not provided, it inherits the value from the target project. If provided, the following conditions must hold for it to take effect: @@ -1320,26 +1300,26 @@ Updates an existing merge request. You can change the target branch, title, or e PUT /projects/:id/merge_requests/:merge_request_iid ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The ID of a merge request. | -| `target_branch` | string | no | The target branch. | -| `title` | string | no | Title of MR. | -| `assignee_id` | integer | no | The ID of the user to assign the merge request to. Set to `0` or provide an empty value to unassign all assignees. | -| `assignee_ids` | integer array | no | The ID of the users to assign the MR to. Set to `0` or provide an empty value to unassign all assignees. | -| `reviewer_ids` | integer array | no | The ID of the users set as a reviewer to the MR. Set the value to `0` or provide an empty value to unset all reviewers. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | -| `milestone_id` | integer | no | The global ID of a milestone to assign the merge request to. Set to `0` or provide an empty value to unassign a milestone.| -| `labels` | string | no | Comma-separated label names for a merge request. Set to an empty string to unassign all labels. | -| `add_labels` | string | no | Comma-separated label names to add to a merge request. | -| `remove_labels` | string | no | Comma-separated label names to remove from a merge request. | -| `description` | string | no | Description of MR. Limited to 1,048,576 characters. | -| `state_event` | string | no | New state (close/reopen). | -| `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging. | -| `squash` | boolean | no | Squash commits into a single commit when merging. | -| `discussion_locked` | boolean | no | Flag indicating if the merge request's discussion is locked. If the discussion is locked only project members can add, edit or resolve comments. | -| `allow_collaboration` | boolean | no | Allow commits from members who can merge to the target branch. | -| `allow_maintainer_to_push` | boolean | no | Alias of `allow_collaboration`. | +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The ID of a merge request. | +| `add_labels` | string | **{dotted-circle}** No | Comma-separated label names to add to a merge request. | +| `allow_collaboration` | boolean | **{dotted-circle}** No | Allow commits from members who can merge to the target branch. | +| `allow_maintainer_to_push` | boolean | **{dotted-circle}** No | Alias of `allow_collaboration`. | +| `assignee_id` | integer | **{dotted-circle}** No | The ID of the user to assign the merge request to. Set to `0` or provide an empty value to unassign all assignees. | +| `assignee_ids` | integer array | **{dotted-circle}** No | The ID of the users to assign the merge request to. Set to `0` or provide an empty value to unassign all assignees. | +| `description` | string | **{dotted-circle}** No | Description of the merge request. Limited to 1,048,576 characters. | +| `discussion_locked` | boolean | **{dotted-circle}** No | Flag indicating if the merge request's discussion is locked. If the discussion is locked only project members can add, edit or resolve comments. | +| `labels` | string | **{dotted-circle}** No | Comma-separated label names for a merge request. Set to an empty string to unassign all labels. | +| `milestone_id` | integer | **{dotted-circle}** No | The global ID of a milestone to assign the merge request to. Set to `0` or provide an empty value to unassign a milestone.| +| `remove_labels` | string | **{dotted-circle}** No | Comma-separated label names to remove from a merge request. | +| `remove_source_branch` | boolean | **{dotted-circle}** No | Flag indicating if a merge request should remove the source branch when merging. | +| `reviewer_ids` | integer array | **{dotted-circle}** No | The ID of the users set as a reviewer to the merge request. Set the value to `0` or provide an empty value to unset all reviewers. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49341) in GitLab 13.8. | +| `squash` | boolean | **{dotted-circle}** No | Squash commits into a single commit when merging. | +| `state_event` | string | **{dotted-circle}** No | New state (close/reopen). | +| `target_branch` | string | **{dotted-circle}** No | The target branch. | +| `title` | string | **{dotted-circle}** No | Title of MR. | Must include at least one non-required attribute from above. @@ -1501,10 +1481,10 @@ Only for administrators and project owners. Deletes the merge request in questio DELETE /projects/:id/merge_requests/:merge_request_iid ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/4/merge_requests/85" @@ -1518,18 +1498,18 @@ Accept and merge changes submitted with MR using this API. PUT /projects/:id/merge_requests/:merge_request_iid/merge ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|--------------------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `merge_commit_message` | string | no | Custom merge commit message. | -| `squash_commit_message` | string | no | Custom squash commit message. | -| `squash` | boolean | no | If `true` the commits are squashed into a single commit on merge. | -| `should_remove_source_branch` | boolean | no | If `true` removes the source branch. | -| `merge_when_pipeline_succeeds` | boolean | no | If `true` the MR is merged when the pipeline succeeds. | -| `sha` | string | no | If present, then this SHA must match the HEAD of the source branch, otherwise the merge fails. | +| Attribute | Type | Required | Description | +|--------------------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | +| `merge_commit_message` | string | **{dotted-circle}** No | Custom merge commit message. | +| `merge_when_pipeline_succeeds` | boolean | **{dotted-circle}** No | If `true`, the merge request is merged when the pipeline succeeds. | +| `sha` | string | **{dotted-circle}** No | If present, then this SHA must match the HEAD of the source branch, otherwise the merge fails. | +| `should_remove_source_branch` | boolean | **{dotted-circle}** No | If `true`, removes the source branch. | +| `squash_commit_message` | string | **{dotted-circle}** No | Custom squash commit message. | +| `squash` | boolean | **{dotted-circle}** No | If `true`, the commits are squashed into a single commit on merge. | ```json { @@ -1681,12 +1661,12 @@ the `approvals_before_merge` parameter: This API returns specific HTTP status codes on failure: -| HTTP Status | Message | Reason | -|:------------|--------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------| -| `401` | `Unauthorized` | This user does not have permission to accept this merge request. | -| `405` | `Method Not Allowed` | The merge request is not able to be merged. | -| `409` | `SHA does not match HEAD of source branch` | The provided `sha` parameter does not match the HEAD of the source. | -| `422` | `Branch cannot be merged` | The merge request failed to merge. | +| HTTP Status | Message | Reason | +|:------------|---------|--------| +| `401` | `Unauthorized` | This user does not have permission to accept this merge request. | +| `405` | `Method Not Allowed` | The merge request is not able to be merged. | +| `409` | `SHA does not match HEAD of source branch` | The provided `sha` parameter does not match the HEAD of the source. | +| `422` | `Branch cannot be merged` | The merge request failed to merge. | For additional important notes on response data, read [Single merge request response notes](#single-merge-request-response-notes). @@ -1709,12 +1689,12 @@ It returns the HEAD commit of `refs/merge-requests/:iid/merge` in the response b GET /projects/:id/merge_requests/:merge_request_iid/merge_ref ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```json { @@ -1736,12 +1716,12 @@ This API returns specific HTTP status codes on failure: POST /projects/:id/merge_requests/:merge_request_iid/cancel_merge_when_pipeline_succeeds ``` -Parameters: +Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```json { @@ -1905,11 +1885,11 @@ you receive a `403 Forbidden` response. PUT /projects/:id/merge_requests/:merge_request_iid/rebase ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `skip_ci` | boolean | no | Set to `true` to skip creating a CI pipeline. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | +| `skip_ci` | boolean | **{dotted-circle}** No | Set to `true` to skip creating a CI pipeline. | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/76/merge_requests/1/rebase" @@ -1956,6 +1936,26 @@ If the rebase operation fails, the response includes the following: } ``` +## Reset approvals of a merge request + +Clear all approvals of merge request. + +Available only for [bot users](../user/project/settings/project_access_tokens.md#bot-users-for-projects) +based on project or group tokens. Users without bot permissions receive a `401 Unauthorized` response. + +```plaintext +PUT /projects/:id/merge_requests/:merge_request_iid/reset_approvals +``` + +| Attribute | Type | Required | Description | +|---------------------|-------------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | + +```shell +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/76/merge_requests/1/reset_approvals" +``` + ## Comments on merge requests Comments are done via the [notes](notes.md) resource. @@ -1968,10 +1968,10 @@ Get all the issues that would be closed by merging the provided merge request. GET /projects/:id/merge_requests/:merge_request_iid/closes_issues ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/76/merge_requests/1/closes_issues" @@ -2044,10 +2044,10 @@ status code `HTTP 304 Not Modified` is returned. POST /projects/:id/merge_requests/:merge_request_iid/subscribe ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/17/subscribe" @@ -2214,10 +2214,10 @@ not subscribed to the merge request, the status code `HTTP 304 Not Modified` is POST /projects/:id/merge_requests/:merge_request_iid/unsubscribe ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/17/unsubscribe" @@ -2384,10 +2384,10 @@ status code `HTTP 304 Not Modified` is returned. POST /projects/:id/merge_requests/:merge_request_iid/todo ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/27/todo" @@ -2513,8 +2513,8 @@ GET /projects/:id/merge_requests/:merge_request_iid/versions | Attribute | Type | Required | Description | |---------------------|---------|----------|---------------------------------------| -| `id` | String | yes | The ID of the project. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| `id` | String | **{check-circle}** Yes | The ID of the project. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/merge_requests/1/versions" @@ -2548,8 +2548,8 @@ Example response: | SHA field | Purpose | |--------------------|-------------------------------------------------------------------------------------| -| `head_commit_sha` | The HEAD commit of the source branch. | | `base_commit_sha` | The merge-base commit SHA between the source branch and the target branches. | +| `head_commit_sha` | The HEAD commit of the source branch. | | `start_commit_sha` | The HEAD commit SHA of the target branch when this version of the diff was created. | ## Get a single MR diff version @@ -2563,9 +2563,9 @@ GET /projects/:id/merge_requests/:merge_request_iid/versions/:version_id | Attribute | Type | Required | Description | |---------------------|---------|----------|-------------------------------------------| -| `id` | String | yes | The ID of the project. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `version_id` | integer | yes | The ID of the merge request diff version. | +| `id` | String | **{check-circle}** Yes | The ID of the project. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | +| `version_id` | integer | **{check-circle}** Yes | The ID of the merge request diff version. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/merge_requests/1/versions/1" @@ -2629,11 +2629,11 @@ Sets an estimated time of work for this merge request. POST /projects/:id/merge_requests/:merge_request_iid/time_estimate ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `duration` | string | yes | The duration in human format, such as `3h30m`. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | +| `duration` | string | **{check-circle}** Yes | The duration in human format, such as `3h30m`. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/time_estimate?duration=3h30m" @@ -2658,10 +2658,10 @@ Resets the estimated time for this merge request to 0 seconds. POST /projects/:id/merge_requests/:merge_request_iid/reset_time_estimate ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of a project's merge_request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of a project's merge request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/reset_time_estimate" @@ -2686,12 +2686,12 @@ Adds spent time for this merge request. POST /projects/:id/merge_requests/:merge_request_iid/add_spent_time ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `duration` | string | yes | The duration in human format, such as `3h30m` | -| `summary` | string | no | A summary of how the time was spent. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | +| `duration` | string | **{check-circle}** Yes | The duration in human format, such as `3h30m` | +| `summary` | string | **{dotted-circle}** No | A summary of how the time was spent. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/add_spent_time?duration=1h" @@ -2716,10 +2716,10 @@ Resets the total spent time for this merge request to 0 seconds. POST /projects/:id/merge_requests/:merge_request_iid/reset_spent_time ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of a project's merge_request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of a project's merge request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/reset_spent_time" @@ -2742,10 +2742,10 @@ Example response: GET /projects/:id/merge_requests/:merge_request_iid/time_stats ``` -| Attribute | Type | Required | Description | -|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | **{check-circle}** Yes | The internal ID of the merge request. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/time_stats" diff --git a/doc/api/metadata.md b/doc/api/metadata.md index 70c29ef5748..0f27960937d 100644 --- a/doc/api/metadata.md +++ b/doc/api/metadata.md @@ -1,6 +1,6 @@ --- -stage: Ecosystem -group: Integrations +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/oauth2.md b/doc/api/oauth2.md index 12704f6fc87..0b7e0ba08eb 100644 --- a/doc/api/oauth2.md +++ b/doc/api/oauth2.md @@ -261,7 +261,7 @@ Check the [RFC spec](https://tools.ietf.org/html/rfc6749#section-4.3) for a detailed flow description. NOTE: -The Resource Owner Password Credentials is disabled for users with +The Resource Owner Password Credentials is disabled for users with [two-factor authentication](../user/profile/account/two_factor_authentication.md) turned on. These users can access the API using [personal access tokens](../user/profile/personal_access_tokens.md) instead. @@ -335,43 +335,6 @@ access_token = client.password.get_token('user@example.com', 'secret') puts access_token.token ``` -<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> - -### Implicit grant flow (removed) - -Implicit grant flow is inherently insecure and the IETF has removed it in [OAuth 2.1](https://oauth.net/2.1/). -It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/288516) in GitLab 14.0 and is -[removed](https://gitlab.com/gitlab-org/gitlab/-/issues/344609) in GitLab 15.0. - -We recommend that you use [Authorization code with PKCE](#authorization-code-with-proof-key-for-code-exchange-pkce) -instead. - -Unlike the authorization code flow, the client receives an `access token` -immediately as a result of the authorization request. The flow does not use the -client secret or the authorization code, as the application -code and storage is accessible on client browsers and mobile devices. - -To request the access token, you should redirect the user to the -`/oauth/authorize` endpoint using `token` response type: - -```plaintext -https://gitlab.example.com/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=token&state=YOUR_UNIQUE_STATE_HASH&scope=REQUESTED_SCOPES -``` - -This prompts the user to approve the applications access to their account -based on the scopes specified in `REQUESTED_SCOPES` and then redirect back to -the `REDIRECT_URI` you provided. The [scope parameter](../integration/oauth_provider.md#authorized-applications) - is a space-separated list of scopes you want to have access to (for example, `scope=read_user+profile` -would request `read_user` and `profile` scopes). The redirect -includes a fragment with `access_token` as well as token details in GET -parameters, for example: - -```plaintext -https://example.com/oauth/redirect#access_token=ABCDExyz123&state=YOUR_UNIQUE_STATE_HASH&token_type=bearer&expires_in=3600 -``` - -<!--- end_remove --> - ## Access GitLab API with `access token` The `access token` allows you to make requests to the API on behalf of a user. @@ -391,7 +354,11 @@ curl --header "Authorization: Bearer OAUTH-TOKEN" "https://gitlab.example.com/ap A token with [scope](../integration/oauth_provider.md#authorized-applications) `read_repository` or `write_repository` can access Git over HTTPS. Use the token as the password. -The username must be `oauth2`, not your username. +The username must be `oauth2`, not your username: + +```plaintext +https://oauth2:<your_access_token>@gitlab.example.com/project_path/project_name.git +``` ## Retrieve the token information diff --git a/doc/api/packages.md b/doc/api/packages.md index 3cd93bd09c1..d91f4f0de7b 100644 --- a/doc/api/packages.md +++ b/doc/api/packages.md @@ -213,6 +213,7 @@ Example response: "delete_api_path": "/namespace1/project1/-/packages/1" }, "created_at": "2019-11-27T03:37:38.711Z", + "last_downloaded_at": "2022-09-07T07:51:50.504Z" "pipelines": [ { "id": 123, diff --git a/doc/api/packages/conan.md b/doc/api/packages/conan.md index 3ac2eeb40b1..637c3d27d75 100644 --- a/doc/api/packages/conan.md +++ b/doc/api/packages/conan.md @@ -38,7 +38,7 @@ The examples in this document all use the instance-level prefix. /packages/conan/v1 ``` -When using the instance-level routes, be aware that there is a +When using the instance-level routes, be aware that there is a [naming restriction](../../user/packages/conan_repository/index.md#package-recipe-naming-convention-for-instance-remotes) for Conan recipes. diff --git a/doc/api/packages/debian.md b/doc/api/packages/debian.md index 4abb7bc7112..598124ba2b9 100644 --- a/doc/api/packages/debian.md +++ b/doc/api/packages/debian.md @@ -212,11 +212,11 @@ curl --header "Private-Token: <personal_access_token>" \ This writes the downloaded file using the remote filename in the current directory. -## Download a binary file's index +## Download a packages index > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64923) in GitLab 14.2. -Download a distribution index. +Download a packages index. ```plaintext GET <route-prefix>/dists/*distribution/:component/binary-:architecture/Packages @@ -229,14 +229,73 @@ GET <route-prefix>/dists/*distribution/:component/binary-:architecture/Packages | `architecture` | string | yes | The distribution architecture type. | ```shell -curl --header "Private-Token: <personal_access_token>" "https://gitlab.example.com/api/v4/projects/1/packages/debian/dists/my-distro/main/amd64/Packages" +curl --header "Private-Token: <personal_access_token>" "https://gitlab.example.com/api/v4/projects/1/packages/debian/dists/my-distro/main/binary-amd64/Packages" ``` Write the output to a file: ```shell curl --header "Private-Token: <personal_access_token>" \ - "https://gitlab.example.com/api/v4/projects/1/packages/debian/dists/my-distro/main/amd64/Packages" \ + "https://gitlab.example.com/api/v4/projects/1/packages/debian/dists/my-distro/main/binary-amd64/Packages" \ + --remote-name +``` + +This writes the downloaded file using the remote filename in the current directory. + +## Download a Debian Installer packages index + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71918) in GitLab 15.4. + +Download a Debian Installer packages index. + +```plaintext +GET <route-prefix>/dists/*distribution/:component/debian-installer/binary-:architecture/Packages +``` + +| Attribute | Type | Required | Description | +| ----------------- | ------ | -------- | ----------- | +| `distribution` | string | yes | The codename or suite of the Debian distribution. | +| `component` | string | yes | The distribution component name. | +| `architecture` | string | yes | The distribution architecture type. | + +```shell +curl --header "Private-Token: <personal_access_token>" "https://gitlab.example.com/api/v4/projects/1/packages/debian/dists/my-distro/main/debian-installer/binary-amd64/Packages" +``` + +Write the output to a file: + +```shell +curl --header "Private-Token: <personal_access_token>" \ + "https://gitlab.example.com/api/v4/projects/1/packages/debian/dists/my-distro/main/debian-installer/binary-amd64/Packages" \ + --remote-name +``` + +This writes the downloaded file using the remote filename in the current directory. + +## Download a source packages index + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71918) in GitLab 15.4. + +Download a source packages index. + +```plaintext +GET <route-prefix>/dists/*distribution/:component/source/Sources +``` + +| Attribute | Type | Required | Description | +| ----------------- | ------ | -------- | ----------- | +| `distribution` | string | yes | The codename or suite of the Debian distribution. | +| `component` | string | yes | The distribution component name. | + +```shell +curl --header "Private-Token: <personal_access_token>" "https://gitlab.example.com/api/v4/projects/1/packages/debian/dists/my-distro/main/source/Sources" +``` + +Write the output to a file: + +```shell +curl --header "Private-Token: <personal_access_token>" \ + "https://gitlab.example.com/api/v4/projects/1/packages/debian/dists/my-distro/main/source/Sources" \ --remote-name ``` diff --git a/doc/api/packages/terraform-modules.md b/doc/api/packages/terraform-modules.md index 24db7094a3c..daafe0579e7 100644 --- a/doc/api/packages/terraform-modules.md +++ b/doc/api/packages/terraform-modules.md @@ -12,8 +12,8 @@ WARNING: This API is used by the [terraform cli](https://www.terraform.io/) and is generally not meant for manual consumption. -For instructions on how to upload and install Maven packages from the GitLab -package registry, see the [Terraform modules registry documentation](../../user/packages/terraform_module_registry/index.md). +For instructions on how to upload and install Terraform modules from the GitLab +infrastructure registry, see the [Terraform modules registry documentation](../../user/packages/terraform_module_registry/index.md). ## List available versions for a specific module @@ -114,7 +114,7 @@ Example response: ## Get specific version for a specific module -Get information about the latest version for a given module. +Get information about a specific version for a given module. ```plaintext GET packages/terraform/modules/v1/:module_namespace/:module_name/:module_system/1.0.0 diff --git a/doc/api/personal_access_tokens.md b/doc/api/personal_access_tokens.md index 620b5c2ed0b..849b5c75684 100644 --- a/doc/api/personal_access_tokens.md +++ b/doc/api/personal_access_tokens.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -134,9 +134,13 @@ curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://git ### Using a request header -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350240) in GitLab 15.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350240) in GitLab 15.0. Limited to tokens with `api` scope. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/369103) in GitLab 15.4, any token can use this endpoint. -Revokes a personal access token that is passed in using a request header. +Revokes a personal access token that is passed in using a request header. Requires: + +- `api` scope in GitLab 15.0 to GitLab 15.3. +- Any scope in GitLab 15.4 and later. ```plaintext DELETE /personal_access_tokens/self diff --git a/doc/api/pipeline_triggers.md b/doc/api/pipeline_triggers.md index 8db071bf811..9f3120bd5d7 100644 --- a/doc/api/pipeline_triggers.md +++ b/doc/api/pipeline_triggers.md @@ -155,7 +155,7 @@ or a [CI/CD job token](../ci/jobs/ci_job_token.md) for authentication. With a CI/CD job token, the [triggered pipeline is a multi-project pipeline](../ci/jobs/ci_job_token.md#trigger-a-multi-project-pipeline-by-using-a-cicd-job-token). The job that authenticates the request becomes associated with the upstream pipeline, -which is visible on the [pipeline graph](../ci/pipelines/multi_project_pipelines.md#multi-project-pipeline-visualization). +which is visible on the [pipeline graph](../ci/pipelines/downstream_pipelines.md#view-multi-project-pipelines-in-pipeline-graphs). If you use a trigger token in a job, the job is not associated with the upstream pipeline. diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index 2e601f6e24a..23c55cfb177 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -80,7 +80,7 @@ Example of response Get one pipeline from a project. -You can also get a single [child pipeline](../ci/pipelines/parent_child_pipelines.md). +You can also get a single [child pipeline](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36494) in GitLab 13.3. ```plaintext @@ -427,7 +427,7 @@ related objects, such as builds, logs, artifacts, and triggers. **This action cannot be undone.** Deleting a pipeline does not automatically delete its -[child pipelines](../ci/pipelines/parent_child_pipelines.md). +[child pipelines](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines). See the [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/39503) for details. diff --git a/doc/api/product_analytics.md b/doc/api/product_analytics.md new file mode 100644 index 00000000000..c3114baff8a --- /dev/null +++ b/doc/api/product_analytics.md @@ -0,0 +1,67 @@ +--- +stage: Analyze +group: Product Analytics +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Product analytics API + +> Introduced in GitLab 15.4 [with a flag](../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `cube_api_proxy`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +NOTE: +Make sure to define the `cube_api_base_url` and `cube_api_key` application settings first using [the API](settings.md). + +## Send request to Cube + +Generate an access token that can be used to query the Cube API. For example: + +```plaintext +POST /projects/:id/product_analytics/request +``` + +| Attribute | Type | Required | Description | +| --------- |------------------| -------- |---------------------------------------------------------------| +| `id` | integer | yes | The ID of a project that the current user has read access to. | + +### Request body + +The body of the request should be a valid Cube query. + +```json +{ + "query": { + "measures": [ + "Jitsu.count" + ], + "timeDimensions": [ + { + "dimension": "Jitsu.utcTime", + "dateRange": "This week" + } + ], + "order": [ + [ + "Jitsu.count", + "desc" + ], + [ + "Jitsu.docPath", + "desc" + ], + [ + "Jitsu.utcTime", + "asc" + ] + ], + "dimensions": [ + "Jitsu.docPath" + ], + "limit": 23 + } +} +``` diff --git a/doc/api/project_level_variables.md b/doc/api/project_level_variables.md index 81bb4a26614..39e7f441b42 100644 --- a/doc/api/project_level_variables.md +++ b/doc/api/project_level_variables.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference, api --- -# Project-level Variables API **(FREE)** +# Project-level CI/CD variables API **(FREE)** ## List project variables @@ -44,9 +44,10 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a ] ``` -## Show variable details +## Get a single variable -Get the details of a project's specific variable. +Get the details of a single variable. If there are multiple variables with the same key, +use `filter` to select the correct `environment_scope`. ```plaintext GET /projects/:id/variables/:key @@ -73,9 +74,11 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a } ``` -## Create variable +## Create a variable -Create a new variable. +Create a new variable. If a variable with the same `key` already exists, the new variable +must have a different `environment_scope`. Otherwise, GitLab returns a message similar to: +`VARIABLE_NAME has already been taken`. ```plaintext POST /projects/:id/variables @@ -107,9 +110,10 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ } ``` -## Update variable +## Update a variable -Update a project's variable. +Update a project's variable. If there are multiple variables with the same key, +use `filter` to select the correct `environment_scope`. ```plaintext PUT /projects/:id/variables/:key @@ -142,9 +146,10 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ } ``` -## Remove variable +## Delete a variable -Remove a project's variable. +Delete a project's variable. If there are multiple variables with the same key, +use `filter` to select the correct `environment_scope`. ```plaintext DELETE /projects/:id/variables/:key @@ -165,10 +170,34 @@ curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://git > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34490) in GitLab 13.2. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/227052) in GitLab 13.4. -This parameter is used for filtering by attributes, such as `environment_scope`. +When multiple variables have the same `key`, [GET](#get-a-single-variable), [PUT](#update-a-variable), +or [DELETE](#delete-a-variable) requests might return: -Example usage: - -```shell -curl --globoff --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables/VARIABLE_1?filter[environment_scope]=production" +```plaintext +There are multiple variables with provided parameters. Please use 'filter[environment_scope]'. ``` + +Use `filter[environment_scope]` to select the variable with the matching `environment_scope` attribute. + +For example: + +- GET: + + ```shell + curl --globoff --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/1/variables/SCOPED_VARIABLE_1?filter[environment_scope]=production" + ``` + +- PUT: + + ```shell + curl --request PUT --globoff --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/1/variables/SCOPED_VARIABLE_1?value=scoped-variable-updated-value&environment_scope=production&filter[environment_scope]=production" + ``` + +- DELETE: + + ```shell + curl --request DELETE --globoff --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/1/variables/SCOPED_VARIABLE_1?filter[environment_scope]=production" + ``` diff --git a/doc/api/project_templates.md b/doc/api/project_templates.md index 7763087e759..14dd0761487 100644 --- a/doc/api/project_templates.md +++ b/doc/api/project_templates.md @@ -30,8 +30,8 @@ GET /projects/:id/templates/:type | Attribute | Type | Required | Description | | ---------- | ------ | -------- | ----------- | -| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | -| `type` | string | yes | The type of the template. Accepted values are: `dockerfiles`, `gitignores`, `gitlab_ci_ymls`, `licenses`, `issues`, `merge_requests` | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). | +| `type` | string | **{check-circle}** Yes | The type of the template. Accepted values are: `dockerfiles`, `gitignores`, `gitlab_ci_ymls`, `licenses`, `issues`, or `merge_requests`. | Example response (licenses): @@ -96,12 +96,12 @@ GET /projects/:id/templates/:type/:name | Attribute | Type | Required | Description | | ---------- | ------ | -------- | ----------- | -| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | -| `type` | string | yes| The type of the template. One of: `dockerfiles`, `gitignores`, `gitlab_ci_ymls`, `licenses`, `issues`, or `merge_requests`. | -| `name` | string | yes | The key of the template, as obtained from the collection endpoint | -| `source_template_project_id` | integer | no | The project ID where a given template is being stored. This is useful when multiple templates from different projects have the same name. If multiple templates have the same name, the match from `closest ancestor` is returned if `source_template_project_id` is not specified | -| `project` | string | no | The project name to use when expanding placeholders in the template. Only affects licenses | -| `fullname` | string | no | The full name of the copyright holder to use when expanding placeholders in the template. Only affects licenses | +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). | +| `name` | string | **{check-circle}** Yes | The key of the template, as obtained from the collection endpoint. | +| `type` | string | **{check-circle}** Yes | The type of the template. One of: `dockerfiles`, `gitignores`, `gitlab_ci_ymls`, `licenses`, `issues`, or `merge_requests`. | +| `fullname` | string | **{dotted-circle}** No | The full name of the copyright holder to use when expanding placeholders in the template. Affects only licenses. | +| `project` | string | **{dotted-circle}** No | The project name to use when expanding placeholders in the template. Affects only licenses. | +| `source_template_project_id` | integer | **{dotted-circle}** No | The project ID where a given template is being stored. Helpful when multiple templates from different projects have the same name. If multiple templates have the same name, the match from `closest ancestor` is returned if `source_template_project_id` is not specified, | Example response (Dockerfile): diff --git a/doc/api/projects.md b/doc/api/projects.md index 75eea394a40..26733801b45 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -417,6 +417,7 @@ GET /users/:user_id/projects "merge_method": "merge", "squash_option": "default_on", "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, "suggestion_commit_message": null, "merge_commit_template": null, "squash_commit_template": null, @@ -539,6 +540,7 @@ GET /users/:user_id/projects "service_desk_enabled": false, "service_desk_address": null, "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, "suggestion_commit_message": null, "merge_commit_template": null, "squash_commit_template": null, @@ -671,6 +673,7 @@ Example response: "merge_method": "merge", "squash_option": "default_on", "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, "suggestion_commit_message": null, "merge_commit_template": null, "squash_commit_template": null, @@ -786,6 +789,7 @@ Example response: "service_desk_enabled": false, "service_desk_address": null, "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, "suggestion_commit_message": null, "merge_commit_template": null, "squash_commit_template": null, @@ -962,6 +966,7 @@ GET /projects/:id "service_desk_address": null, "autoclose_referenced_issues": true, "suggestion_commit_message": null, + "enforce_auth_checks_on_uploads": true, "merge_commit_template": null, "squash_commit_template": null, "marked_for_deletion_at": "2020-04-03", // Deprecated and will be removed in API v5 in favor of marked_for_deletion_on @@ -1218,7 +1223,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your-token>" \ | `mirror` **(PREMIUM)** | boolean | **{dotted-circle}** No | Enables pull mirroring in a project. | | `namespace_id` | integer | **{dotted-circle}** No | Namespace for the new project (defaults to the current user's namespace). | | `only_allow_merge_if_all_discussions_are_resolved` | boolean | **{dotted-circle}** No | Set whether merge requests can only be merged when all the discussions are resolved. | -| `only_allow_merge_if_pipeline_succeeds` | boolean | **{dotted-circle}** No | Set whether merge requests can only be merged with successful pipelines. This setting is named [**Pipelines must succeed**](../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) in the project settings. | +| `only_allow_merge_if_pipeline_succeeds` | boolean | **{dotted-circle}** No | Set whether merge requests can only be merged with successful pipelines. This setting is named [**Pipelines must succeed**](../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) in the project settings. | | `operations_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. | | `packages_enabled` | boolean | **{dotted-circle}** No | Enable or disable packages repository feature. | | `pages_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, `enabled`, or `public`. | @@ -1277,6 +1282,7 @@ POST /projects/user/:user_id | `default_branch` | string | **{dotted-circle}** No | The [default branch](../user/project/repository/branches/default.md) name. Requires `initialize_with_readme` to be `true`. | | `description` | string | **{dotted-circle}** No | Short project description. | | `emails_disabled` | boolean | **{dotted-circle}** No | Disable email notifications. | +| `enforce_auth_checks_on_uploads` | boolean | **{dotted-circle}** No | Enforce [auth checks](../security/user_file_uploads.md#enable-authorization-checks-for-all-media-files) on uploads. | | `external_authorization_classification_label` **(PREMIUM)** | string | **{dotted-circle}** No | The classification label for the project. | | `forking_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. | | `group_with_project_templates_id` **(PREMIUM)** | integer | **{dotted-circle}** No | For group-level custom templates, specifies ID of group from which all the custom project templates are sourced. Leave empty for instance-level templates. Requires `use_custom_template` to be true. | @@ -1370,6 +1376,7 @@ Supported attributes: | `default_branch` | string | **{dotted-circle}** No | The [default branch](../user/project/repository/branches/default.md) name. | | `description` | string | **{dotted-circle}** No | Short project description. | | `emails_disabled` | boolean | **{dotted-circle}** No | Disable email notifications. | +| `enforce_auth_checks_on_uploads` | boolean | **{dotted-circle}** No | Enforce [auth checks](../security/user_file_uploads.md#enable-authorization-checks-for-all-media-files) on uploads. | | `external_authorization_classification_label` **(PREMIUM)** | string | **{dotted-circle}** No | The classification label for the project. | | `forking_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. | | `import_url` | string | **{dotted-circle}** No | URL the repository was imported from. | @@ -1542,6 +1549,7 @@ Example responses: "merge_method": "merge", "squash_option": "default_on", "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, "suggestion_commit_message": null, "merge_commit_template": null, "container_registry_image_prefix": "registry.example.com/diaspora/diaspora-project-site", @@ -1648,6 +1656,7 @@ Example response: "merge_method": "merge", "squash_option": "default_on", "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, "suggestion_commit_message": null, "merge_commit_template": null, "container_registry_image_prefix": "registry.example.com/diaspora/diaspora-project-site", @@ -1752,6 +1761,7 @@ Example response: "merge_method": "merge", "squash_option": "default_on", "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, "suggestion_commit_message": null, "merge_commit_template": null, "container_registry_image_prefix": "registry.example.com/diaspora/diaspora-project-site", @@ -1952,6 +1962,7 @@ Example response: "merge_method": "merge", "squash_option": "default_on", "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, "suggestion_commit_message": null, "merge_commit_template": null, "container_registry_image_prefix": "registry.example.com/diaspora/diaspora-project-site", @@ -2079,6 +2090,7 @@ Example response: "merge_method": "merge", "squash_option": "default_on", "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, "suggestion_commit_message": null, "merge_commit_template": null, "container_registry_image_prefix": "registry.example.com/diaspora/diaspora-project-site", @@ -2248,6 +2260,19 @@ Returned object: } ``` +## Remove a project avatar + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92604) in GitLab 15.4. + +To remove a project avatar, use a blank value for the `avatar` attribute. + +Example request: + +```shell +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ + --data "avatar=" "https://gitlab.example.com/api/v4/projects/5" +``` + ## Share project with group Allow to share project with group. @@ -2596,6 +2621,50 @@ DELETE /projects/:id/push_rule |-----------|----------------|------------------------|-------------| | `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). | +## Get groups to which a user can transfer a project + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371006) in GitLab 15.4 + +Retrieve a list of groups to which the user can transfer a project. + +```plaintext +GET /projects/:id/transfer_locations +``` + +| Attribute | Type | Required | Description | +|-------------|----------------|------------------------|-------------| +| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). | +| `search` | string | **{dotted-circle}** No | The group names to search for. | + +Example request: + +```shell +curl --request GET "https://gitlab.example.com/api/v4/projects/1/transfer_locations" +``` + +Example response: + +```json +[ + { + "id": 27, + "web_url": "https://gitlab.example.com/groups/gitlab", + "name": "GitLab", + "avatar_url": null, + "full_name": "GitLab", + "full_path": "GitLab" + }, + { + "id": 31, + "web_url": "https://gitlab.example.com/groups/foobar", + "name": "FooBar", + "avatar_url": null, + "full_name": "FooBar", + "full_path": "FooBar" + } +] +``` + ## Transfer a project to a new namespace > The `_links.cluster_agents` attribute in the response [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/347047) in GitLab 14.10. diff --git a/doc/api/protected_environments.md b/doc/api/protected_environments.md index 46cb461b64b..5b52d11feda 100644 --- a/doc/api/protected_environments.md +++ b/doc/api/protected_environments.md @@ -11,7 +11,7 @@ type: concepts, howto ## Valid access levels -The access levels are defined in the `ProtectedEnvironment::DeployAccessLevel::ALLOWED_ACCESS_LEVELS` method. +The access levels are defined in the `ProtectedEnvironments::DeployAccessLevel::ALLOWED_ACCESS_LEVELS` method. Currently, these levels are recognized: ```plaintext @@ -54,6 +54,7 @@ Example response: "name":"production", "deploy_access_levels":[ { + "id": 12, "access_level":40, "access_level_description":"Maintainers", "user_id":null, @@ -61,7 +62,7 @@ Example response: "group_inheritance_type": 0 } ], - "required_approval_count": 0 + "required_approval_count": 0 } ] ``` @@ -90,6 +91,7 @@ Example response: "name":"production", "deploy_access_levels":[ { + "id": 12, "access_level": 40, "access_level_description": "Maintainers", "user_id": null, @@ -97,7 +99,7 @@ Example response: "group_inheritance_type": 0 } ], - "required_approval_count": 0 + "required_approval_count": 0 } ``` @@ -109,6 +111,20 @@ Protects a single environment: POST /projects/:id/protected_environments ``` +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `name` | string | yes | The name of the environment. | +| `deploy_access_levels` | array | yes | Array of access levels allowed to deploy, with each described by a hash. | +| `required_approval_count` | integer | no | The number of approvals required to deploy to this environment. | +| `approval_rules` | array | no | Array of access levels allowed to approve, with each described by a hash. See [Multiple approval rules](../ci/environments/deployment_approvals.md#multiple-approval-rules) for more information. | + +Elements in the `deploy_access_levels` and `approval_rules` array should be one of `user_id`, `group_id` or +`access_level`, and take the form `{user_id: integer}`, `{group_id: integer}` or +`{access_level: integer}`. Optionally you can specify the `group_inheritance_type` on each as one of the [valid group inheritance types](#group-inheritance-types). + +Each user must have access to the project and each group must [have this project shared](../user/project/members/share_project_with_groups.md). + ```shell curl --header 'Content-Type: application/json' --request POST \ --data '{"name": "production", "deploy_access_levels": [{"group_id": 9899826}], "approval_rules": [{"group_id": 134}, {"group_id": 135, "required_approvals": 2}]}' \ @@ -116,19 +132,84 @@ curl --header 'Content-Type: application/json' --request POST \ "https://gitlab.example.com/api/v4/projects/22034114/protected_environments" ``` +Example response: + +```json +{ + "name": "production", + "deploy_access_levels": [ + { + "id": 12, + "access_level": 40, + "access_level_description": "protected-access-group", + "user_id": null, + "group_id": 9899826, + "group_inheritance_type": 0 + } + ], + "required_approval_count": 0, + "approval_rules": [ + { + "id": 38, + "user_id": null, + "group_id": 134, + "access_level": null, + "access_level_description": "qa-group", + "required_approvals": 1, + "group_inheritance_type": 0 + }, + { + "id": 39, + "user_id": null, + "group_id": 135, + "access_level": null, + "access_level_description": "security-group", + "required_approvals": 2, + "group_inheritance_type": 0 + } + ] +} +``` + +## Update a protected environment + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/351854) in GitLab 15.4. + +Updates a single environment. + +```plaintext +PUT /projects/:id/protected_environments/:name +``` + | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | | `name` | string | yes | The name of the environment. | -| `deploy_access_levels` | array | yes | Array of access levels allowed to deploy, with each described by a hash. | -| `required_approval_count` | integer | no | The number of approvals required to deploy to this environment. This is part of Deployment Approvals, which isn't yet available for use. For details, see [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/343864). | +| `deploy_access_levels` | array | no | Array of access levels allowed to deploy, with each described by a hash. | +| `required_approval_count` | integer | no | The number of approvals required to deploy to this environment. | | `approval_rules` | array | no | Array of access levels allowed to approve, with each described by a hash. See [Multiple approval rules](../ci/environments/deployment_approvals.md#multiple-approval-rules) for more information. | Elements in the `deploy_access_levels` and `approval_rules` array should be one of `user_id`, `group_id` or `access_level`, and take the form `{user_id: integer}`, `{group_id: integer}` or `{access_level: integer}`. Optionally you can specify the `group_inheritance_type` on each as one of the [valid group inheritance types](#group-inheritance-types). -Each user must have access to the project and each group must [have this project shared](../user/project/members/share_project_with_groups.md). +To update: + +- **`user_id`**: Ensure the updated user has access to the project. You must also pass the `id` of either a `deploy_access_level` or `approval_rule` in the respective hash. +- **`group_id`**: Ensure the updated group [have this project shared](../user/project/members/share_project_with_groups.md). You must also pass the `id` of either a `deploy_access_level` or `approval_rule` in the respective hash. + +To delete: + +- You must pass `_destroy` set to `true`. See the following examples. + +### Example: Create a `deploy_access_level` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"deploy_access_levels": [{"group_id": 9899829, access_level: 40}], "required_approval_count": 1}' \ + --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/22034114/protected_environments/production" +``` Example response: @@ -137,32 +218,128 @@ Example response: "name": "production", "deploy_access_levels": [ { + "id": 12, "access_level": 40, "access_level_description": "protected-access-group", "user_id": null, - "group_id": 9899826, + "group_id": 9899829, + "group_inheritance_type": 1 + } + ], + "required_approval_count": 0 +} +``` + +### Example: Update a `deploy_access_level` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"deploy_access_levels": [{"id": 12, "group_id": 22034120}], "required_approval_count": 2}' \ + --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/22034114/protected_environments/production" +``` + +```json +{ + "name": "production", + "deploy_access_levels": [ + { + "id": 12, + "access_level": 40, + "access_level_description": "protected-access-group", + "user_id": null, + "group_id": 22034120, "group_inheritance_type": 0 } ], - "required_approval_count": 0, - "approval_rules": [ - { - "user_id": null, - "group_id": 134, - "access_level": null, - "access_level_description": "qa-group", - "required_approvals": 1, - "group_inheritance_type": 0 - }, - { - "user_id": null, - "group_id": 135, - "access_level": null, - "access_level_description": "security-group", - "required_approvals": 2, - "group_inheritance_type": 0 - } - ] + "required_approval_count": 2 +} +``` + +### Example: Delete a `deploy_access_level` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"deploy_access_levels": [{"id": 12, "_destroy": true}], "required_approval_count": 0}' \ + --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/22034114/protected_environments/production" +``` + +Example response: + +```json +{ + "name": "production", + "deploy_access_levels": [], + "required_approval_count": 0 +} +``` + +### Example: Create an `approval_rule` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"approval_rules": [{"group_id": 134, "required_approvals": 1}]}' \ + --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/22034114/protected_environments/production" +``` + +Example response: + +```json +{ + "name": "production", + "approval_rules": [ + { + "id": 38, + "user_id": null, + "group_id": 134, + "access_level": null, + "access_level_description": "qa-group", + "required_approvals": 1, + "group_inheritance_type": 0 + } + ] +} +``` + +### Example: Update an `approval_rule` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"approval_rules": [{"id": 38, "group_id": 135, "required_approvals": 2}]}' \ + --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/22034114/protected_environments/production" +``` + +```json +{ + "name": "production", + "approval_rules": [ + { + "id": 38, + "user_id": null, + "group_id": 135, + "access_level": null, + "access_level_description": "security-group", + "required_approvals": 2, + "group_inheritance_type": 0 + } + ] +} +``` + +### Example: Delete an `approval_rule` record + +```shell +curl --header 'Content-Type: application/json' --request PUT \ + --data '{"approval_rules": [{"id": 38, "_destroy": true}]}' \ + --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/22034114/protected_environments/production" +``` + +Example response: + +```json +{ + "name": "production", + "approval_rules": [] } ``` @@ -174,11 +351,11 @@ Unprotects the given protected environment: DELETE /projects/:id/protected_environments/:name ``` -```shell -curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/protected_environments/staging" -``` - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | | `name` | string | yes | The name of the protected environment. | + +```shell +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/protected_environments/staging" +``` diff --git a/doc/api/releases/index.md b/doc/api/releases/index.md index 1332eea26c0..e286fefc462 100644 --- a/doc/api/releases/index.md +++ b/doc/api/releases/index.md @@ -386,7 +386,7 @@ POST /projects/:id/releases | `assets:links:url` | string | required by: `assets:links` | The URL of the link. Link URLs must be unique within the release. | | `assets:links:filepath` | string | no | Optional path for a [Direct Asset link](../../user/project/releases/release_fields.md#permanent-links-to-release-assets). | `assets:links:link_type` | string | no | The type of the link: `other`, `runbook`, `image`, `package`. Defaults to `other`. -| `released_at` | datetime | no | The date when the release is/was ready. Defaults to the current time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `released_at` | datetime | no | Date and time for the release. Defaults to the current time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). Only provide this field if creating an [upcoming](../../user/project/releases/index.md#upcoming-releases) or [historical](../../user/project/releases/index.md#historical-releases) release. | Example request: diff --git a/doc/api/repositories.md b/doc/api/repositories.md index bf2ead43519..7d94edc0872 100644 --- a/doc/api/repositories.md +++ b/doc/api/repositories.md @@ -342,7 +342,7 @@ tags using these formats: - `vX.Y.Z` - `X.Y.Z` -Where `X.Y.Z` is a version that follows [semantic versioning](https://semver.org/). +Where `X.Y.Z` is a version that follows [semantic versioning](https://semver.org/). For example, consider a project with the following tags: - v1.0.0-pre1 diff --git a/doc/api/resource_label_events.md b/doc/api/resource_label_events.md index 9c05d32c992..da265972f28 100644 --- a/doc/api/resource_label_events.md +++ b/doc/api/resource_label_events.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/resource_state_events.md b/doc/api/resource_state_events.md index b2e886618d5..8e957df8145 100644 --- a/doc/api/resource_state_events.md +++ b/doc/api/resource_state_events.md @@ -8,8 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35210/) in GitLab 13.2. -Resource state events keep track of what happens to GitLab [issues](../user/project/issues/index.md) and -[merge requests](../user/project/merge_requests/index.md). +Resource state events keep track of what happens to GitLab [issues](../user/project/issues/index.md) +[merge requests](../user/project/merge_requests/index.md) and [epics starting with GitLab 15.4](../user/group/epics/index.md) Use them to track which state was set, who did it, and when it happened. @@ -212,3 +212,105 @@ Example response: "state": "closed" } ``` + +## Epics + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97554) in GitLab 15.4. + +### List group epic state events + +Returns a list of all state events for a single epic. + +```plaintext +GET /groups/:id/epics/:epic_id/resource_state_events +``` + +| Attribute | Type | Required | Description | +|-------------| -------------- | -------- |--------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding). | +| `epic_id` | integer | yes | The ID of an epic. | + +Example request: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/epics/11/resource_state_events" +``` + +Example response: + +```json +[ + { + "id": 142, + "user": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://gitlab.example.com/root" + }, + "created_at": "2018-08-20T13:38:20.077Z", + "resource_type": "Epic", + "resource_id": 11, + "state": "opened" + }, + { + "id": 143, + "user": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://gitlab.example.com/root" + }, + "created_at": "2018-08-21T14:38:20.077Z", + "resource_type": "Epic", + "resource_id": 11, + "state": "closed" + } +] +``` + +### Get single epic state event + +Returns a single state event for a specific group epic. + +```plaintext +GET /groups/:id/epics/:epic_id/resource_state_events/:resource_state_event_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +|---------------------------| -------------- | -------- |-------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding). | +| `epic_id` | integer | yes | The ID of an epic. | +| `resource_state_event_id` | integer | yes | The ID of a state event. | + +Example request: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/5/epics/11/resource_state_events/143" +``` + +Example response: + +```json +{ + "id": 143, + "user": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://gitlab.example.com/root" + }, + "created_at": "2018-08-21T14:38:20.077Z", + "resource_type": "Epic", + "resource_id": 11, + "state": "closed" +} +``` diff --git a/doc/api/settings.md b/doc/api/settings.md index d11269113a1..467fa6bfc37 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -89,6 +89,7 @@ Example response: "asset_proxy_url": "https://assets.example.com", "asset_proxy_whitelist": ["example.com", "*.example.com", "your-instance.com"], "asset_proxy_allowlist": ["example.com", "*.example.com", "your-instance.com"], + "maven_package_requests_forwarding": true, "npm_package_requests_forwarding": true, "pypi_package_requests_forwarding": true, "snippet_size_limit": 52428800, @@ -201,6 +202,7 @@ Example response: "allow_local_requests_from_hooks_and_services": true, "allow_local_requests_from_web_hooks_and_services": true, "allow_local_requests_from_system_hooks": false, + "maven_package_requests_forwarding": true, "npm_package_requests_forwarding": true, "pypi_package_requests_forwarding": true, "snippet_size_limit": 52428800, @@ -276,7 +278,7 @@ listed in the descriptions of the relevant settings. | `deactivate_dormant_users` | boolean | no | Enable [automatic deactivation of dormant users](../user/admin_area/moderate_users.md#automatically-deactivate-dormant-users). | | `deactivate_dormant_users_period` | integer | no | Length of time (in days) after which a user is considered dormant. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336747) in GitLab 15.3. | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts. | -| `default_branch_name` | string | no | [Instance-level custom initial branch name](../user/project/repository/branches/default.md#instance-level-custom-initial-branch-name) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225258) in GitLab 13.2). | +| `default_branch_name` | string | no | [Instance-level custom initial branch name](../user/project/repository/branches/default.md#instance-level-custom-initial-branch-name). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225258) in GitLab 13.2. | | `default_branch_protection` | integer | no | Determine if developers can push to the default branch. Can take: `0` _(not protected, both users with the Developer role or Maintainer role can push new commits and force push)_, `1` _(partially protected, users with the Developer role or Maintainer role can push new commits, but cannot force push)_ or `2` _(fully protected, users with the Developer or Maintainer role cannot push new commits, but users with the Developer or Maintainer role can; no one can force push)_ as a parameter. Default is `2`. | | `default_ci_config_path` | string | no | Default CI/CD configuration file and path for new projects (`.gitlab-ci.yml` if not set). | | `default_group_visibility` | string | no | What visibility level new groups receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | @@ -286,12 +288,12 @@ listed in the descriptions of the relevant settings. | `default_snippet_visibility` | string | no | What visibility level new snippets receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `delayed_project_deletion` **(PREMIUM SELF)** | boolean | no | Enable delayed project deletion by default in new groups. Default is `false`. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), can only be enabled when `delayed_group_deletion` is true. | | `delayed_group_deletion` **(PREMIUM SELF)** | boolean | no | Enable delayed group deletion. Default is `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352959) in GitLab 15.0. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), disables and locks the group-level setting for delayed protect deletion when set to `false`. | -| `delete_inactive_projects` | boolean | no | Enable inactive project deletion feature. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/84519) in GitLab 14.10. [Became operational](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85689) in GitLab 15.0 (with feature flag `inactive_projects_deletion`, disabled by default). | +| `delete_inactive_projects` | boolean | no | Enable inactive project deletion feature. Default is `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/84519) in GitLab 14.10. [Became operational without feature flag](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96803) in GitLab 15.4. | | `deletion_adjourned_period` **(PREMIUM SELF)** | integer | no | The number of days to wait before deleting a project or group that is marked for deletion. Value must be between `1` and `90`. Defaults to `7`. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), a hook on `deletion_adjourned_period` sets the period to `1` on every update, and sets both `delayed_project_deletion` and `delayed_group_deletion` to `false` if the period is `0`. | | `diff_max_patch_bytes` | integer | no | Maximum [diff patch size](../user/admin_area/diff_limits.md), in bytes. | | `diff_max_files` | integer | no | Maximum [files in a diff](../user/admin_area/diff_limits.md). | | `diff_max_lines` | integer | no | Maximum [lines in a diff](../user/admin_area/diff_limits.md). | -| `disable_feed_token` | boolean | no | Disable display of RSS/Atom and calendar feed tokens ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231493) in GitLab 13.7) | +| `disable_feed_token` | boolean | no | Disable display of RSS/Atom and calendar feed tokens. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231493) in GitLab 13.7. | | `disabled_oauth_sign_in_sources` | array of strings | no | Disabled OAuth sign-in sources. | | `dns_rebinding_protection_enabled` | boolean | no | Enforce DNS rebinding attack protection. | | `domain_denylist_enabled` | boolean | no | (**If enabled, requires:** `domain_denylist`) Allows blocking sign-ups from emails from specific domains. | @@ -377,7 +379,7 @@ listed in the descriptions of the relevant settings. | `max_artifacts_size` | integer | no | Maximum artifacts size in MB. | | `max_attachment_size` | integer | no | Limit attachment size in MB. | | `max_export_size` | integer | no | Maximum export size in MB. 0 for unlimited. Default = 0 (unlimited). | -| `max_import_size` | integer | no | Maximum import size in MB. 0 for unlimited. Default = 0 (unlimited) [Modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50MB to 0 in GitLab 13.8. | +| `max_import_size` | integer | no | Maximum import size in MB. 0 for unlimited. Default = 0 (unlimited). [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to 0 in GitLab 13.8. | | `max_pages_size` | integer | no | Maximum size of pages repositories in MB. | | `max_personal_access_token_lifetime` **(ULTIMATE SELF)** | integer | no | Maximum allowable lifetime for access tokens in days. | | `max_ssh_key_lifetime` **(ULTIMATE SELF)** | integer | no | Maximum allowable lifetime for SSH keys in days. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1007) in GitLab 14.6. | @@ -385,10 +387,12 @@ listed in the descriptions of the relevant settings. | `max_number_of_repository_downloads` **(ULTIMATE SELF)** | integer | no | Maximum number of unique repositories a user can download in the specified time period before they are banned. Default: 0, Maximum: 10,000 repositories. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87980) in GitLab 15.1. | | `max_number_of_repository_downloads_within_time_period` **(ULTIMATE SELF)** | integer | no | Reporting time period (in seconds). Default: 0, Maximum: 864000 seconds (10 days). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87980) in GitLab 15.1. | | `git_rate_limit_users_allowlist` **(ULTIMATE SELF)** | array of strings | no | List of usernames excluded from Git anti-abuse rate limits. Default: `[]`, Maximum: 100 usernames. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90815) in GitLab 15.2. | +| `auto_ban_user_on_excessive_projects_download` **(ULTIMATE SELF)** | boolean | no | When enabled, users will get automatically banned from the application when they download more than the maximum number of unique projects in the time period specified by `max_number_of_repository_downloads` and `max_number_of_repository_downloads_within_time_period` respectively. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/94153) in GitLab 15.4 | | `mirror_available` | boolean | no | Allow repository mirroring to configured by project Maintainers. If disabled, only Administrators can configure repository mirroring. | | `mirror_capacity_threshold` **(PREMIUM)** | integer | no | Minimum capacity to be available before scheduling more mirrors preemptively. | | `mirror_max_capacity` **(PREMIUM)** | integer | no | Maximum number of mirrors that can be synchronizing at the same time. | | `mirror_max_delay` **(PREMIUM)** | integer | no | Maximum time (in minutes) between updates that a mirror can have when scheduled to synchronize. | +| `maven_package_requests_forwarding` **(PREMIUM)** | boolean | no | Use repo.maven.apache.org as a default remote repository when the package is not found in the GitLab Package Registry for Maven. | | `npm_package_requests_forwarding` **(PREMIUM)** | boolean | no | Use npmjs.org as a default remote repository when the package is not found in the GitLab Package Registry for npm. | | `pypi_package_requests_forwarding` **(PREMIUM)** | boolean | no | Use pypi.org as a default remote repository when the package is not found in the GitLab Package Registry for PyPI. | | `outbound_local_requests_whitelist` | array of strings | no | Define a list of trusted domains or IP addresses to which local requests are allowed when local requests for hooks and services are disabled. diff --git a/doc/api/status_checks.md b/doc/api/status_checks.md index 92e003bf80d..1cedd2d3730 100644 --- a/doc/api/status_checks.md +++ b/doc/api/status_checks.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" type: reference, api diff --git a/doc/api/system_hooks.md b/doc/api/system_hooks.md index a883c3c1613..14339460270 100644 --- a/doc/api/system_hooks.md +++ b/doc/api/system_hooks.md @@ -10,7 +10,7 @@ All methods require administrator authorization. You can configure the URL endpoint of the system hooks from the GitLab user interface: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. Select **System Hooks** (`/admin/hooks`). Read more about [system hooks](../administration/system_hooks.md). diff --git a/doc/api/tags.md b/doc/api/tags.md index 903cb361587..45c621f534e 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -8,6 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## List project repository tags +> `version` value for the `order_by` attribute [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95150) in GitLab 15.4. + Get a list of repository tags from a project, sorted by update date and time in descending order. This endpoint can be accessed without authentication if the repository is publicly accessible. @@ -19,9 +21,9 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string| yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user| -| `order_by` | string | no | Return tags ordered by `name` or `updated` fields. Default is `updated` | -| `sort` | string | no | Return tags sorted in `asc` or `desc` order. Default is `desc` | +| `id` | integer or string| yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `order_by` | string | no | Return tags ordered by `name`, `updated`, or `version`. Default is `updated`. | +| `sort` | string | no | Return tags sorted in `asc` or `desc` order. Default is `desc`. | | `search` | string | no | Return list of tags matching the search criteria. You can use `^term` and `term$` to find tags that begin and end with `term` respectively. No other regular expressions are supported. | ```json @@ -115,7 +117,7 @@ Parameters: | Attribute | Type | Required | Description | | --------------------- | -------------- | -------- | --------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `tag_name` | string | yes | The name of a tag | | `ref` | string | yes | Create tag using commit SHA, another tag name, or branch name | | `message` | string | no | Creates annotated tag | @@ -172,5 +174,5 @@ Parameters: | Attribute | Type | Required | Description | | ---------- | -------------- | -------- | --------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `tag_name` | string | yes | The name of a tag | diff --git a/doc/api/topics.md b/doc/api/topics.md index ee88a43ff1c..38d99244bb6 100644 --- a/doc/api/topics.md +++ b/doc/api/topics.md @@ -217,7 +217,7 @@ curl --request PUT \ > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80725) in GitLab 14.9. -You must be an administrator to delete a project. +You must be an administrator to delete a project topic. When you delete a project topic, you also delete the topic assignment for projects. ```plaintext @@ -237,3 +237,43 @@ curl --request DELETE \ --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/topics/1" ``` + +## Merge topics + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95501) in GitLab 15.4. + +You must be an administrator to merge a source topic into a target topic. +When you merge topics, you delete the source topic and move all assigned projects to the target topic. + +```plaintext +POST /topics/merge +``` + +Supported attributes: + +| Attribute | Type | Required | Description | +| ----------------- | ------- | ---------------------- | -------------------------- | +| `source_topic_id` | integer | **{check-circle}** Yes | ID of source project topic | +| `target_topic_id` | integer | **{check-circle}** Yes | ID of target project topic | + +Example request: + +```shell +curl --request POST \ + --data "source_topic_id=2&target_topic_id=1" \ + --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/topics/merge" +``` + +Example response: + +```json +{ + "id": 1, + "name": "topic1", + "title": "Topic 1", + "description": null, + "total_projects_count": 0, + "avatar_url": null +} +``` diff --git a/doc/api/version.md b/doc/api/version.md index 7d072e23410..efdf2b8b626 100644 --- a/doc/api/version.md +++ b/doc/api/version.md @@ -1,6 +1,6 @@ --- -stage: Ecosystem -group: Integrations +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/vulnerabilities.md b/doc/api/vulnerabilities.md index 66d0579bacb..c90dc226661 100644 --- a/doc/api/vulnerabilities.md +++ b/doc/api/vulnerabilities.md @@ -1,5 +1,5 @@ --- -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/vulnerability_exports.md b/doc/api/vulnerability_exports.md index 6f6a661dbd5..59943ede3e6 100644 --- a/doc/api/vulnerability_exports.md +++ b/doc/api/vulnerability_exports.md @@ -1,5 +1,5 @@ --- -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/api/vulnerability_findings.md b/doc/api/vulnerability_findings.md index 46479009d7d..fcfef848f14 100644 --- a/doc/api/vulnerability_findings.md +++ b/doc/api/vulnerability_findings.md @@ -1,5 +1,5 @@ --- -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/architecture/blueprints/_template.md b/doc/architecture/blueprints/_template.md new file mode 100644 index 00000000000..e99ce61970a --- /dev/null +++ b/doc/architecture/blueprints/_template.md @@ -0,0 +1,142 @@ +--- +status: proposed +creation-date: yyyy-mm-dd +authors: [ "@username" ] +coach: "@username" +owning-section: "~section::<section>" +participating-sections: [] +approvers: [ "@product-manager", "@engineering-manager" ] +--- + +<!-- +**Note:** Please remove comment blocks for sections you've filled in. +When your blueprint is complete, all of these comment blocks should be removed. + +To get started with a blueprint you can use this template to inform you about +what you may want to document in it at the beginning. This content will change +/ evolve as you move forward with the proposal. You are not constrained by the +content in this template. If you have a good idea about what should be in your +blueprint, you can ignore the template, but if you don't know yet what should +be in it, this template might be handy. + +- **Fill out this file as best you can.** At minimum, you should fill in the + "Summary", and "Motivation" sections. These can be brief and may be a copy + of issue or epic descriptions if the initiative is already on Product's + roadmap. +- **Create a MR for this blueprint.** Assign it to an Architecture Evolution + Coach (i.e. a Principal+ engineer). +- **Merge early and iterate.** Avoid getting hung up on specific details and + instead aim to get the goals of the blueprint clarified and merged quickly. + The best way to do this is to just start with the high-level sections and fill + out details incrementally in subsequent MRs. + +Just because a blueprint is merged does not mean it is complete or approved. +Any blueprint is a working document and subject to change at any time. + +When editing blueprints, aim for tightly-scoped, single-topic MRs to keep +discussions focused. If you disagree with what is already in a document, open a +new MR with suggested changes. + +If there are new details that belong in the blueprint, edit the blueprint. Once +a feature has become "implemented", major changes should get new blueprints. + +The canonical place for the latest set of instructions (and the likely source +of this file) is [here](/doc/architecture/blueprints/_template.md). +--> + +# {+ Title of Blueprint +} + +<!-- +This is the title of your blueprint. Keep it short, simple, and descriptive. A +good title can help communicate what the blueprint is and should be considered +as part of any review. +--> + +[[_TOC_]] + +## Summary + +<!-- +This section is very important, because very often it is the only section that +will be read by team members. We sometimes call it an "Executive summary", +because executives usually don't have time to read entire document like this. +Focus on writing this section in a way that anyone can understand what is says, +the audience here is everyone: executives, product managers, engineers, wider +community members. + +A good summary is probably at least a paragraph in length. +--> + +## Motivation + +<!-- +This section is for explicitly listing the motivation, goals and non-goals of +this blueprint. Describe why the change is important, all the opportunities, +and the benefits to users. + +The motivation section can optionally provide links to issues that demonstrate +interest in a blueprint within the wider GitLab community. Links to +documentation for competing products and services is also encouraged in cases +where they demonstrate clear gaps in the functionality GitLab provides. + +For concrete proposals we recommend laying out goals and non-goals explicitly, +but this section may be framed in terms of problem statements, challenges, or +opportunities. The latter may be a more suitable framework in cases where the +problem is not well-defined or design details not yet established. +--> + +### Goals + +<!-- +List the specific goals / opportunities of the blueprint. + +- What is it trying to achieve? +- How will we know that this has succeeded? +- What are other less tangible opportunities here? +--> + +### Non-Goals + +<!-- +Listing non-goals helps to focus discussion and make progress. This section is +optional. + +- What is out of scope for this blueprint? +--> + +## Proposal + +<!-- +This is where we get down to the specifics of what the proposal actually is, +but keep it simple! This should have enough detail that reviewers can +understand exactly what you're proposing, but should not include things like +API designs or implementation. The "Design Details" section below is for the +real nitty-gritty. +--> + +## Design and implementation details + +<!-- +This section should contain enough information that the specifics of your +change are understandable. This may include API specs (though not always +required) or even code snippets. If there's any ambiguity about HOW your +proposal will be implemented, this is the place to discuss them. + +If you are not sure how many implementation details you should include in the +blueprint, the rule of thumb here is to provide enough context for people to +understand the proposal. As you move forward with the implementation, you may +need to add more implementation details to the blueprint, as those may become +an important context for important technical decisions made along the way. A +blueprint is also a register of such technical decisions. If a technical +decision requires additional context before it can be made, you probably should +document this context in a blueprint. If it is a small technical decision that +can be made in a merge request by an author and a maintainer, you probably do +not need to document it here. The impact a technical decision will have is +another helpful information - if a technical decision is very impactful, +documenting it, along with associated implementation details, is advisable. + +If it's helpful to include workflow diagrams or any other related images. +Diagrams authored in GitLab flavored markdown are preferred. In cases where +that is not feasible, images should be placed under `images/` in the same +directory as the `index.md` for the proposal. +--> diff --git a/doc/architecture/blueprints/ci_data_decay/index.md b/doc/architecture/blueprints/ci_data_decay/index.md index 7c0bdf299db..23c8e9df1bb 100644 --- a/doc/architecture/blueprints/ci_data_decay/index.md +++ b/doc/architecture/blueprints/ci_data_decay/index.md @@ -48,7 +48,7 @@ PostgreSQL database running on GitLab.com. This volume contributes to significant performance problems, development challenges and is often related to production incidents. -We also expect a [significant growth in the number of builds executed on GitLab.com](../ci_scale/index.md) +We also expect a [significant growth in the number of builds executed on GitLab.com](../ci_scale/index.md) in the upcoming years. ## Opportunity @@ -61,7 +61,7 @@ pipelines that are older than a few months might help us to move this data out of the primary database, to a different storage, that is more performant and cost effective. -It is already possible to prevent processing builds +It is already possible to prevent processing builds [that have been archived](../../../user/admin_area/settings/continuous_integration.md#archive-jobs). When a build gets archived it will not be possible to retry it, but we still do keep all the processing metadata in the database, and it consumes resources @@ -232,7 +232,7 @@ In progress. ## Timeline -- 2021-01-21: Parent [CI Scaling](../ci_scale/) blueprint [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52203) created. +- 2021-01-21: Parent [CI Scaling](../ci_scale/index.md) blueprint [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52203) created. - 2021-04-26: CI Scaling blueprint approved and merged. - 2021-09-10: CI/CD data time decay blueprint discussions started. - 2022-01-07: CI/CD data time decay blueprint [merged](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70052). diff --git a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md index 868dae4fc6c..baec14e3f0f 100644 --- a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md +++ b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md @@ -74,7 +74,12 @@ violates our [principle of 100 GB max size](../database_scaling/size-limits.md). We also want to [build alerting](https://gitlab.com/gitlab-com/gl-infra/tamland/-/issues/5) to notify us when this number is exceeded. -We’ve seen numerous S1 and S2 database-related production environment +Large SQL tables increase index maintenance time, during which freshly deleted tuples +cannot be cleaned by `autovacuum`. This highlight the need for small tables. +We will measure how much bloat we accumulate when (re)indexing huge tables. Base on this analysis, +we will be able to set up SLO (dead tuples / bloat), associated with (re)indexing. + +We've seen numerous S1 and S2 database-related production environment incidents, over the last couple of months, for example: - S1: 2022-03-17 [Increase in writes in `ci_builds` table](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/6625) @@ -130,7 +135,7 @@ remaining database tables when it becomes necessary. It is also important to avoid large data migrations. We store almost 6 terabytes of data in the biggest CI/CD tables, in many different columns and indexes. Migrating this amount of data might be challenging and could cause -instability in the production environment. Due to this concern, we’ve developed +instability in the production environment. Due to this concern, we've developed a way to attach an existing database table as a partition zero without downtime and excessive database locking, what has been demonstrated in one of the [first proofs of concept](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80186). @@ -145,7 +150,7 @@ Our plan is to use logical partition IDs. We want to start with the `ci_pipelines` table and create a `partition_id` column with a `DEFAULT` value of `100` or `1000`. Using a `DEFAULT` value avoids the challenge of backfilling this value for every row. Adding a `CHECK` constraint prior to attaching the -first partition tells PostgreSQL that we’ve already ensured consistency and +first partition tells PostgreSQL that we've already ensured consistency and there is no need to check it while holding an exclusive table lock when attaching this table as a partition to the routing table (partitioned schema definition). We will increment this value every time we create a new partition @@ -159,21 +164,32 @@ and artifacts, will share the same value. We want to add the `partition_id` column into all 6 problematic tables because we can avoid backfilling this data when we decide it is time to start partitioning them. -We want to partition CI/CD data iteratively, so we will start with the -pipelines table, and create at least one, but likely two, partitions. The -pipelines table will be partitioned using the `LIST` partitioning strategy. It -is possible that, after some time, `p_ci_pipelines` will store data in two -partitions with IDs of `100` and `101`. Then we will try partitioning -`ci_builds`. Therefore we might want to use `RANGE` partitioning in -`p_ci_builds` with IDs `100` and `101`, because builds for the two logical -partitions used will still be stored in a single table. +We want to partition CI/CD data iteratively. We plan to start with the +`ci_builds_metadata` table, because this is the fastest growing table in the CI +database and want to contain this rapid growth. This table has also the most +simple access patterns - a row from it is being read when a build is exposed to +a runner, and other access patterns are relatively simple too. Starting with +`p_ci_builds_metadata` will allow us to achieve tangible and quantifiable +results earlier, and will become a new pattern that makes partitioning the +largest table possible. We will partition builds metadata using the `LIST` +partitioning strategy. + +Once we have many partitions attached to `p_ci_builds_metadata`, with many +`partition_ids` we will choose another CI table to partition next. In that case +we might want to use `RANGE` partitioning in for that next table because +`p_ci_builds_metadata` will already have many physical partitions, and +therefore many logical `partition_ids` will be used at that time. For example, +if we choose `ci_builds` as the next partitioning candidate, after having +partitioned `p_ci_builds_metadata`, it will have many different values stored +in `ci_builds.partition_id`. Using `RANGE` partitioning in that case might be +easier. Physical partitioning and logical partitioning will be separated, and a -strategy will be determined when we implement partitioning for the respective -database tables. Using `RANGE` partitioning works similarly to using `LIST` -partitioning in database tables other than `ci_pipelines`, but because we can -guarantee continuity of `partition_id` values, using `RANGE` partitioning might -be a better strategy. +strategy will be determined when we implement physical partitioning for the +respective database tables. Using `RANGE` partitioning works similarly to using +`LIST` partitioning in database tables, but because we can guarantee continuity +of `partition_id` values, using `RANGE` partitioning might be a better +strategy. ## Why do we want to use explicit logical partition ids? @@ -201,9 +217,30 @@ find this number, though we might not need to do this. The single and uniform `partition_id` value for pipeline data gives us more choices later on than primary-keys-based partitioning. +## Altering partitioned tables + +It will still be possible to run `ALTER TABLE` statements against partitioned tables, +similarly to how the tables behaved before partitioning. When PostgreSQL runs +an `ALTER TABLE` statement against a parent partitioned table, it acquires the same +lock on all child partitions and updates each to keep them in sync. This differs from +running `ALTER TABLE` on a non-partitioned table in a few key ways: + +- PostgreSQL acquires `ACCESS EXCLUSIVE` locks against a larger number of tables, but + not a larger amount of data, than it would were the table not partitioned. + Each partition will be locked similarly to the parent table, and all will be updated + in a single transaction. +- Lock duration will be increased based on the number of partitions involved. + All `ALTER TABLE` statements executed on the GitLab database (other than `VALIDATE CONSTRAINT`) + take small constant amounts of time per table modified. PostgreSQL will need + to modify each partition in sequence, increasing the runtime of the lock. This + time will still remain very small until there are many partitions involved. +- If thousands of partitions are involved in an `ALTER TABLE`, we will need to verify that + the value of `max_locks_per_transaction` is high enough to support all of the locks that + need to be taken during the operation. + ## Splitting large partitions into smaller ones -We want to start with the initial `pipeline_id` number `100` (or higher, like +We want to start with the initial `partition_id` number `100` (or higher, like `1000`, depending on our calculations and estimations). We do not want to start from 1, because existing tables are also large already, and we might want to split them into smaller partitions. If we start with `100`, we will be able to @@ -217,6 +254,18 @@ smaller ones (it's not yet clear if we will need to do this), we might be able to just use background migrations to update partition IDs, and PostgreSQL is smart enough to move rows between partitions on its own. +### Naming conventions + +A partitioned table is called a **routing** table and it will use the `p_` +prefix which should help us with building automated tooling for query analysis. + +A table partition will be simply called **partition** and it can use the a +physical partition ID as suffix, leaded by a `p` letter, for example +`ci_builds_p101`. Existing CI tables will become **zero partitions** of the +new routing tables. Depending on the chosen +[partitioning strategy](#how-do-we-want-to-partition-cicd-data) for a given +table, it is possible to have many logical partitions per one physical partition. + ## Storing partitions metadata in the database In order to build an efficient mechanism that will be responsible for creating @@ -225,8 +274,8 @@ metadata table, called `ci_partitions`. In that table we would store metadata about all the logical partitions, with many pipelines per partition. We may need to store a range of pipeline ids per logical partition. Using it we will be able to find the `partition_id` number for a given pipeline ID and we will -also find information about which logical partitions are “active” or -“archived”, which will help us to implement a time-decay pattern using database +also find information about which logical partitions are "active" or +"archived", which will help us to implement a time-decay pattern using database declarative partitioning. `ci_partitions` table will store information about a partition identifier, @@ -302,9 +351,116 @@ scope block takes an argument). Preloading instance dependent scopes is not supported. ``` -We also need to build a proof of concept for removing data on the PostgreSQL -side (using foreign keys with `ON DELETE CASCADE`) and removing data through -Rails associations, as this might be an important area of uncertainty. +### Foreign keys + +Foreign keys must reference columns that either are a primary key or form a +unique constraint. We can define them using these strategies: + +#### Between routing tables sharing partition ID + +For relations that are part of the same pipeline hierarchy it is possible to +share the `partition_id` column to define the foreign key constraint: + +```plaintext +p_ci_pipelines: + - id + - partition_id + +p_ci_builds: + - id + - partition_id + - pipeline_id +``` + +In this case, `p_ci_builds.partition_id` indicates the partition for the build +and also for the pipeline. We can add a FK on the routing table using: + +```sql +ALTER TABLE ONLY p_ci_builds + ADD CONSTRAINT fk_on_pipeline_and_partition + FOREIGN KEY (pipeline_id, partition_id) + REFERENCES p_ci_pipelines(id, partition_id) ON DELETE CASCADE; +``` + +#### Between routing tables with different partition IDs + +It's not possible to reuse the `partition_id` for all relations in the CI domain, +so in this case we'll need to store the value as a different attribute. For +example, when canceling redundant pipelines we store on the old pipeline row +the ID of the new pipeline that cancelled it as `auto_canceled_by_id`: + +```plaintext +p_ci_pipelines: + - id + - partition_id + - auto_canceled_by_id + - auto_canceled_by_partition_id +``` + +In this case we can't ensure that the canceling pipeline is part of the same +hierarchy as the canceled pipelines, so we need an extra attribute to store its +partition, `auto_canceled_by_partition_id`, and the FK becomes: + +```sql +ALTER TABLE ONLY p_ci_pipelines + ADD CONSTRAINT fk_cancel_redundant_pieplines + FOREIGN KEY (auto_canceled_by_id, auto_canceled_by_partition_id) + REFERENCES p_ci_pipelines(id, partition_id) ON DELETE SET NULL; +``` + +#### Between routing tables and regular tables + +Not all of the tables in the CI domain will be partitioned, so we'll have routing +tables that will reference non-partitioned tables, for example we reference +`external_pull_requests` from `ci_pipelines`: + +```sql +FOREIGN KEY (external_pull_request_id) +REFERENCES external_pull_requests(id) +ON DELETE SET NULL +``` + +In this case we only need to move the FK definition from the partition level +to the routing table so that new pipeline partitions may use it: + +```sql +ALTER TABLE p_ci_pipelines + ADD CONSTRAINT fk_external_request + FOREIGN KEY (external_pull_request_id) + REFERENCES external_pull_requests(id) ON DELETE SET NULL; +``` + +#### Between regular tables and routing tables + +Most of the tables from the CI domain reference at least one table that will be +turned into a routing tables, for example `ci_pipeline_messages` references +`ci_pipelines`. These definitions will need to be updated to use the routing +tables and for this they will need a `partition_id` column: + +```plaintext +p_ci_pipelines: + - id + - partition_id + +ci_pipeline_messages: + - id + - pipeline_id + - pipeline_partition_id +``` + +The foreign key can be defined by using: + +```sql +ALTER TABLE ci_pipeline_messages ADD CONSTRAINT fk_pipeline_partitioned + FOREIGN KEY (pipeline_id, pipeline_partition_id) + REFERENCES p_ci_pipelines(id, partition_id) ON DELETE CASCADE; +``` + +The old FK definition will need to be removed, otherwise new inserts in the +`ci_pipeline_messages` with pipeline IDs from non-zero partition will fail with +reference errors. + +### Indexes We [learned](https://gitlab.com/gitlab-org/gitlab/-/issues/360148) that `PostgreSQL` does not allow to create a single index (unique or otherwise) across all partitions of a table. @@ -465,7 +621,7 @@ strategy. The strategy, described in this document, is subject to iteration as well. Whenever we find a better way to reduce the risk and improve our plan, we should update this document as well. -We’ve managed to find a way to avoid large-scale data migrations, and we are +We've managed to find a way to avoid large-scale data migrations, and we are building an iterative strategy for partitioning CI/CD data. We documented our strategy here to share knowledge and solicit feedback from other team members. diff --git a/doc/architecture/blueprints/ci_pipeline_components/index.md b/doc/architecture/blueprints/ci_pipeline_components/index.md new file mode 100644 index 00000000000..94ec3e2f894 --- /dev/null +++ b/doc/architecture/blueprints/ci_pipeline_components/index.md @@ -0,0 +1,209 @@ +--- +stage: Stage +group: Pipeline Authoring +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +comments: false +description: 'Create a catalog of shareable pipeline constructs' +--- + + +# CI/CD pipeline components catalog + +## Summary + +## Goals + +The goal of the CI/CD pipeline components catalog is to make the reusing pipeline configurations +easier and more efficient. +Providing a way to discover, understand and learn how to reuse pipeline constructs allows for a more streamlined experience. +Having a CI/CD pipeline components catalog also sets a framework for users to collaborate on pipeline constructs so that they can be evolved +and improved over time. + +This blueprint defines the architectural guidelines on how to build a CI/CD catalog of pipeline components. +This blueprint also defines the long-term direction for iterations and improvements to the solution. + +## Challenges + +- GitLab CI/CD can have a steep learning curve for new users. Users must read the documentation and + [YAML reference](../../../ci/yaml/index.md) to understand how to configure their pipelines. +- Developers are struggling to reuse existing CI/CD templates with the result of having to reinvent the wheel and write + YAML configurations repeatedly. +- GitLab [CI templates](../../../development/cicd/templates.md#template-directories) provide users with + scaffolding pipeline or jobs for specific purposes. + However versioning them is challenging today due to being shipped with the GitLab instance. + See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/17716) for more information. +- Users of GitLab CI/CD (pipeline authors) today have their own ad-hoc way to organize shared pipeline + configurations inside their organization. Those configurations tend to be mostly undocumented. +- The only discoverable configurations are GitLab CI templates. However they don't have any inline documentation + so it becomes harder to know what they do and how to use them without copy-pasting the content in the + editor and read the actual YAML. +- It's harder to adopt additional GitLab features (CD, security, test, etc.). +- There is no framework for testing reusable CI configurations. + Many configurations are not unit tested against single changes. +- Communities, partners, 3rd parties, individual contributors, must go through the + [GitLab Contribution process](https://about.gitlab.com/community/contribute/) to contribute to GitLab managed + templates. See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/323727) for more information. +- GitLab has more than 100 of templates with some of them barely maintained after their addition. + +### Problems with GitLab CI templates + +- GitLab CI Templates have not been designed with deterministic behavior in mind. +- GitLab CI Templates have not been design with reusability in mind. +- `Jobs/` templates hard-code the `stage:` attribute but the user of the template must somehow override + or know in advance what stage is needed. + - The user should be able to import the job inside a given stage or pass the stage names as input parameter + when using the component. + - Failures in mapping the correct stage can result in confusing errors. +- Some templates are designed to work with AutoDevops but are not generic enough + ([example](https://gitlab.com/gitlab-org/gitlab/-/blob/2c0e8e4470001442e999391df81e19732b3439e6/lib/gitlab/ci/templates/AWS/Deploy-ECS.gitlab-ci.yml)). +- Many CI templates, especially those [language specific](https://gitlab.com/gitlab-org/gitlab/-/tree/2c0e8e4470001442e999391df81e19732b3439e6/lib/gitlab/ci/templates) + are tutorial/scaffolding-style templates. + - They are meant to show the user how a typical pipeline would look like but it requires high customization from the user perspective. + - They require a different UX: copy-paste in the position of the Pipeline Editor cursor. +- Some templates like `SAST.latest.gitlab-ci.yml` add multiple jobs conditionally to the same pipeline. + - Ideally these jobs could run as a child pipeline and make the reports available to the parent pipeline. + - [This epic](https://gitlab.com/groups/gitlab-org/-/epics/8205) is necessary for Parent-child pipelines to be used. +- Some templates incorrectly use `variables`, `image` and other top-level keywords but that defines them in all pipeline jobs, + not just those defined in the template. + - This technique introduces inheritance issues when a template modifies jobs unnecessarily. + +## Opportunities + +- Having a catalog of pipeline constructs where users can search and find what they need can greatly lower + the bar for new users. +- Customers are already trying to rollout their ad-hoc catalog of shared configurations. We could provide a + standardized way to write, package and share pipeline constructs directly in the product. +- As we implement new pipeline constructs (for example, reusable job steps) they could be items of the + catalog. The catalog can boost the adoption of new constructs. +- The catalog can be a place where we strengthen our relationship with partners, having components offered + and maintained by our partners. +- With discoverability and better versioning mechanism we can have more improvements and better collaboration. +- Competitive landscape is showing the need for such feature + - [R2DevOps](https://r2devops.io) implements a catalog of CI templates for GitLab pipelines. + - [GitHub Actions](https://github.com/features/actions) provides an extensive catalog of reusable job steps. + +## Implementation guidelines + +- Start with the smallest user base. Dogfood the feature for `gitlab-org` and `gitlab-com` groups. + Involve the Engineering Productivity and other groups authoring pipeline configurations to test + and validate our solutions. +- Ensure we can integrate all the feedback gathered, even if that means changing the technical design or + UX. Until we make the feature GA we should have clear expectations with early adopters. +- Reuse existing functionality as much as possible. Don't reinvent the wheel on the initial iterations. + For example: reuse project features like title, description, avatar to build a catalog. +- Leverage GitLab features for the development lifecycle of the components (testing via `.gitlab-ci.yml`, + release management, Pipeline Editor, etc.). +- Design the catalog with self-managed support in mind. +- Allow the catalog an the workflow to support future types of pipeline constructs and new ways of using them. +- Design components and catalog following industry best practice related to building deterministic package managers. + +## Glossary + +This section defines some terms that are used throughout this document. With these terms we are only +identifying abstract concepts and are subject to changes as we refine the design by discovering new insights. + +- **Component** Is the reusable unit of pipeline configuration. +- **Project** Is the GitLab project attached to a repository. A project can contain multiple components. +- **Catalog** is the collection of projects that are set to contain components. +- **Version** is the release name of a tag in the project, which allows components to be pinned to a specific revision. + +## Characteristics of a component + +For best experience with any systems made of components it's fundamental that components are single purpose, +isolated, reusable and resolvable. + +- **Single purpose**: a component must focus on a single goal and the scope be as small as possible. +- **Isolation**: when a component is used in a pipeline, its implementation details should not leak outside the + component itself and into the main pipeline. +- **Reusability:** a component is designed to be used in different pipelines. + Depending on the assumptions it's built on a component can be more or less generic. + Generic components are more reusable but may require more customization. +- **Resolvable:** When a component depends on another component, this dependency needs to be explicit and trackable. Hidden dependencies can lead to myriads of problems. + +## Proposal + +Prerequisites to create a component: + +- Create a project. Description and avatar are highly recommended to improve discoverability. +- Add a `README.md` in the top level directory that documents the component. + What it does, how to use it, how to contribute, etc. + This file is mandatory. +- Add a `.gitlab-ci.yml` in the top level directory to test that the components works as expected. + This file is highly recommended. + +Characteristics of a component: + +- It must have a **name** to be referenced to and **description** for extra details. +- It must specify its **type** which defines how it can be used (raw configuration to be `include`d, child pipeline workflow, job step). +- It must define its **content** based on the type. +- It must specify **input parameters** that it accepts. Components should depend on input parameters for dynamic values and not environment variables. +- It can optionally define **output data** that it returns. +- Its YAML specification should be **validated statically** (for example: using JSON schema validators). +- It should be possible to use specific **versions** of a component by referencing official releases and SHA. +- It should be possible to use components defined locally in the same repository. + +## Limits + +Any MVC that exposes a feature should be added with limitations from the beginning. +It's safer to add new features with restrictions than trying to limit a feature after it's being used. +We can always soften the restrictions later depending on user demand. + +Some limits we could consider adding: + +- number of components that a single project can contain/export +- number of imports that a `.gitlab-ci.yml` file can use +- number of imports that a component can declare/use +- max level of nested imports +- max length of the exported component name + +## Iterations + +1. Experimentation phase + - Build an MVC behind a feature flag with `namespace` actor. + - Enable the feature flag only for `gitlab-com` and `gitlab-org` namespaces to initiate the dogfooding. + - Refine the solution and UX based on feedback. + - Find customers to be early adopters of this feature and iterate on their feedback. +1. Design new pipeline constructs (in parallel with other phases) + - Start the technical and design process to work on proposals for new pipeline constructs (steps, workflows, templates). + - Implement new constructs. The catalog must be compatible with them. + - Dogfood new constructs and iterate on feedback. + - Release new constructs on private catalogs. +1. Release the private catalog for groups on Ultimate plan. + - Iterate on feedback. +1. Release the public catalog for all GitLab users (prospect feature) + - Publish new versions of GitLab CI templates as components using the new constructs whenever possible. + - Allow self-managed administrators to populate their self-managed catalog by importing/updating + components from GitLab.com or from repository exports. + - Iterate on feedback. + +## Who + +Proposal: + +<!-- vale gitlab.Spelling = NO --> + +| Role | Who +|------------------------------|-------------------------| +| Author | Fabio Pitino | +| Engineering Leader | ? | +| Product Manager | Dov Hershkovitch | +| Architecture Evolution Coach | Kamil Trzciński | + +DRIs: + +| Role | Who +|------------------------------|------------------------| +| Leadership | ? | +| Product | Dov Hershkovitch | +| Engineering | ? | +| UX | Nadia Sotnikova | + +Domain experts: + +| Area | Who +|------------------------------|------------------------| +| Verify / Pipeline authoring | Avielle Wolfe | +| Verify / Pipeline authoring | Furkan Ayhan | +| Verify / Pipeline execution | Fabio Pitino | + +<!-- vale gitlab.Spelling = YES --> diff --git a/doc/architecture/blueprints/ci_scale/index.md b/doc/architecture/blueprints/ci_scale/index.md index 5822ae2b5ed..75c4d05c334 100644 --- a/doc/architecture/blueprints/ci_scale/index.md +++ b/doc/architecture/blueprints/ci_scale/index.md @@ -115,13 +115,13 @@ of the CI/CD Apdex score, and sometimes even causes a significant performance degradation in the production environment. There are multiple other strategies that can improve performance and -reliability. We can use [Redis queuing](https://gitlab.com/gitlab-org/gitlab/-/issues/322972), or +reliability. We can use [Redis queuing](https://gitlab.com/gitlab-org/gitlab/-/issues/322972), or [a separate table that will accelerate SQL queries used to build queues](https://gitlab.com/gitlab-org/gitlab/-/issues/322766) and we want to explore them. -**Status**: As of October 2021 the new architecture +**Status**: As of October 2021 the new architecture [has been implemented on GitLab.com](https://gitlab.com/groups/gitlab-org/-/epics/5909#note_680407908). -The following epic tracks making it generally available: +The following epic tracks making it generally available: [Make the new pending builds architecture generally available](https://gitlab.com/groups/gitlab-org/-/epics/6954). ### Moving big amounts of data is challenging @@ -171,7 +171,7 @@ Work required to achieve our next CI/CD scaling target is tracked in the 1. ✓ Migrate primary keys to big integers on GitLab.com. 1. ✓ Implement the new architecture of builds queuing on GitLab.com. 1. [Make the new builds queuing architecture generally available](https://gitlab.com/groups/gitlab-org/-/epics/6954). -1. [Partition CI/CD data using time-decay pattern](../ci_data_decay/). +1. [Partition CI/CD data using time-decay pattern](../ci_data_decay/index.md). ## Status diff --git a/doc/architecture/blueprints/cloud_native_build_logs/index.md b/doc/architecture/blueprints/cloud_native_build_logs/index.md index 0c941e332cb..3a06d73141b 100644 --- a/doc/architecture/blueprints/cloud_native_build_logs/index.md +++ b/doc/architecture/blueprints/cloud_native_build_logs/index.md @@ -12,7 +12,7 @@ Cloud native and the adoption of Kubernetes has been recognised by GitLab to be one of the top two biggest tailwinds that are helping us grow faster as a company behind the project. -This effort is described in a more details +This effort is described in a more details [in the infrastructure team handbook](https://about.gitlab.com/handbook/engineering/infrastructure/production/kubernetes/gitlab-com/). ## Traditional build logs @@ -88,7 +88,7 @@ even tried to replace NFS with Since that time it has become apparent that the cost of operations and maintenance of a NFS cluster is significant and that if we ever decide to -migrate to Kubernetes +migrate to Kubernetes [we need to decouple GitLab from a shared local storage and NFS](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/426#note_375646396). 1. NFS might be a single point of failure @@ -113,7 +113,7 @@ of complexity, maintenance cost and enormous, negative impact on availability. The work needed to make the new architecture production ready and enabled on GitLab.com had been tracked in [Cloud Native Build Logs on GitLab.com](https://gitlab.com/groups/gitlab-org/-/epics/4275) epic. -Enabling this feature on GitLab.com is a subtask of +Enabling this feature on GitLab.com is a subtask of [making the new architecture generally available](https://gitlab.com/groups/gitlab-org/-/epics/3791) for everyone. ## Status diff --git a/doc/architecture/blueprints/cloud_native_gitlab_pages/index.md b/doc/architecture/blueprints/cloud_native_gitlab_pages/index.md index 89c3a4cd6b4..431bc19ad84 100644 --- a/doc/architecture/blueprints/cloud_native_gitlab_pages/index.md +++ b/doc/architecture/blueprints/cloud_native_gitlab_pages/index.md @@ -17,7 +17,7 @@ Cloud Native and the adoption of Kubernetes has been recognised by GitLab to be one of the top two biggest tailwinds that are helping us grow faster as a company behind the project. -This effort is described in more detail +This effort is described in more detail [in the infrastructure team handbook page](https://about.gitlab.com/handbook/engineering/infrastructure/production/kubernetes/gitlab-com/). GitLab Pages is tightly coupled with NFS and in order to unblock Kubernetes @@ -55,7 +55,7 @@ even tried to replace NFS with Since that time it has become apparent that the cost of operations and maintenance of a NFS cluster is significant and that if we ever decide to -migrate to Kubernetes +migrate to Kubernetes [we need to decouple GitLab from a shared local storage and NFS](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/426#note_375646396). 1. NFS might be a single point of failure @@ -83,7 +83,7 @@ graph TD C -- Serves static content --> E(Visitors) ``` -This new architecture has been briefly described in +This new architecture has been briefly described in [the blog post](https://about.gitlab.com/blog/2020/08/03/how-gitlab-pages-uses-the-gitlab-api-to-serve-content/) too. diff --git a/doc/architecture/blueprints/database_scaling/size-limits.md b/doc/architecture/blueprints/database_scaling/size-limits.md index 284f6402d3c..0bb1ae9efb4 100644 --- a/doc/architecture/blueprints/database_scaling/size-limits.md +++ b/doc/architecture/blueprints/database_scaling/size-limits.md @@ -138,7 +138,7 @@ There is no standard solution to reduce table sizes - there are many! 1. **Retention**: Delete unnecessary data, for example expire old and unneeded records. 1. **Remove STI**: We still use [single-table inheritance](../../../development/database/single_table_inheritance.md) in a few places, which is considered an anti-pattern. Redesigning this, we can split data into multiple tables. 1. **Index optimization**: Drop unnecessary indexes and consolidate overlapping indexes if possible. -1. **Optimise data types**: Review data type decisions and optimise data types where possible (example: use integer instead of text for an enum column) +1. **Optimize data types**: Review data type decisions and optimize data types where possible (example: use integer instead of text for an enum column) 1. **Partitioning**: Apply a partitioning scheme if there is a common access dimension. 1. **Normalization**: Review relational modeling and apply normalization techniques to remove duplicate data 1. **Vertical table splits**: Review column usage and split table vertically. diff --git a/doc/architecture/blueprints/feature_flags_development/index.md b/doc/architecture/blueprints/feature_flags_development/index.md index eaca7da6bd7..08253ac883c 100644 --- a/doc/architecture/blueprints/feature_flags_development/index.md +++ b/doc/architecture/blueprints/feature_flags_development/index.md @@ -23,7 +23,7 @@ The extensive usage of feature flags poses a few challenges - Each feature flag that we add to codebase is a ~"technical debt" as it adds a matrix of configurations. - Testing each combination of feature flags is close to impossible, so we - instead try to optimise our testing of feature flags to the most common + instead try to optimize our testing of feature flags to the most common scenarios. - There's a growing challenge of maintaining a growing number of feature flags. We sometimes forget how our feature flags are configured or why we haven't @@ -115,8 +115,8 @@ These are reason why these changes are needed: ## Iterations -This work is being done as part of dedicated epic: -[Improve internal usage of Feature Flags](https://gitlab.com/groups/gitlab-org/-/epics/3551). +This work is being done as part of dedicated epic: +[Improve internal usage of Feature Flags](https://gitlab.com/groups/gitlab-org/-/epics/3551). This epic describes a meta reasons for making these changes. ## Who diff --git a/doc/architecture/blueprints/graphql_api/index.md b/doc/architecture/blueprints/graphql_api/index.md index eb045de491e..1ee322c412b 100644 --- a/doc/architecture/blueprints/graphql_api/index.md +++ b/doc/architecture/blueprints/graphql_api/index.md @@ -44,11 +44,11 @@ It is an opportunity to learn from our experience in evolving the REST API, for the scale, and to apply this knowledge onto the GraphQL development efforts. We can do that by building query-to-feature correlation mechanisms, adding scalable state synchronization support and aligning GraphQL with other -architectural initiatives being executed in parallel, like +architectural initiatives being executed in parallel, like [the support for direct uploads](https://gitlab.com/gitlab-org/gitlab/-/issues/280819). GraphQL should be secure by default. We can avoid common security mistakes by -building mechanisms that will help us to enforce +building mechanisms that will help us to enforce [OWASP GraphQL recommendations](https://cheatsheetseries.owasp.org/cheatsheets/GraphQL_Cheat_Sheet.html) that are relevant to us. diff --git a/doc/architecture/blueprints/object_storage/index.md b/doc/architecture/blueprints/object_storage/index.md index b70339c8b8d..7a4ecd0e5a8 100644 --- a/doc/architecture/blueprints/object_storage/index.md +++ b/doc/architecture/blueprints/object_storage/index.md @@ -31,8 +31,8 @@ underlying implementation for shared, distributed, highly-available (HA) file storage. Over time, we have built support for object storage across the -application, solving specific problems in a -[multitude of iterations](https://about.gitlab.com/company/team/structure/working-groups/object-storage/#company-efforts-on-uploads). +application, solving specific problems in a +[multitude of iterations](https://about.gitlab.com/company/team/structure/working-groups/object-storage/#company-efforts-on-uploads). This has led to increased complexity across the board, from development (new features and bug fixes) to installation: @@ -67,7 +67,7 @@ This has led to increased complexity across the board, from development The following is a brief description of the main directions we can take to remove the pain points affecting our object storage implementation. -This is also available as [a YouTube video](https://youtu.be/X9V_w8hsM8E) recorded for the +This is also available as [a YouTube video](https://youtu.be/X9V_w8hsM8E) recorded for the [Object Storage Working Group](https://about.gitlab.com/company/team/structure/working-groups/object-storage/). ### Simplify GitLab architecture by shipping MinIO @@ -78,7 +78,7 @@ local storage and object storage. With local storage, there is the assumption of a shared storage between components. This can be achieved by having a single box -installation, without HA, or with a NFS, which +installation, without HA, or with a NFS, which [we no longer recommend](../../../administration/nfs.md). We have a testing gap on object storage. It also requires Workhorse @@ -134,7 +134,7 @@ access to new features without infrastructure chores. Our implementation is built on top of a 3rd-party framework where every object storage client is a 3rd-party library. Unfortunately some -of them are unmaintained. +of them are unmaintained. [We have customers who cannot push 5GB Git LFS objects](https://gitlab.com/gitlab-org/gitlab/-/issues/216442), but with such a vital feature implemented in 3rd-party libraries we are slowed down in fixing it, and we also rely on external maintainers @@ -214,7 +214,7 @@ Proposal: DRIs: -The DRI for this blueprint is the +The DRI for this blueprint is the [Object Storage Working Group](https://about.gitlab.com/company/team/structure/working-groups/object-storage/). <!-- vale gitlab.Spelling = YES --> diff --git a/doc/architecture/blueprints/pods/index.md b/doc/architecture/blueprints/pods/index.md new file mode 100644 index 00000000000..fc33a4f441b --- /dev/null +++ b/doc/architecture/blueprints/pods/index.md @@ -0,0 +1,162 @@ +--- +stage: enablement +group: pods +comments: false +description: 'Pods' +--- + +# Pods + +DISCLAIMER: +This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc. + +This document is a work-in-progress and represents a very early state of the Pods design. Significant aspects are not documented, though we expect to add them in the future. + +## Summary + +Pods is a new architecture for our Software as a Service platform that is horizontally-scalable, resilient, and provides a more consistent user experience. It may also provide additional features in the future, such as data residency control (regions) and federated features. + +## Terminology + +We use the following terms to describe components and properties of the Pods architecture. + +### Pod + +A Pod is a set of infrastructure components that contains multiple workspaces that belong to different organizations. The components include both datastores (PostgreSQL, Redis etc.) and stateless services (web etc.). The infrastructure components provided within a Pod are shared among workspaces but not shared with other Pods. This isolation of infrastructure components means that Pods are independent from each other. + +#### Pod properties + +- Each pod is independent from the others +- Infrastructure components are shared by workspaces within a Pod +- More Pods can be provisioned to provide horizontal scalability +- A failing Pod does not lead to failure of other Pods +- Noisy neighbor effects are limited to within a Pod +- Pods are not visible to organizations; it is an implementation detail +- Pods may be located in different geographical regions (for example, EU, US, JP, UK) + +Discouraged synonyms: GitLab instance, cluster, shard + +### Workspace + +A [workspace](../../../user/workspace/index.md) is the name for the top-level namespace that is used by organizations to manage everything GitLab. It will provide similar administrative capabilities to a self-managed instance. + +See more in the [workspace group overview](https://about.gitlab.com/direction/manage/workspace/#overview). + +#### Workspace properties + +- Workspaces are isolated from each other by default +- A workspace is located on a single Pod +- Workspaces share the resources provided by a Pod + +### Top-Level namespace + +A top-level namespace is the logical object container in the code that represents all groups, subgroups and projects that belong to an organization. + +A top-level namespace is the root of nested collection namespaces and projects. The namespace and its related entities form a tree-like hierarchy: Namespaces are the nodes of the tree, projects are the leaves. An organization usually contains a single top-level namespace, called a workspace. + +Example: + +`https://gitlab.com/gitlab-org/gitlab/`: + +- `gitlab-org` is a `top-level namespace`; the root for all groups and projects of an organization +- `gitlab` is a `project`; a project of the organization. + +Discouraged synonyms: Root-level namespace + +#### Top-level namespace properties + +Same as workspaces. + +### Users + +Users are available globally and not restricted to a single Pod. Users can create multiple workspaces and they may be members of several workspaces and contribute to them. Because users' activity is not limited to an individual Pod, their activity needs to be aggregated across Pods to reflect all their contributions (for example TODOs). This means, the Pods architecture may need to provide a central dashboard. + +#### User properties + +- Users are shared globally across all Pods +- Users can create multiple workspaces +- Users can be a member of multiple workspaces + +## Goals + +### Scalability + +The main goal of this new shared-infrastructure architecture is to provide additional scalability for our SaaS Platform. GitLab.com is largely monolithic and we have estimated (internal) that the current architecture has scalability limitations, even when database partitioning and decomposition are taken into account. + +Pods provide a horizontally scalable solution because additional Pods can be created based on demand. Pods can be provisioned and tuned as needed for optimal scalability. + +### Increased availability + +A major challenge for shared-infrastructure architectures is a lack of isolation between workspaces. This can lead to noisy neighbor effects. A organization's behavior inside a workspace can impact all other workspaces. This is highly undesirable. Pods provide isolation at the pod level. A group of organizations is fully isolated from other organizations located on a different Pod. This minimizes noisy neighbor effects while still benefiting from the cost-efficiency of shared infrastructure. + +Additionally, Pods provide a way to implement disaster recovery capabilities. Entire Pods may be replicated to read-only standbys with automatic failover capabilities. + +### A consistent experience + +Organizations should have the same user experience on our SaaS platform as they do on a self-managed GitLab instance. + +### Regions + +GitLab.com is only hosted within the United States of America. Organizations located in other regions have voiced demand for local SaaS offerings. Pods provide a path towards [GitLab Regions](https://gitlab.com/groups/gitlab-org/-/epics/6037) because Pods may be deployed within different geographies. Depending on which of the organization's data is located outside a Pod, this may solve data residency and compliance problems. + +## Market segment + +Pods would provide a solution for organizations in the small to medium business (up to 100 users) and the mid-market segment (up to 2000 users). +(See [segmentation definitions](https://about.gitlab.com/handbook/sales/field-operations/gtm-resources/#segmentation).) +Larger organizations may benefit substantially from [GitLab Dedicated](../../../subscriptions/gitlab_dedicated/index.md). + +## High-level architecture problems to solve + +A number of technical issues need to be resolved to implement Pods (in no particular order). This section will be expanded. + +1. How are users of an organization routed to the correct Pod containing their workspace? +1. How do users authenticate? +1. How are Pods rebalanced? +1. How are Pods provisioned? +1. How can Pods implement disaster recovery capabilities? + +## Iteration 1 + +Ultimately, a Pods architecture should offer the same user experience as self-managed and GitLab dedicated. However, at this moment GitLab.com has many more "social-network"-like capabilities that will be difficult to implement with a Pods architecture. We should evaluate if the SMB and mid market segment is interested in these features, or if not having them is acceptable in most cases. + +The first iteration of Pods will still contain some limitations that would break cross-workspace workflows. This means it may only be acceptable for new customers, or for existing customers that are briefed. + +Limitations are: + +- An organization can create only a single workspace. +- Workspaces are isolated from each other. This means cross-workspace workflows are broken. + +## Iteration 2 + +Based on user research, we may want to change certain features to work across namespaces to allow organizations to interact with each other in specific circumstances. We may also allow organizations to have more than one workspace. This is particularly relevant for organizations with sub-divisions, or multi-national organizations that want to have workspaces in different regions. + +Additional features: + +- Specific features allow for cross-workspace interactions, for example forking, search. +- An organization can own multiple workspaces on different Pods. + +### Links + +- [Internal Pods presentation](https://docs.google.com/presentation/d/1x1uIiN8FR9fhL7pzFh9juHOVcSxEY7d2_q4uiKKGD44/edit#slide=id.ge7acbdc97a_0_155) +- [Pods Epic](https://gitlab.com/groups/gitlab-org/-/epics/7582) +- [Database Group investigation](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/database/doc/root-namespace-sharding.html) +- [Shopify Pods architecture](https://shopify.engineering/a-pods-architecture-to-allow-shopify-to-scale) +- [Opstrace architecture](https://gitlab.com/gitlab-org/opstrace/opstrace/-/blob/main/docs/architecture/overview.md) + +### Who + +| Role | Who +|------------------------------|-------------------------| +| Author | Fabian Zimmer | +| Architecture Evolution Coach | Kamil Trzciński | +| Engineering Leader | TBD | +| Product Manager | Fabian Zimmer | +| Domain Expert / Database | TBD | + +DRIs: + +| Role | Who +|------------------------------|------------------------| +| Leadership | TBD | +| Product | Fabian Zimmer | +| Engineering | Thong Kuah | diff --git a/doc/architecture/blueprints/rate_limiting/index.md b/doc/architecture/blueprints/rate_limiting/index.md new file mode 100644 index 00000000000..692cef4b11d --- /dev/null +++ b/doc/architecture/blueprints/rate_limiting/index.md @@ -0,0 +1,411 @@ +--- +stage: none +group: unassigned +comments: false +description: 'Next Rate Limiting Architecture' +--- + +# Next Rate Limiting Architecture + +## Summary + +Introducing reasonable application limits is a very important step in any SaaS +platform scaling strategy. The more users a SaaS platform has, the more +important it is to introduce sensible rate limiting and policies enforcement +that will help to achieve availability goals, reduce the problem of noisy +neighbours for users and ensure that they can keep using a platform +successfully. + +This is especially true for GitLab.com. Our goal is to have a reasonable and +transparent strategy for enforcing application limits, which will become a +definition of a responsible usage, to help us with keeping our availability and +user satisfaction at a desired level. + +We've been introducing various application limits for many years already, but +we've never had a consistent strategy for doing it. What we want to build now is +a consistent framework used by engineers and product managers, across entire +application stack, to define, expose and enforce limits and policies. + +Lack of consistency in defining limits, not being able to expose them to our +users, support engineers and satellite services, has negative impact on our +productivity, makes it difficult to introduce new limits and eventually +prevents us from enforcing responsible usage on all layers of our application +stack. + +This blueprint has been written to consolidate our limits and to describe the +vision of our next rate limiting and policies enforcement architecture. + +_Disclaimer: The following contains information related to upcoming products, +features, and functionality._ + +_It is important to note that the information presented is for informational +purposes only. Please do not rely on this information for purchasing or +planning purposes._ + +_As with all projects, the items mentioned in this document and linked pages are +subject to change or delay. The development, release and timing of any +products, features, or functionality remain at the sole discretion of GitLab +Inc._ + +## Goals + +**Implement a next architecture for rate limiting and policies definition.** + +## Challenges + +- We have many ways to define application limits, in many different places. +- It is difficult to understand what limits have been applied to a request. +- It is difficult to introduce new limits, even more to define policies. +- Finding what limits are defined requires performing a codebase audit. +- We don't have a good way to expose limits to satellite services like Registry. +- We enforce a number of different policies via opaque external systems + (Pipeline Validation Service, Bouncer, Watchtower, Cloudflare, Haproxy). +- There is not standardized way to define policies in a way consistent with defining limits. +- It is difficult to understand when a user is approaching a limit threshold. +- There is no way to automatically notify a user when they are approaching thresholds. +- There is no single way to change limits for a namespace / project / user / customer. +- There is no single way to monitor limits through real-time metrics. +- There is no framework for hierarchical limit configuration (instance / namespace / sub-group / project). +- We allow disabling rate-limiting for some marquee SaaS customers, but this + increases a risk for those same customers. We should instead be able to set + higher limits. + +## Opportunity + +We want to build a new framework, making it easier to define limits, quotas and +policies, and to enforce / adjust them in a controlled way, through robust +monitoring capabilities. + +<!-- markdownlint-disable MD029 --> + +1. Build a framework to define and enforce limits in GitLab Rails. +2. Build an API to consume limits in satellite service and expose them to users. +3. Extract parts of this framework into a dedicated GitLab Limits Service. + +<!-- markdownlint-enable MD029 --> + +The most important opportunity here is consolidation happening on multiple +levels: + +1. Consolidate on the application limits tooling used in GitLab Rails. +1. Consolidate on the process of adding and managing application limits. +1. Consolidate on the behavior of hierarchical cascade of limits and overrides. +1. Consolidate on the application limits tooling used across entire application stack. +1. Consolidate on the policies enforcement tooling used across entire company. + +Once we do that we will unlock another opportunity: to ship the new framework / +tooling as a GitLab feature to unlock these consolidation benefits for our +users, customers and entire wider community audience. + +### Limits, quotas and policies + +This document aims to describe our technical vision for building the next rate +limiting architecture for GitLab.com. We refer to this architectural evolution +as "the next rate limiting architecture", but this is a mental shortcut, +because we actually want to build a better framework that will make it easier +for us to manage not only rate limits, but also quotas and policies. + +Below you can find a short definition of what we understand by a limit, by a +quota and by a policy. + +- **Limit:** A constraint on application usage, typically used to mitigate + risks to performance, stability, and security. + - _Example:_ API calls per second for a given IP address + - _Example:_ `git clone` events per minute for a given user + - _Example:_ maximum artifact upload size of 1GB +- **Quota:** A global constraint in application usage that is aggregated across an + entire namespace over the duration of their billing cycle. + - _Example:_ 400 CI/CD minutes per namespace per month + - _Example:_ 10GB transfer per namespace per month +- **Policy:** A representation of business logic that is decoupled from application + code. Decoupled policy definitions allow logic to be shared across multiple services + and/or "hot-loaded" at runtime without releasing a new version of the application. + - _Example:_ decode and verify a JWT, determine whether the user has access to the + given resource based on the JWT's scopes and claims + - _Example:_ deny access based on group-level constraints + (such as IP allowlist, SSO, and 2FA) across all services + +Technically, all of these are limits, because rate limiting is still +"limiting", quota is usually a business limit, and policy limits what you can +do with the application to enforce specific rules. By referring to a "limit" in +this document we mean a limit that is defined to protect business, availability +and security. + +### Framework to define and enforce limits + +First we want to build a new framework that will allow us to define and enforce +application limits, in the GitLab Rails project context, in a more consistent +and established way. In order to do that, we will need to build a new +abstraction that will tell engineers how to define a limit in a structured way +(presumably using YAML or Cue format) and then how to consume the limit in the +application itself. + +We already do have many limits defined in the application, we can use them to +triangulate to find a reasonable abstraction that will consolidate how we +define, use and enforce limits. + +We envision building a simple Ruby library here (we can add it to LabKit) that +will make it trivial for engineers to check if a certain limit has been +exceeded or not. + +```yaml +name: my_limit_name +actors: user +context: project, group, pipeline +type: rate / second +group: pipeline::execution +limits: + warn: 2B / day + soft: 100k / s + hard: 500k / s +``` + +```ruby +Gitlab::Limits::RateThreshold.enforce(:my_limit_name) do |threshold| + actor = current_user + context = current_project + + threshold.available do |limit| + # ... + end + + threshold.approaching do |limit| + # ... + end + + threshold.exceeded do |limit| + # ... + end +end +``` + +In the example above, when `my_limit_name` is defined in YAML, engineers will +be check the current state and execute appropriate code block depending on the +past usage / resource consumption. + +Things we want to build and support by default: + +1. Comprehensive dashboards showing how often limits are being hit. +1. Notifications about the risk of hitting limits. +1. Automation checking if limits definitions are being enforced properly. +1. Different types of limits - time bound / number per resource etc. +1. A panel that makes it easy to override limits per plan / namespace. +1. Logging that will expose limits applied in Kibana. +1. An automatically generated documentation page describing all the limits. + +### API to expose limits and policies + +Once we have an established a consistent way to define application limits we +can build a few API endpoints that will allow us to expose them to our users, +customers and other satellite services that may want to consume them. + +Users will be able to ask the API about the limits / thresholds that have been +set for them, how often they are hitting them, and what impact those might have +on their business. This kind of transparency can help them with communicating +their needs to customer success team at GitLab, and we will be able to +communicate how the responsible usage is defined at a given moment. + +Because of how GitLab architecture has been built, GitLab Rails application, in +most cases, behaves as a central enterprise service bus (ESB) and there are a +few satellite services communicating with it. Services like Container Registry, +GitLab Runners, Gitaly, Workhorse, KAS could use the API to receive a set of +application limits those are supposed to enforce. This will still allow us to +define all of them in a single place. + +We should, however, avoid the possible negative-feedback-loop, that will put +additional strain on the Rails application when there is a sudden increase in +usage happening. This might be a big customer starting a new automation that +traverses our API or a Denial of Service attack. In such cases, the additional +traffic will reach GitLab Rails and subsequently also other satellite services. +Then the satellite services may need to consult Rails again to obtain new +instructions / policies around rate limiting the increased traffic. This can +put additional strain on Rails application and eventually degrade performance +even more. In order to avoid this problem, we should extract the API endpoints +to separate service (see the section below) if the request rate to those +endpoints depends on the volume of incoming traffic. Alternatively we can keep +those endpoints in Rails if the increased traffic will not translate into +increase of requests rate or increase in resources consumption on these API +endpoints on the Rails side. + +#### Decoupled Limits Service + +At some point we may decide that it is time to extract a stateful backend +responsible for storing metadata around limits, all the counters and state +required, and exposing API, out of Rails. + +It is impossible to make a decision about extracting such a decoupled limits +service yet, because we will need to ship more proof-of-concept work, and +concrete iterations to inform us better about when and how we should do that. We +will depend on the Evolution Architecture practice to guide us towards either +extracting Decoupled Limits Service or not doing that at all. + +As we evolve this blueprint, we will document our findings and insights about +how this service should look like, in this section of the document. + +### GitLab Policy Service + +_Disclaimer_: Extracting a GitLab Policy Service might be out of scope +of the current workstream organized around implementing this blueprint. + +Not all limits can be easily described in YAML. There are some more complex +policies that require a bit more sophisticated approach and a declarative +programming language used to enforce them. One example of such a language might be +[Rego](https://www.openpolicyagent.org/docs/latest/policy-language/) language. +It is a standardized way to define policies in +[OPA - Open Policy Agent](https://www.openpolicyagent.org/). At GitLab we are +already using OPA in some departments. We envision the need to additional +consolidation to not only consolidate on the tooling we are using internally at +GitLab, but to also transform the Next Rate Limiting Architecture into +something we can make a part of the product itself. + +Today, we already do have a policy service we are using to decide whether a +pipeline can be created or not. There are many policies defined in +[Pipeline Validation Service](https://gitlab.com/gitlab-org/modelops/anti-abuse/pipeline-validation-service). +There is a significant opportunity here in transforming Pipeline Validation +Service into a general purpose GitLab Policy Service / GitLab Policy Agent that +will be well integrated into the GitLab product itself. + +Generalizing Pipeline Validation Service into GitLab Policy Service can bring a +few interesting benefits: + +1. Consolidate on our tooling across the company to improve efficiency. +1. Integrate our GitLab Rails limits framework to resolve policies using the policy service. +1. Do not struggle to define complex policies in YAML and hack evaluating them in Ruby. +1. Build a policy for GraphQL queries limiting using query execution cost estimation. +1. Make it easier to resolve policies that do not need "hierarchical limits" structure. +1. Make GitLab Policy Service part of the product and integrate it into the single application. + +We envision using GitLab Policy Service to be place to define policies that do +not require knowing anything about the hierarchical structure of the limits. +There are limits that do not need this, like IP addresses allow-list, spam +checks, configuration validation etc. + +We defined "Policy" as a stateless, functional-style, limit. It takes input +arguments and evaluates to either true or false. It should not require a global +counter or any other volatile global state to get evaluated. It may still +require to have a globally defined rules / configuration, but this state is not +volatile in a same way a rate limiting counter may be, or a megabytes consumed +to evaluate quota limit. + +#### Policies used internally and externally + +The GitLab Policy Service might be used in two different ways: + +1. Rails limits framework will use it as a source of policies enforced internally. +1. The policy service feature will be used as a backend to store policies defined by users. + +These are two slightly different use-cases: first one is about using +internally-defined policies to ensure the stability / availably of a GitLab +instance (GitLab.com or self-managed instance). The second use-case is about +making GitLab Policy Service a feature that users will be able to build on top +of. + +Both use-cases are valid but we will need to make technical decision about how +to separate them. Even if we decide to implement them both in a single service, +we will need to draw a strong boundary between the two. + +The same principle might apply to Decouple Limits Service described in one of +the sections of this document above. + +#### The two limits / policy services + +It is possible that GitLab Policy Service and Decoupled Limits Service can +actually be the same thing. It, however, depends on the implementation details +that we can't predict yet, and the decision about merging these services +together will need to be informed by subsequent interations' feedback. + +## Hierarchical limits + +GitLab application aggregates users, projects, groups and namespaces in a +hierarchical way. This hierarchical structure has been designed to make it +easier to manage permissions, streamline workflows, and allow users and +customers to store related projects, repositories, and other artifacts, +together. + +It is important to design the new rate limiting framework in a way that it +built on top of this hierarchical structure and engineers, customers, SREs and +other stakeholders can understand how limits are being applied, enforced and +overridden within the hierarchy of namespaces, groups and projects. + +We want to reduce the cognitive load required to understand how limits are +being managed within the existing permissions structure. We might need to build +a simple and easy-to-understand formula for how our application decides which +limits and thresholds to apply for a given request and a given actor: + +> GitLab will read default limits for every operation, all overrides configured +> and will choose a limit with the highest precedence configured. A limit +> precedence needs to be explicitly configured for every override, a default +> limit has precedence 100. + +One way in which we can simplify limits management in general is to: + +1. Have default limits / thresholds defined in YAML files with a default precedence 100. +1. Allow limits to be overridden through the API, store overrides in the database. +1. Every limit / threshold override needs to have an integer precedence value provided. +1. Build an API that will take an actor and expose limits applicable for it. +1. Build a dashboard showing actors with non-standard limits / overrides. +1. Build a observability around this showing in Kibana when non-standard limits are being used. + +The points above represent an idea to use precedence score (or Z-Index for +limits), but there may be better solutions, like just defining a direction of +overrides - a lower limit might always override a limit defined higher in the +hierarchy. Choosing a proper solution will require a thoughtful research. + +## Principles + +1. Try to avoid building rate limiting framework in a tightly coupled way. +1. Build application limits API in a way that it can be easily extracted to a separate service. +1. Build application limits definition in a way that is independent from the Rails application. +1. Build tooling that produce consistent behavior and results across programming languages. +1. Build the new framework in a way that we can extend to allow self-managed admins to customize limits. +1. Maintain consistent features and behavior across SaaS and self-managed codebase. +1. Be mindful about a cognitive load added by the hierarchical limits, aim to reduce it. + +## Status + +Request For Comments. + +## Timeline + +- 2022-04-27: [Rate Limit Architecture Working Group](https://about.gitlab.com/company/team/structure/working-groups/rate-limit-architecture/) started. +- 2022-06-07: Working Group members [started submitting technical proposals](https://gitlab.com/gitlab-org/gitlab/-/issues/364524) for the next rate limiting architecture. +- 2022-06-15: We started [scoring proposals](https://docs.google.com/spreadsheets/d/1DFHU1kSdTnpydwM5P2RK8NhVBNWgEHvzT72eOhB8F9E) submitted by Working Group members. +- 2022-07-06: A fourth, [consolidated proposal](https://gitlab.com/gitlab-org/gitlab/-/issues/364524#note_1017640650), has been submitted. +- 2022-07-12: Started working on the design document following [Architecture Evolution Workflow](https://about.gitlab.com/handbook/engineering/architecture/workflow/). +- 2022-09-08: The initial version of the blueprint has been merged. + +## Who + +Proposal: + +<!-- vale gitlab.Spelling = NO --> + +| Role | Who +|------------------------------|-------------------------| +| Author | Grzegorz Bizon | +| Author | Fabio Pitino | +| Author | Marshall Cottrell | +| Author | Hayley Swimelar | +| Engineering Leader | Sam Goldstein | +| Product Manager | | +| Architecture Evolution Coach | | +| Recommender | | +| Recommender | | +| Recommender | | +| Recommender | | + +DRIs: + +| Role | Who +|------------------------------|------------------------| +| Leadership | | +| Product | | +| Engineering | | + +Domain experts: + +| Area | Who +|------------------------------|------------------------| +| | | + +<!-- vale gitlab.Spelling = YES --> diff --git a/doc/architecture/blueprints/runner_scaling/index.md b/doc/architecture/blueprints/runner_scaling/index.md index 494aaa6a641..8f7062a1148 100644 --- a/doc/architecture/blueprints/runner_scaling/index.md +++ b/doc/architecture/blueprints/runner_scaling/index.md @@ -33,7 +33,7 @@ This design choice was crucial for the GitLab Runner success. Since that time the auto-scaling feature has been used by many users and customers and enabled rapid growth of CI/CD adoption on GitLab.com. -We can not, however, continue using Docker Machine. Work on that project +We can not, however, continue using Docker Machine. Work on that project [was paused in July 2018](https://github.com/docker/machine/issues/4537) and there was no development made since that time (except for some highly important security fixes). In 2018, after Docker Machine entered the "maintenance mode", @@ -76,7 +76,7 @@ mechanism with a reliable and flexible mechanism. We might be unable to build a drop-in replacement for Docker Machine, as there are presumably many reasons why it has been deprecated. It is very difficult to maintain compatibility with so many cloud providers, and it seems that Docker Machine has been deprecated -in favor of Docker Desktop, which is not a viable replacement for us. +in favor of Docker Desktop, which is not a viable replacement for us. [This issue](https://github.com/docker/roadmap/issues/245) contains a discussion about how people are using Docker Machine right now, and it seems that GitLab CI is one of the most frequent reasons for people to keep using Docker Machine. diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 88d59b1f223..d34f44ea9ba 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -84,7 +84,8 @@ test-job: paths: - .yarn-cache/ script: - - bundle install --path=vendor + - bundle config set --local path 'vendor/ruby' + - bundle install - yarn install --cache-folder .yarn-cache - echo Run tests... ``` @@ -301,8 +302,7 @@ test: If your project uses [pip](https://pip.pypa.io/en/stable/) to install Python dependencies, the following example defines `cache` globally so that -all jobs inherit it. Python libraries are installed in a virtual environment under `venv/`. -pip's cache is defined under `.cache/pip/` and both are cached per-branch: +all jobs inherit it. pip's cache is defined under `.cache/pip/` and is cached per-branch: ```yaml # @@ -317,13 +317,9 @@ variables: # Pip's cache doesn't store the python packages # https://pip.pypa.io/en/stable/reference/pip_install/#caching -# -# If you want to also cache the installed packages, you have to install -# them in a virtualenv and cache it as well. cache: paths: - .cache/pip - - venv/ before_script: - python -V # Print out python version for debugging @@ -358,7 +354,8 @@ cache: before_script: - ruby -v # Print out ruby version for debugging - - bundle install -j $(nproc) --path vendor/ruby # Install dependencies into ./vendor/ruby + - bundle config set --local path 'vendor/ruby' # The location to install the specified gems to + - bundle install -j $(nproc) # Install dependencies into ./vendor/ruby rspec: script: @@ -384,14 +381,16 @@ cache: test_job: stage: test before_script: - - bundle install --without production --path vendor/ruby + - bundle config set --local path 'vendor/ruby' + - bundle install --without production script: - bundle exec rspec deploy_job: stage: production before_script: - - bundle install --without test --path vendor/ruby + - bundle config set --local path 'vendor/ruby' # The location to install the specified gems to + - bundle install --without test script: - bundle exec deploy ``` @@ -472,7 +471,7 @@ and should only be disabled in an environment where all users with Developer rol To use the same cache for all branches: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Clear the **Use separate caches for protected branches** checkbox. @@ -569,7 +568,7 @@ The next time the pipeline runs, the cache is stored in a different location. You can clear the cache in the GitLab UI: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. In the top right, select **Clear runner caches**. diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index c536ff59b84..cfe7be064fb 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -15,7 +15,8 @@ GitLab CI/CD can be used with Bitbucket Cloud by: To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In GitLab, create a project: - 1. On the top menu, select **Projects > Create new project**. + 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. + 1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository**. 1. Select **Repository by URL**. 1. Fill in the fields with information from the repository in Bitbucket: diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index a928d315c6b..a48822fc214 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -34,7 +34,8 @@ repositories: `repo` and `admin:repo_hook` so that GitLab can access your project, update commit statuses, and create a web hook to notify GitLab of new commits. 1. In GitLab, create a project: - 1. On the top menu, select **Projects > Create new project**. + 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. + 1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository**. 1. Select **GitHub**. 1. For **Personal access token**, paste the token. @@ -61,7 +62,8 @@ To manually enable GitLab CI/CD for your repository: 1. Enter a **Token description** and update the scope to allow `repo` so that GitLab can access your project and update commit statuses. 1. In GitLab, create a project: - 1. On the top menu, select **Projects > Create new project**. + 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. + 1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository** and **Repository by URL**. 1. In the **Git repository URL** field, enter the HTTPS URL for your GitHub repository. If your project is private, use the personal access token you just created for authentication. diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index e3a8141ed88..7c0889a329a 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -9,8 +9,8 @@ type: index, howto >[Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4642) in GitLab 10.6. -GitLab CI/CD can be used with [GitHub](github_integration.md), [Bitbucket Cloud](bitbucket_integration.md), or any other -Git server. +GitLab CI/CD can be used with [GitHub](github_integration.md), [Bitbucket Cloud](bitbucket_integration.md), +or any other Git server, though there are some [limitations](#limitations). Instead of moving your entire project to GitLab, you can connect your external repository to get the benefits of GitLab CI/CD. @@ -24,7 +24,8 @@ snippets disabled. These features To connect to an external repository: -1. On the top bar, select **Menu > Projects > Create new project**. +1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository**. 1. Select **GitHub** or **Repository by URL**. 1. Complete the fields. @@ -86,7 +87,11 @@ The variable names are prefixed with `CI_EXTERNAL_PULL_REQUEST_`. ### Limitations -This feature currently does not support Pull Requests from fork repositories. Any Pull Requests from fork repositories are ignored. [Read more](https://gitlab.com/gitlab-org/gitlab/-/issues/5667). +This feature does not support: + +- The [manual connection method](github_integration.md#connect-manually) required for GitHub Enterprise. + If the integration is connected manually, external pull requests [do not trigger pipelines](https://gitlab.com/gitlab-org/gitlab/-/issues/323336#note_884820753). +- Pull requests from fork repositories. [Pull Requests from fork repositories are ignored](https://gitlab.com/gitlab-org/gitlab/-/issues/5667). Given that GitLab creates 2 pipelines, if changes are pushed to a remote branch that references an open Pull Request, both contribute to the status of the Pull Request diff --git a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md index aea7b492d4e..2d1c3f927e2 100644 --- a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md +++ b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md @@ -79,7 +79,7 @@ and [Container Registry](../../../user/packages/container_registry/index.md).  -1. Visit **Packages & Registries > Container Registry**. Make sure the application image has been +1. Visit **Packages and registries > Container Registry**. Make sure the application image has been pushed.  diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index 5df396e796e..82d914f0a6a 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -50,6 +50,7 @@ deploy: script: - aws s3 ... - aws create-deployment ... + environment: production ``` GitLab provides a Docker image that includes the AWS CLI: @@ -215,3 +216,14 @@ To deploy to EC2, complete the following steps. - Your built application is pushed to your S3 bucket then and deployed to your EC2 instance, based on the related JSON object's content. The deployment job finishes when the deployment to EC2 is done or has failed. + +## Troubleshooting + +### Error `'ascii' codec can't encode character '\uxxxx'` + +This error can occur when the response from the `aws-cli` utility used by the Cloud Deploy images contains a Unicode character. The Cloud Deploy images we provide do not have a defined locale and default to using ASCII. To resolve this error, add the following CI/CD variable: + +```yaml +variables: + LANG: "UTF-8" +``` diff --git a/doc/ci/cloud_services/azure/index.md b/doc/ci/cloud_services/azure/index.md new file mode 100644 index 00000000000..901b36afde6 --- /dev/null +++ b/doc/ci/cloud_services/azure/index.md @@ -0,0 +1,157 @@ +--- +stage: Verify +group: Pipeline Authoring +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Configure OpenID Connect in Azure to retrieve temporary credentials + +This tutorial demonstrates how to use a JSON web token (JWT) in a GitLab CI/CD job +to retrieve temporary credentials from Azure without needing to store secrets. + +To get started, configure OpenID Connect (OIDC) for identity federation between GitLab and Azure. +For more information on using OIDC with GitLab, read [Connect to cloud services](../index.md). + +Prerequisites: + +- Access to an existing Azure Subscription with `Owner` access level. +- Access to the corresponding Azure Active Directory Tenant with at least the `Application Developer` access level. +- A local installation of the [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli). + Alternatively, you can follow all the steps below with the [Azure Cloud Shell](https://shell.azure.com/). +- A GitLab project. + +To complete this tutorial: + +1. [Create Azure AD application and service principal](#create-azure-ad-application-and-service-principal). +1. [Create Azure AD federated identity credentials](#create-azure-ad-federated-identity-credentials). +1. [Grant permissions for the service principal](#grant-permissions-for-the-service-principal). +1. [Retrieve a temporary credential](#retrieve-a-temporary-credential). + +For more information, review Azure's documentation on [Workload identity federation](https://docs.microsoft.com/azure/active-directory/develop/workload-identity-federation). + +## Create Azure AD application and service principal + +To create an [Azure AD application](https://docs.microsoft.com/cli/azure/ad/app?view=azure-cli-latest#az-ad-app-create) +and service principal: + +1. In the Azure CLI, create the AD application: + + ```shell + appId=$(az ad app create --display-name gitlab-oidc --query appId -otsv) + ``` + + Save the `appId` (Application client ID) output, as you need it later + to configure your GitLab CI/CD pipeline. + +1. Create a corresponding [Service Principal](https://docs.microsoft.com/cli/azure/ad/sp?view=azure-cli-latest#az-ad-sp-create): + + ```shell + az ad sp create --id $appId --query appId -otsv + ``` + +Instead of the Azure CLI, you can [use the Azure Portal to create these resources](https://docs.microsoft.com/azure/active-directory/develop/howto-create-service-principal-portal). + +## Create Azure AD federated identity credentials + +To create the federated identity credentials for the above Azure AD application: + +```shell +objectId=$(az ad app show --id $appId --query id -otsv) + +cat <<EOF > body.json +{ + "name": "gitlab-federated-identity", + "issuer": "https://gitlab.example.com", + "subject": "project_path:<mygroup>/<myproject>:ref_type:branch:ref:<branch>", + "description": "GitLab service account federated identity", + "audiences": [ + "https://gitlab.example.com" + ] +} +EOF + +az rest --method POST --uri "https://graph.microsoft.com/beta/applications/$objectId/federatedIdentityCredentials" --body @body.json +``` + +For issues related to the values of `issuer`, `subject` or `audiences`, see the +[troubleshooting](#troubleshooting) details. + +Optionally, you can now verify the Azure AD application and the Azure AD federated +identity credentials from the Azure Portal: + +1. Open the [Azure Active Directory App Registration](https://portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/RegisteredApps) + view and select the appropriate app registration by searching for the display name `gitlab-oidc`. +1. On the overview page you can verify details like the `Application (client) ID`, + `Object ID`, and `Tenant ID`. +1. Under `Certificates & secrets`, go to `Federated credentials` to review your + Azure AD federated identity credentials. + +## Grant permissions for the service principal + +After you create the credentials, use [`role assignment`](https://docs.microsoft.com/cli/azure/role/assignment?view=azure-cli-latest#az-role-assignment-create) +to grant permissions to the above service principal to access to Azure resources: + +```shell +az role assignment create --assignee $appId --role Reader --scope /subscriptions/<subscription-id> +``` + +You can find your subscription ID in: + +- The [Azure Portal](https://docs.microsoft.com/azure/azure-portal/get-subscription-tenant-id#find-your-azure-subscription). +- The [Azure CLI](https://docs.microsoft.com/cli/azure/manage-azure-subscriptions-azure-cli#get-the-active-subscription). + +## Retrieve a temporary credential + +After you configure the Azure AD application and federated identity credentials, +the CI/CD job can retrieve a temporary credential by using the [Azure CLI](https://docs.microsoft.com/cli/azure/reference-index?view=azure-cli-latest#az-login): + +```yaml +default: + image: mcr.microsoft.com/azure-cli:latest + +variables: + AZURE_CLIENT_ID: "<client-id>" + AZURE_TENANT_ID: "<tenant-id>" + +auth: + script: + - az login --service-principal -u $AZURE_CLIENT_ID -t $AZURE_TENANT_ID --federated-token $CI_JOB_JWT_V2 + - az account show +``` + +The CI/CD variables are: + +- `AZURE_CLIENT_ID`: The [application client ID you saved earlier](#create-azure-ad-application-and-service-principal). +- `AZURE_TENANT_ID`: Your Azure Active Directory. You can + [find it by using the Azure CLI or Azure Portal](https://docs.microsoft.com/azure/active-directory/fundamentals/active-directory-how-to-find-tenant). +- `CI_JOB_JWT_V2`: The JSON web token is a [predefined CI/CD variable](../../variables/predefined_variables.md). + +## Troubleshooting + +### "No matching federated identity record found" + +If you receive the error `ERROR: AADSTS70021: No matching federated identity record found for presented assertion.` +you should verify: + +- The `Issuer` defined in the Azure AD federated identity credentials, for example + `https://gitlab.com` or your own GitLab URL. +- The `Subject identifier` defined in the Azure AD federated identity credentials, + for example `project_path:<mygroup>/<myproject>:ref_type:branch:ref:<branch>`. + - For the `gitlab-group/gitlab-project` project and `main` branch it would be: + `project_path:gitlab-group/gitlab-project:ref_type:branch:ref:main`. + - The correct values of `mygroup` and `myproject` can be retrieved by checking the URL + when accessing your GitLab project or by selecting the **Clone** option in the project. +- The `Audience` defined in the Azure AD federated identity credentials, for example `https://gitlab.com` + or your own GitLab URL. + +You can review these settings, as well as your `AZURE_CLIENT_ID` and `AZURE_TENANT_ID` +CI/CD variables, from the Azure Portal: + +1. Open the [Azure Active Directory App Registration](https://portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/RegisteredApps) + view and select the appropriate app registration by searching for the display name `gitlab-oidc`. +1. On the overview page you can verify details like the `Application (client) ID`, + `Object ID`, and `Tenant ID`. +1. Under `Certificates & secrets`, go to `Federated credentials` to review your + Azure AD federated identity credentials. + +Review [Connect to cloud services](../index.md) for further details. diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md index 1493a930099..93fedb0ffca 100644 --- a/doc/ci/cloud_services/index.md +++ b/doc/ci/cloud_services/index.md @@ -16,7 +16,7 @@ GitLab CI/CD supports [OpenID Connect (OIDC)](https://openid.net/connect/faq/) t - Account on GitLab. - Access to a cloud provider that supports OIDC to configure authorization and create roles. -The original implementation of `CI_JOB_JWT` supports [HashiCorp Vault integration](../examples/authenticating-with-hashicorp-vault/). The updated implementation of `CI_JOB_JWT_V2` supports additional cloud providers with OIDC including AWS, GCP, and Vault. +The original implementation of `CI_JOB_JWT` supports [HashiCorp Vault integration](../examples/authenticating-with-hashicorp-vault/index.md). The updated implementation of `CI_JOB_JWT_V2` supports additional cloud providers with OIDC including AWS, Azure, GCP, and Vault. NOTE: Configuring OIDC enables JWT token access to the target environments for all pipelines. @@ -25,8 +25,9 @@ review for the pipeline, focusing on the additional access. You can use the [sof as a starting point, and for more information about supply chain attacks, see [How a DevOps Platform helps protect against supply chain attacks](https://about.gitlab.com/blog/2021/04/28/devops-platform-supply-chain-attacks/). -WARNING: -The `CI_JOB_JWT_V2` variable is under development [(alpha)](../../policy/alpha-beta-support.md#alpha-features) and is not yet suitable for production use. +The `CI_JOB_JWT_V2` variable is available for testing, but the full feature is planned +to be generally available when [issue 360657](https://gitlab.com/gitlab-org/gitlab/-/issues/360657) +is complete. ## Use cases @@ -38,7 +39,7 @@ The `CI_JOB_JWT_V2` variable is under development [(alpha)](../../policy/alpha-b ## How it works -Each job has a JSON web token (JWT) provided as a CI/CD [predefined variable](../variables/predefined_variables.md) named `CI_JOB_JWT` or `CI_JOB_JWT_V2`. This JWT can be used to authenticate with the OIDC-supported cloud provider such as AWS, GCP, or Vault. +Each job has a JSON web token (JWT) provided as a CI/CD [predefined variable](../variables/predefined_variables.md) named `CI_JOB_JWT` or `CI_JOB_JWT_V2`. This JWT can be used to authenticate with the OIDC-supported cloud provider such as AWS, Azure, GCP, or Vault. The following fields are included in the JWT: @@ -112,7 +113,7 @@ sequenceDiagram ``` -1. Create an OIDC identity provider in the cloud (for example, AWS, GCP, Vault). +1. Create an OIDC identity provider in the cloud (for example, AWS, Azure, GCP, Vault). 1. Create a conditional role in the cloud service that filters to a group, project, branch, or tag. 1. The CI/CD job includes a predefined variable `CI_JOB_JWT_V2` that is a JWT token. You can use this token for authorization with your cloud API. 1. The cloud verifies the token, validates the conditional role from the payload, and returns a temporary credential. @@ -138,4 +139,5 @@ To configure the trust between GitLab and OIDC, you must create a conditional ro To connect with your cloud provider, see the following tutorials: - [Configure OpenID Connect in AWS](aws/index.md) +- [Configure OpenID Connect in Azure](azure/index.md) - [Configure OpenID Connect in Google Cloud](google_cloud/index.md) diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index ea4ad25637b..4c9cb2923d7 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -345,7 +345,7 @@ not without its own challenges: root file system, you can use the job's working directory as a mount point for child containers. For example, if you have files you want to share with a child container, you might create a subdirectory under `/builds/$CI_PROJECT_PATH` - and use it as your mount point. For a more detailed explanation, view + and use it as your mount point. For a more detailed explanation, view [issue #41227](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41227). ```yaml @@ -406,7 +406,7 @@ sudo gitlab-runner register -n \ ##### Enable registry mirror for `docker:dind` service When the Docker daemon starts inside of the service container, it uses -the default configuration. You may want to configure a +the default configuration. You may want to configure a [registry mirror](https://docs.docker.com/registry/recipes/mirror/) for performance improvements and to ensure you don't reach Docker Hub rate limits. diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index fdd8b6d38b8..f985b02be33 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -406,7 +406,7 @@ image. This image is private and requires you to log in into a private container To configure access for `<aws_account_id>.dkr.ecr.<region>.amazonaws.com`, follow these steps: -1. Make sure `docker-credential-ecr-login` is available in the GitLab Runner `$PATH`. +1. Make sure [`docker-credential-ecr-login`](https://github.com/awslabs/amazon-ecr-credential-helper) is available in the GitLab Runner `$PATH`. 1. Have any of the following [AWS credentials setup](https://github.com/awslabs/amazon-ecr-credential-helper#aws-credentials). Make sure that GitLab Runner can access the credentials. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 712fd7b45d6..ee8d73dd113 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -39,8 +39,6 @@ few important details: GitLab CI/CD. - The entrypoint needs to be [overridden](using_docker_images.md#override-the-entrypoint-of-an-image), otherwise the build script doesn't run. -- A Docker `config.json` file needs to be created with the authentication - information for the desired container registry. In the following example, kaniko is used to: @@ -50,7 +48,7 @@ In the following example, kaniko is used to: The job runs only when a tag is pushed. A `config.json` file is created under `/kaniko/.docker` with the needed GitLab Container Registry credentials taken from the [predefined CI/CD variables](../variables/index.md#predefined-cicd-variables) -GitLab CI/CD provides. +GitLab CI/CD provides. These are automatically read by the Kaniko tool. In the last step, kaniko uses the `Dockerfile` under the root directory of the project, builds the Docker image and pushes it to the @@ -60,13 +58,10 @@ project's Container Registry while tagging it with the Git tag: build: stage: build image: - name: gcr.io/kaniko-project/executor:debug + name: gcr.io/kaniko-project/executor:v1.9.0-debug entrypoint: [""] script: - - mkdir -p /kaniko/.docker - - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 -w 0)\"}}}" > /kaniko/.docker/config.json - - >- - /kaniko/executor + - /kaniko/executor --context "${CI_PROJECT_DIR}" --dockerfile "${CI_PROJECT_DIR}/Dockerfile" --destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}" @@ -86,7 +81,6 @@ you must add the corresponding CI/CD variables for authentication to the `config If you use a custom GitLab Runner behind an http(s) proxy, kaniko needs to be set up accordingly. This means: -- Adding the proxy to `/kaniko/.docker/config.json` - Passing the `http_proxy` environment variables as build arguments so the Dockerfile instructions can use the proxy when building the image. @@ -95,25 +89,20 @@ The previous example can be extended as follows: ```yaml build: stage: build + variables: + http_proxy: <your-proxy> + https_proxy: <your-proxy> + no_proxy: <your-no-proxy> image: - name: gcr.io/kaniko-project/executor:debug + name: gcr.io/kaniko-project/executor:v1.9.0-debug entrypoint: [""] script: - - mkdir -p /kaniko/.docker - - |- - KANIKOPROXYBUILDARGS="" - KANIKOCFG="\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}" - if [ "x${http_proxy}" != "x" -o "x${https_proxy}" != "x" ]; then - KANIKOCFG="${KANIKOCFG}, \"proxies\": { \"default\": { \"httpProxy\": \"${http_proxy}\", \"httpsProxy\": \"${https_proxy}\", \"noProxy\": \"${no_proxy}\"}}" - KANIKOPROXYBUILDARGS="--build-arg http_proxy=${http_proxy} --build-arg https_proxy=${https_proxy} --build-arg no_proxy=${no_proxy}" - fi - KANIKOCFG="{ ${KANIKOCFG} }" - echo "${KANIKOCFG}" > /kaniko/.docker/config.json - - >- - /kaniko/executor + - /kaniko/executor --context "${CI_PROJECT_DIR}" + --build-arg http_proxy=$http_proxy + --build-arg https_proxy=$https_proxy + --build-arg no_proxy=$no_proxy --dockerfile "${CI_PROJECT_DIR}/Dockerfile" - "${KANIKOPROXYBUILDARGS}" --destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}" rules: - if: $CI_COMMIT_TAG @@ -142,8 +131,6 @@ store: ```yaml before_script: - - mkdir -p /kaniko/.docker - - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json - | echo "-----BEGIN CERTIFICATE----- ... diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index dac52a4540e..bac06972c7b 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -30,7 +30,7 @@ When you disable GitLab CI/CD: To disable GitLab CI/CD in your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. In the **Repository** section, turn off **CI/CD**. @@ -40,7 +40,7 @@ To disable GitLab CI/CD in your project: To enable GitLab CI/CD in your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. In the **Repository** section, turn on **CI/CD**. diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index 23fa5d45196..26461d9827f 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -65,9 +65,6 @@ co-exist and multiple approval rules takes the precedence over the unified appro #### Unified approval setting -NOTE: -At this time, it is not possible to require approvals for an existing protected environment. The workaround is to unprotect the environment and configure approvals when re-protecting the environment. - There are two ways to configure approvals for a protected environment: 1. Using the [UI](protected_environments.md#protecting-environments) @@ -137,7 +134,7 @@ Prerequisites: To approve or reject a deployment to a protected environment using the UI: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the environment's name. 1. In the deployment's row, select **Approval options** (**{thumb-up}**). @@ -169,7 +166,7 @@ curl --data "status=approved&comment=Looks good to me" \ ### Using the UI -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the environment being deployed to. 1. Look for the `blocked` label. diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 5b2e2045bdc..90efc7ba9ef 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Deployment safety **(FREE)** -[Deployment jobs](../jobs/#deployment-jobs) are a specific kind of CI/CD +[Deployment jobs](../jobs/index.md#deployment-jobs) are a specific kind of CI/CD job. They can be more sensitive than other jobs in a pipeline, and might need to be treated with extra care. GitLab has several features that help maintain deployment security and stability. @@ -66,7 +66,7 @@ For more information, see [Resource Group documentation](../resource_groups/inde ## Skip outdated deployment jobs The effective execution order of pipeline jobs can vary from run to run, which -could cause undesired behavior. For example, a [deployment job](../jobs/#deployment-jobs) +could cause undesired behavior. For example, a [deployment job](../jobs/index.md#deployment-jobs) in a newer pipeline could finish before a deployment job in an older pipeline. This creates a race condition where the older deployment finishes later, overwriting the "newer" deployment. @@ -131,7 +131,7 @@ All users with the Maintainer role for the project have access to production sec that can deploy to a production environment, you can create a separate project and configure a new permission model that isolates the CD permissions from the original project and prevents the original users with the Maintainer role for the project from accessing the production secret and CD configuration. You can -connect the CD project to your development projects by using [multi-project pipelines](../pipelines/multi_project_pipelines.md). +connect the CD project to your development projects by using [multi-project pipelines](../pipelines/downstream_pipelines.md#multi-project-pipelines). ## Protect `.gitlab-ci.yml` from change diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md index c3c1e7868fd..11e9fe90e25 100644 --- a/doc/ci/environments/environments_dashboard.md +++ b/doc/ci/environments/environments_dashboard.md @@ -21,7 +21,7 @@ diagnose if there is a block at a particular point, or if there's a more systemic problem you need to investigate. You can access the dashboard on the top bar by selecting -**Menu > Environments**. +**Main menu > Environments**.  diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 7747f5e9b78..c34a978709b 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -30,7 +30,7 @@ Prerequisites: To view a list of environments and deployments: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. The environments are displayed. @@ -57,7 +57,7 @@ You can create an environment and deployment in the UI or in your `.gitlab-ci.ym In the UI: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select **New environment**. 1. Enter a name and external URL. @@ -364,7 +364,7 @@ If there is a problem with a deployment, you can retry it or roll it back. To retry or rollback a deployment: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the environment. 1. To the right of the deployment name: @@ -642,7 +642,7 @@ When the environment is stopped, the system runs `on_stop` actions You can view a deployment's expiration date in the GitLab UI. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the name of the deployment. @@ -652,7 +652,7 @@ In the top left, next to the environment name, the expiration date is displayed. You can manually override a deployment's expiration date. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the deployment name. 1. On the top right, select the thumbtack (**{thumbtack}**). @@ -670,7 +670,7 @@ You can delete [stopped environments](#stop-an-environment) in the GitLab UI or To delete a stopped environment in the GitLab UI: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the **Stopped** tab. 1. Next to the environment you want to delete, select **Delete environment**. @@ -785,7 +785,7 @@ Limitations of GitLab Auto Rollback: GitLab Auto Rollback is turned off by default. To turn it on: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Automatic deployment rollbacks**. 1. Select the checkbox for **Enable automatic rollbacks**. @@ -945,7 +945,7 @@ the `review/feature-1` spec takes precedence over `review/*` and `*` specs. Renaming an environment through the UI is not possible. Instead, you need to delete the old environment and create a new one: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Find the environment and stop it. 1. Delete the environment. diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 17eccc38747..e63777dc0e0 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -24,9 +24,13 @@ Maintainer role. ## Protecting environments +Prerequisites: + +- When granting the **Allowed to deploy** permission to a group or sub-group, the user configuring the protected environment must be a **direct member** of the group or sub-group to be added. Otherwise, the group or sub-group will not show up in the dropdown. For more information see [issue #345140](https://gitlab.com/gitlab-org/gitlab/-/issues/345140). + To protect an environment: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Protected environments**. 1. From the **Environment** list, select the environment you want to protect. @@ -238,7 +242,7 @@ To protect a group-level environment, make sure your environments have the corre > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325249) in GitLab 15.1. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Protected environments**. 1. From the **Environment** list, select the [deployment tier of environments](index.md#deployment-tier-of-environments) you want to protect. diff --git a/doc/ci/examples/deployment/index.md b/doc/ci/examples/deployment/index.md index b083dbb8177..b097f1fac76 100644 --- a/doc/ci/examples/deployment/index.md +++ b/doc/ci/examples/deployment/index.md @@ -47,6 +47,7 @@ staging: script: - gem install dpl - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY + environment: staging ``` In the above example we use Dpl to deploy `my-app-staging` to Heroku server with API key stored in `HEROKU_STAGING_API_KEY` secure variable. @@ -70,6 +71,7 @@ staging: - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY only: - main + environment: staging ``` The first line `apt-get update -yq` updates the list of available packages, @@ -93,6 +95,7 @@ staging: - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY only: - main + environment: staging production: stage: deploy @@ -101,6 +104,7 @@ production: - dpl --provider=heroku --app=my-app-production --api_key=$HEROKU_PRODUCTION_API_KEY only: - tags + environment: production ``` We created two deploy jobs that are executed on different events: @@ -117,7 +121,7 @@ We also use two secure variables: To store API keys as secure variables: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Variables**. diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 4d247a4ff74..5ff99755242 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -228,6 +228,7 @@ deploy_terraform: script: # Your Review App deployment scripts - for a working example please check https://gitlab.com/Flockademic/Flockademic/blob/5a45f1c2412e93810fab50e2dab8949e2d0633c7/.gitlab-ci.yml#L315 - echo + environment: production e2e:firefox: stage: confidence-check services: diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 666c4d444d8..9b9f87fffbb 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -176,7 +176,7 @@ Using phpenv also allows to easily configure the PHP environment with: phpenv config-add my_config.ini ``` -*__Important note:__ It seems `phpenv/phpenv` +**Important note:** It seems `phpenv/phpenv` [is abandoned](https://github.com/phpenv/phpenv/issues/57). There is a fork at [`madumlao/phpenv`](https://github.com/madumlao/phpenv) that tries to bring the project back to life. [`CHH/phpenv`](https://github.com/CHH/phpenv) also diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index 2e9e2dd33bf..eaa7d8ebcfa 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -95,7 +95,7 @@ As part of publishing a package, semantic-release increases the version number i 1. Under **select scopes**, select the **api** checkbox. 1. Select **Create project access token**. 1. Copy the value. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Variables**. 1. Select **Add variable**. diff --git a/doc/ci/index.md b/doc/ci/index.md index 80752830b85..be7088ab153 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -146,36 +146,30 @@ See also the [Why CI/CD?](https://docs.google.com/presentation/d/1OGgk2Tcxbpl7DJ As GitLab CI/CD has evolved, certain breaking changes have been necessary. -#### 14.0 - -- No breaking changes. - -#### 13.0 - -- [Remove Backported `os.Expand`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4915). -- [Remove Fedora 29 package support](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/16158). -- [Remove macOS 32-bit support](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25466). -- [Removed `debug/jobs/list?v=1` endpoint](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6361). -- [Remove support for array of strings when defining services for Docker executor](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4922). -- [Remove `--docker-services` flag on register command](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6404). -- [Remove legacy build directory caching](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4180). -- [Remove `FF_USE_LEGACY_VOLUMES_MOUNTING_ORDER` feature flag](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6581). -- [Remove support for Windows Server 1803](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6553). - -#### 12.0 - -- [Use `refspec` to clone/fetch Git repository](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4069). -- [Old cache configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4070). -- [Old metrics server configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4072). -- [Remove `FF_K8S_USE_ENTRYPOINT_OVER_COMMAND`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4073). -- [Remove Linux distributions that reach EOL](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1130). -- [Update command line API for helper images](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4013). -- [Remove old `git clean` flow](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4175). - -#### 11.0 - -- No breaking changes. - -#### 10.0 - -- No breaking changes. +For GitLab 15.0 and later, all breaking changes are documented on the following pages: + +- [Deprecations](../update/deprecations.md) +- [Removals](../update/removals.md) + +The breaking changes for [GitLab Runner](https://docs.gitlab.com/runner/) in earlier +major version releases are: + +- 14.0: No breaking changes. +- 13.0: + - [Remove Backported `os.Expand`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4915). + - [Remove Fedora 29 package support](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/16158). + - [Remove macOS 32-bit support](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25466). + - [Removed `debug/jobs/list?v=1` endpoint](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6361). + - [Remove support for array of strings when defining services for Docker executor](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4922). + - [Remove `--docker-services` flag on register command](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6404). + - [Remove legacy build directory caching](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4180). + - [Remove `FF_USE_LEGACY_VOLUMES_MOUNTING_ORDER` feature flag](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6581). + - [Remove support for Windows Server 1803](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6553). +- 12.0: + - [Use `refspec` to clone/fetch Git repository](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4069). + - [Old cache configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4070). + - [Old metrics server configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4072). + - [Remove `FF_K8S_USE_ENTRYPOINT_OVER_COMMAND`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4073). + - [Remove Linux distributions that reach EOL](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1130). + - [Update command line API for helper images](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4013). + - [Remove old `git clean` flow](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4175). diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index e6a9f1fa646..03c905184cf 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -18,7 +18,7 @@ taken to protect the users. NOTE: [Shared runners on GitLab.com](../runners/index.md) do not -provide an interactive web terminal. Follow +provide an interactive web terminal. Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/24674) for progress on adding support. For groups and projects hosted on GitLab.com, interactive web terminals are available when using your own group or project runner. @@ -27,7 +27,7 @@ terminals are available when using your own group or project runner. Two things need to be configured for the interactive web terminal to work: -- The runner needs to have +- The runner needs to have [`[session_server]` configured properly](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section) - If you are using a reverse proxy with your GitLab instance, web terminals need to be [enabled](../../administration/integration/terminal.md#enabling-and-disabling-terminal-support) @@ -54,7 +54,7 @@ Not all executors are NOTE: The `docker` executor does not keep running after the build script is finished. At that point, the terminal automatically -disconnects and does not wait for the user to finish. Please follow +disconnects and does not wait for the user to finish. Please follow [this issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3605) for updates on improving this behavior. diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index 93f22da648a..812683ef2c1 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -24,12 +24,6 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints - [Releases](../../api/releases/index.md) and [Release links](../../api/releases/links.md). - [Terraform plan](../../user/infrastructure/index.md). -NOTE: -There's an open issue, -[GitLab-#333444](https://gitlab.com/gitlab-org/gitlab/-/issues/333444), -which prevents you from using a job token with internal projects. This bug only impacts self-managed -GitLab instances. - The token has the same permissions to access the API as the user that caused the job to run. A user can cause a job to run by pushing a commit, triggering a manual job, being the owner of a scheduled pipeline, and so on. Therefore, this user must be assigned to @@ -95,7 +89,7 @@ The job token scope is only for controlling access to private projects. ### Configure the job token scope limit -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Token Access**. 1. Toggle **Limit CI_JOB_TOKEN access** to enabled. @@ -121,6 +115,7 @@ trigger_pipeline: - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=main "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" rules: - if: $CI_COMMIT_TAG + environment: production ``` If you use the `CI_PIPELINE_SOURCE` [predefined CI/CD variable](../variables/predefined_variables.md) diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index fd6afb1a0ad..806837e3dc8 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -47,7 +47,7 @@ Clicking an individual job shows you its job log, and allows you to: To view the full list of jobs that ran in a project: -1. On the top bar, select **Menu > Projects** and find the project. +1. On the top bar, select **Main menu > Projects** and find the project. 1. On the left sidebar, select **CI/CD > Jobs**. You can filter the list by [job status](#the-order-of-jobs-in-a-pipeline). diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 0d5357e63ad..5a94c2e9bbc 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -243,8 +243,8 @@ check the value of the `$CI_PIPELINE_SOURCE` variable: | `external` | When you use CI services other than GitLab. | | `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). | | `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/merged_results_pipelines.md), and [merge trains](../pipelines/merge_trains.md). | -| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../pipelines/parent_child_pipelines.md) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | -| `pipeline` | For [multi-project pipelines](../pipelines/multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](../yaml/index.md#trigger) keyword. | +| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | +| `pipeline` | For [multi-project pipelines](../pipelines/downstream_pipelines.md#multi-project-pipelines) created by [using the API with `CI_JOB_TOKEN`](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api), or the [`trigger`](../yaml/index.md#trigger) keyword. | | `push` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedule` | For [scheduled pipelines](../pipelines/schedules.md). | | `trigger` | For pipelines created by using a [trigger token](../triggers/index.md#configure-cicd-jobs-to-run-in-triggered-pipelines). | @@ -578,6 +578,8 @@ To run a manual job, you must have permission to merge to the assigned branch: or deployment view. 1. Next to the manual job, select **Play** (**{play}**). +You can also [add custom CI/CD variables when running a manual job](index.md#specifying-variables-when-running-manual-jobs). + ### Protect manual jobs **(PREMIUM)** Use [protected environments](../environments/protected_environments.md) @@ -645,6 +647,7 @@ timed rollout 10%: script: echo 'Rolling out 10% ...' when: delayed start_in: 30 minutes + environment: production ``` To stop the active timer of a delayed job, select **Unschedule** (**{time-out}**). @@ -698,6 +701,7 @@ deploystacks: parallel: matrix: - PROVIDER: [aws, ovh, gcp, vultr] + environment: production/$PROVIDER ``` You can also [create a multi-dimensional matrix](../yaml/index.md#parallelmatrix). @@ -722,6 +726,7 @@ deploystacks: STACK: [monitoring, backup] - PROVIDER: [gcp, vultr] STACK: [data] + environment: $PROVIDER/$STACK ``` This example generates 6 parallel `deploystacks` trigger jobs, each with different values @@ -754,6 +759,7 @@ deploystacks: STACK: [data] tags: - ${PROVIDER}-${STACK} + environment: $PROVIDER/$STACK ``` #### Fetch artifacts from a `parallel:matrix` job @@ -784,6 +790,7 @@ deploy: dependencies: - "ruby: [2.7, aws]" script: echo hello + environment: production ``` Quotes around the `dependencies` entry are required. @@ -957,6 +964,33 @@ For example: Pattern matching is case-sensitive by default. Use the `i` flag modifier to make a pattern case-insensitive. For example: `/pattern/i`. +#### Store the regex pattern in a variable + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35438) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `ci_fix_rules_if_comparison_with_regexp_variable`, disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/359740) and feature flag `ci_fix_rules_if_comparison_with_regexp_variable` removed in GitLab 15.1. + +Variables on the right side of `=~` and `!~` expressions are evaluated as regular expressions. +The regular expression must be enclosed in forward slashes (`/`). For example: + +```yaml +variables: + pattern: '/^ab.*/' + +regex-job1: + variables: + teststring: 'abcde' + script: echo "This job will run, because 'abcde' matches the /^ab.*/ pattern." + rules: + - if: '$teststring =~ $pattern' + +regex-job2: + variables: + teststring: 'fghij' + script: echo "This job will not run, because 'fghi' does not match the /^ab.*/ pattern." + rules: + - if: '$teststring =~ $pattern' +``` + ### Join variable expressions together with `&&` or `||` > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/62867) in GitLab 12.0 diff --git a/doc/ci/lint.md b/doc/ci/lint.md index 0811cc87e91..8c64d968b8b 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -24,7 +24,7 @@ configuration added with the [`includes` keyword](yaml/index.md#include). To check CI/CD configuration with the CI lint tool: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. In the top right, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. @@ -45,7 +45,7 @@ Prerequisites: To simulate a pipeline: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. In the top right, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index 7255d9aec82..efe11466674 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -136,6 +136,7 @@ job3: job4: stage: deploy script: make deploy + environment: production ``` #### Scheduled run @@ -196,6 +197,7 @@ deploy_prod: script: - echo "Deploy to production server" when: manual + environment: production ``` ### Filter job by branch @@ -222,6 +224,7 @@ deploy: - echo "Deploy job" rules: - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^rc-/ + environment: production ``` ### Caching diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index c59116ea8ed..35c5a7e56c8 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -190,7 +190,7 @@ pdf: Additionally, we have package management features like built-in container and package registries that you can leverage. You can see the complete list of packaging features in the -[Packages & Registries](../../user/packages/index.md) documentation. +[Packages and registries](../../user/packages/index.md) documentation. ## Integrated features diff --git a/doc/ci/mobile_devops.md b/doc/ci/mobile_devops.md new file mode 100644 index 00000000000..6eb56434a1b --- /dev/null +++ b/doc/ci/mobile_devops.md @@ -0,0 +1,42 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference +--- + +# Mobile DevOps + +GitLab Mobile DevOps is a collection of features and tools designed for mobile developers +and teams to automate their build and release process using GitLab CI/CD. Mobile DevOps +is an experimental feature developed by [GitLab Incubation Engineering](https://about.gitlab.com/handbook/engineering/incubation/). + +Mobile DevOps is still in development, but you can: + +- [Request a feature](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=feature_request). +- [Report a bug](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=report_bug). +- [Share feedback](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=general_feedback). + +## Code Signing + +[Project-level Secure Files](secure_files/index.md) makes it easier to manage key stores, provision profiles, +and signing certificates directly in a GitLab project. + +For a guided walkthrough of this feature, watch the [video demo](https://youtu.be/O7FbJu3H2YM). + +## Review Apps for Mobile + +You can use [Review Apps](review_apps/index.md) to preview changes directly from a merge request. +Review Apps for Mobile brings that capability to mobile developers through an integration +with [Appetize](https://appetize.io/). + +Watch a [video walkthrough](https://youtu.be/X15mI19TXa4) of this feature, or visit the +[setup instructions](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/readme/-/issues/15) +to get started. + +## Mobile SAST + +You can use [Static Application Security Testing (SAST)](../user/application_security/sast/index.md) +to run static analyzers on code to check for known security vulnerabilities. Mobile SAST +expands this functionality for mobile teams with an [experimental SAST feature](../user/application_security/sast/index.md#experimental-features) +based on [Mobile Security Framework (MobSF)](https://github.com/MobSF/Mobile-Security-Framework-MobSF). diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index 0fd8fac7741..87c2b3f1c71 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -160,15 +160,23 @@ checkbox appears. Select it to start a new merge request after you commit the ch ### `Configuration validation currently not available` message -This message is due to a problem with the syntax validation in the pipeline editor. -If GitLab is unable to communicate with the service that validates the syntax, the -information in these sections may not display properly: - -- The syntax status on the **Edit** tab (valid or invalid). -- The **Visualize** tab. -- The **Lint** tab. -- The **View merged YAML** tab. - -You can still work on your CI/CD configuration and commit the changes you made without -any issues. As soon as the service becomes available again, the syntax validation -should display immediately. +This message is caused by a problem validating the syntax in the pipeline editor. +It can happen when: + +- GitLab is unable to communicate with the service that validates the syntax, so the + information in these sections may not display properly: + + - The syntax status on the **Edit** tab (valid or invalid). + - The **Visualize** tab. + - The **Lint** tab. + - The **View merged YAML** tab. + + You can still work on your CI/CD configuration and commit the changes you made without + any issues. As soon as the service becomes available again, the syntax validation + should display immediately. + +- Using [`include`](../yaml/index.md#include), but the included configuration files create a loop. + For example, `.gitlab-ci.yml` includes `file1.yml`, which includes `file2.yml`, + which includes `file1.yml`, creating a loop between `file1.yml` and `file2.yml`. + + Remove one of the `include` lines to eliminate the loop and resolve the issue. diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index 4b7d3845361..02a74883244 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -49,7 +49,7 @@ Prerequisite: To change the default quota that applies to all namespaces: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Continuous Integration and Deployment**. 1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. @@ -70,7 +70,7 @@ Prerequisite: To set a quota of CI/CD minutes for a namespace: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Groups**. 1. For the group you want to update, select **Edit**. 1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. @@ -95,7 +95,7 @@ Prerequisite: To view CI/CD minutes being used for your group: -1. On the top bar, select **Menu > Groups** and find your group. The group must not be a subgroup. +1. On the top bar, select **Main menu > Groups** and find your group. The group must not be a subgroup. 1. On the left sidebar, select **Settings > Usage Quotas**. 1. Select the **Pipelines** tab. @@ -148,7 +148,7 @@ You can purchase additional CI/CD minutes for your group. You cannot transfer purchased CI/CD minutes from one group to another, so be sure to select the correct group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Usage Quotas**. 1. Select **Pipelines**. 1. Select **Buy additional minutes**. @@ -201,9 +201,12 @@ can be higher than the end-to-end duration of a pipeline. The cost factors for jobs running on shared runners on GitLab.com are: -- `0.008` for public projects, and projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). - For every 125 minutes of job execution time, you use 1 CI/CD minute. - `1` for internal and private projects. +- `0.5` for public projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). +- `0.008` for public forks of public projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). For every 125 minutes of job execution time, + you use 1 CI/CD minute. +- `0.04` for other public projects, after September 1, 2022 (previously `0.008`). + For every 25 minutes of job execution time, you use 1 CI/CD minute. - Calculated differently for [community contributions to GitLab projects](#cost-factor-for-community-contributions-to-gitlab-projects). The cost factors on self-managed instances are: @@ -236,12 +239,14 @@ GitLab administrators can add a namespace to the reduced cost factor ### Additional costs on GitLab SaaS -GitLab SaaS shared runners have different cost factors, depending on the runner type (Linux, Windows, macOS) and the virtual machine configuration. +GitLab SaaS runners have different cost factors, depending on the runner type (Linux, Windows, macOS) and the virtual machine configuration. -| GitLab SaaS runner type | Virtual machine configuration | CI/CD minutes cost factor | +| GitLab SaaS runner type | Machine Type | CI/CD minutes cost factor | | :--------- | :------------------- | :--------- | -| Linux OS + Docker executor| 1 vCPU, 3.75 GB RAM |1| -| macOS + shell executor | 4 vCPU, 10 GB RAM| 6 | +| Linux OS + Docker executor| Small |1| +| Linux OS + Docker executor| Medium |2| +| Linux OS + Docker executor| Large |3| +| macOS + shell executor | Large| 6 | ### Monthly reset of CI/CD minutes diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md new file mode 100644 index 00000000000..6c7ec8e98ec --- /dev/null +++ b/doc/ci/pipelines/downstream_pipelines.md @@ -0,0 +1,647 @@ +--- +stage: Verify +group: Pipeline Authoring +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Downstream pipelines **(FREE)** + +A downstream pipeline is any GitLab CI/CD pipeline triggered by another pipeline. +A downstream pipeline can be either: + +- A [parent-child pipeline](downstream_pipelines.md#parent-child-pipelines), which is a downstream pipeline triggered + in the same project as the first pipeline. +- A [multi-project pipeline](#multi-project-pipelines), which is a downstream pipeline triggered + in a different project than the first pipeline. + +Parent-child pipelines and multi-project pipelines can sometimes be used for similar purposes, +but there are some key differences. + +Parent-child pipelines: + +- Run under the same project, ref, and commit SHA as the parent pipeline. +- Affect the overall status of the ref the pipeline runs against. For example, + if a pipeline fails for the main branch, it's common to say that "main is broken". + The status of child pipelines don't directly affect the status of the ref, unless the child + pipeline is triggered with [`strategy:depend`](../yaml/index.md#triggerstrategy). +- Are automatically canceled if the pipeline is configured with [`interruptible`](../yaml/index.md#interruptible) + when a new pipeline is created for the same ref. +- Display only the parent pipelines in the pipeline index page. Child pipelines are + visible when visiting their parent pipeline's page. +- Are limited to 2 levels of nesting. A parent pipeline can trigger multiple child pipelines, + and those child pipeline can trigger multiple child pipelines (`A -> B -> C`). + +Multi-project pipelines: + +- Are triggered from another pipeline, but the upstream (triggering) pipeline does + not have much control over the downstream (triggered) pipeline. However, it can + choose the ref of the downstream pipeline, and pass CI/CD variables to it. +- Affect the overall status of the ref of the project it runs in, but does not + affect the status of the triggering pipeline's ref, unless it was triggered with + [`strategy:depend`](../yaml/index.md#triggerstrategy). +- Are not automatically canceled in the downstream project when using [`interruptible`](../yaml/index.md#interruptible) + if a new pipeline runs for the same ref in the upstream pipeline. They can be + automatically canceled if a new pipeline is triggered for the same ref on the downstream project. +- Multi-project pipelines are standalone pipelines because they are normal pipelines + that happened to be triggered by an external project. They are all visible on the pipeline index page. +- Are independent, so there are no nesting limits. + +## Multi-project pipelines + +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. + +You can set up [GitLab CI/CD](../index.md) across multiple projects, so that a pipeline +in one project can trigger a downstream pipeline in another project. You can visualize the entire pipeline +in one place, including all cross-project interdependencies. + +For example, you might deploy your web application from three different projects in GitLab. +Each project has its own build, test, and deploy process. With multi-project pipelines you can +visualize the entire pipeline, including all build and test stages for all three projects. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see the [Multi-project pipelines demo](https://www.youtube.com/watch?v=g_PIwBM1J84). + +Multi-project pipelines are also useful for larger products that require cross-project interdependencies, like those +with a [microservices architecture](https://about.gitlab.com/blog/2016/08/16/trends-in-version-control-land-microservices/). +Learn more in the [Cross-project Pipeline Triggering and Visualization demo](https://about.gitlab.com/learn/) +at GitLab@learn, in the Continuous Integration section. + +If you trigger a pipeline in a downstream private project, on the upstream project's pipelines page, +you can view: + +- The name of the project. +- The status of the pipeline. + +If you have a public project that can trigger downstream pipelines in a private project, +make sure there are no confidentiality problems. + +### Trigger a multi-project pipeline from a job in your `.gitlab-ci.yml` file + +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. + +When you use the [`trigger`](../yaml/index.md#trigger) keyword to create a multi-project +pipeline in your `.gitlab-ci.yml` file, you create what is called a *trigger job*. For example: + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + variables: + ENVIRONMENT: staging + stage: deploy + trigger: my/deployment +``` + +In this example, after the `rspec` job succeeds in the `test` stage, +the `staging` trigger job starts. The initial status of this +job is `pending`. + +GitLab then creates a downstream pipeline in the +`my/deployment` project and, as soon as the pipeline is created, the +`staging` job succeeds. The full path to the project is `my/deployment`. + +You can view the status for the pipeline, or you can display +[the downstream pipeline's status instead](#mirror-the-status-of-a-downstream-pipeline-in-the-trigger-job). + +The user that creates the upstream pipeline must be able to create pipelines in the +downstream project (`my/deployment`) too. If the downstream project is not found, +or the user does not have [permission](../../user/permissions.md) to create a pipeline there, +the `staging` job is marked as _failed_. + +#### Specify a downstream pipeline branch + +You can specify a branch name for the downstream pipeline to use. +GitLab uses the commit on the head of the branch to +create the downstream pipeline. + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + stage: deploy + trigger: + project: my/deployment + branch: stable-11-2 +``` + +Use: + +- The `project` keyword to specify the full path to a downstream project. + In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/367660), variable expansion is + supported. +- The `branch` keyword to specify the name of a branch in the project specified by `project`. + In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/10126), variable expansion is + supported. + +Pipelines triggered on a protected branch in a downstream project use the [role](../../user/permissions.md) +of the user that ran the trigger job in the upstream project. If the user does not +have permission to run CI/CD pipelines against the protected branch, the pipeline fails. See +[pipeline security for protected branches](index.md#pipeline-security-on-protected-branches). + +#### Use `rules` or `only`/`except` with multi-project pipelines + +You can use CI/CD variables or the [`rules`](../yaml/index.md#rulesif) keyword to +[control job behavior](../jobs/job_control.md) for multi-project pipelines. When a +downstream pipeline is triggered with the [`trigger`](../yaml/index.md#trigger) keyword, +the value of the [`$CI_PIPELINE_SOURCE` predefined variable](../variables/predefined_variables.md) +is `pipeline` for all its jobs. + +If you use [`only/except`](../yaml/index.md#only--except) to control job behavior, use the +[`pipelines`](../yaml/index.md#onlyrefs--exceptrefs) keyword. + +### Trigger a multi-project pipeline by using the API + +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/31573) to GitLab Free in 12.4. + +When you use the [`CI_JOB_TOKEN` to trigger pipelines](../jobs/ci_job_token.md), +GitLab recognizes the source of the job token. The pipelines become related, +so you can visualize their relationships on pipeline graphs. + +These relationships are displayed in the pipeline graph by showing inbound and +outbound connections for upstream and downstream pipeline dependencies. + +When using: + +- CI/CD variables or [`rules`](../yaml/index.md#rulesif) to control job behavior, the value of + the [`$CI_PIPELINE_SOURCE` predefined variable](../variables/predefined_variables.md) is + `pipeline` for multi-project pipeline triggered through the API with `CI_JOB_TOKEN`. +- [`only/except`](../yaml/index.md#only--except) to control job behavior, use the + `pipelines` keyword. + +## Parent-child pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16094) in GitLab 12.7. + +As pipelines grow more complex, a few related problems start to emerge: + +- The staged structure, where all steps in a stage must be completed before the first + job in next stage begins, causes arbitrary waits, slowing things down. +- Configuration for the single global pipeline becomes very long and complicated, + making it hard to manage. +- Imports with [`include`](../yaml/index.md#include) increase the complexity of the configuration, and create the potential + for namespace collisions where jobs are unintentionally duplicated. +- Pipeline UX can become unwieldy with so many jobs and stages to work with. + +Additionally, sometimes the behavior of a pipeline needs to be more dynamic. The ability +to choose to start sub-pipelines (or not) is a powerful ability, especially if the +YAML is dynamically generated. + + + +Similarly to [multi-project pipelines](#multi-project-pipelines), a pipeline can trigger a +set of concurrently running downstream child pipelines, but in the same project: + +- Child pipelines still execute each of their jobs according to a stage sequence, but + would be free to continue forward through their stages without waiting for unrelated + jobs in the parent pipeline to finish. +- The configuration is split up into smaller child pipeline configurations. Each child pipeline contains only relevant steps which are + easier to understand. This reduces the cognitive load to understand the overall configuration. +- Imports are done at the child pipeline level, reducing the likelihood of collisions. + +Child pipelines work well with other GitLab CI/CD features: + +- Use [`rules: changes`](../yaml/index.md#ruleschanges) to trigger pipelines only when + certain files change. This is useful for monorepos, for example. +- Since the parent pipeline in `.gitlab-ci.yml` and the child pipeline run as normal + pipelines, they can have their own behaviors and sequencing in relation to triggers. + +See the [`trigger`](../yaml/index.md#trigger) keyword documentation for full details on how to +include the child pipeline configuration. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Parent-Child Pipelines feature demo](https://youtu.be/n8KpBSqZNbk). + +NOTE: +The artifact containing the generated YAML file must not be larger than 5MB. + +### Trigger a parent-child pipeline + +The simplest case is [triggering a child pipeline](../yaml/index.md#trigger) using a +local YAML file to define the pipeline configuration. In this case, the parent pipeline +triggers the child pipeline, and continues without waiting: + +```yaml +microservice_a: + trigger: + include: path/to/microservice_a.yml +``` + +You can include multiple files when defining a child pipeline. The child pipeline's +configuration is composed of all configuration files merged together: + +```yaml +microservice_a: + trigger: + include: + - local: path/to/microservice_a.yml + - template: Security/SAST.gitlab-ci.yml +``` + +In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/205157), +you can use [`include:file`](../yaml/index.md#includefile) to trigger child pipelines +with a configuration file in a different project: + +```yaml +microservice_a: + trigger: + include: + - project: 'my-group/my-pipeline-library' + ref: 'main' + file: '/path/to/child-pipeline.yml' +``` + +The maximum number of entries that are accepted for `trigger:include` is three. + +### Merge request child pipelines + +To trigger a child pipeline as a [merge request pipeline](merge_request_pipelines.md) we need to: + +- Set the trigger job to run on merge requests: + +```yaml +# parent .gitlab-ci.yml +microservice_a: + trigger: + include: path/to/microservice_a.yml + rules: + - if: $CI_MERGE_REQUEST_ID +``` + +- Configure the child pipeline by either: + + - Setting all jobs in the child pipeline to evaluate in the context of a merge request: + + ```yaml + # child path/to/microservice_a.yml + workflow: + rules: + - if: $CI_MERGE_REQUEST_ID + + job1: + script: ... + + job2: + script: ... + ``` + + - Alternatively, setting the rule per job. For example, to create only `job1` in + the context of merge request pipelines: + + ```yaml + # child path/to/microservice_a.yml + job1: + script: ... + rules: + - if: $CI_MERGE_REQUEST_ID + + job2: + script: ... + ``` + +### Dynamic child pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9. + +Instead of running a child pipeline from a static YAML file, you can define a job that runs +your own script to generate a YAML file, which is then used to trigger a child pipeline. + +This technique can be very powerful in generating pipelines targeting content that changed or to +build a matrix of targets and architectures. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Create child pipelines using dynamically generated configurations](https://youtu.be/nMdfus2JWHM). + +We also have an example project using +[Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet) +which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. +You could use a similar process for other templating languages like +[Dhall](https://dhall-lang.org/) or [ytt](https://get-ytt.io/). + +The artifact path is parsed by GitLab, not the runner, so the path must match the +syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows +runner for testing, the path separator for the trigger job would be `/`. Other CI/CD +configuration for jobs, like scripts, that use the Windows runner would use `\`. + +For example, to trigger a child pipeline from a dynamically generated configuration file: + +```yaml +generate-config: + stage: build + script: generate-ci-config > generated-config.yml + artifacts: + paths: + - generated-config.yml + +child-pipeline: + stage: test + trigger: + include: + - artifact: generated-config.yml + job: generate-config +``` + +The `generated-config.yml` is extracted from the artifacts and used as the configuration +for triggering the child pipeline. + +In GitLab 12.9, the child pipeline could fail to be created in certain cases, causing the parent pipeline to fail. +This is [resolved](https://gitlab.com/gitlab-org/gitlab/-/issues/209070) in GitLab 12.10. + +### Nested child pipelines + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29651) in GitLab 13.4. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243747) in GitLab 13.5. + +Parent and child pipelines were introduced with a maximum depth of one level of child +pipelines, which was later increased to two. A parent pipeline can trigger many child +pipelines, and these child pipelines can trigger their own child pipelines. It's not +possible to trigger another level of child pipelines. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Nested Dynamic Pipelines](https://youtu.be/C5j3ju9je2M). + +## View a downstream pipeline + +In the [pipeline graph view](index.md#view-full-pipeline-graph), downstream pipelines display +as a list of cards on the right of the graph. + +### Retry a downstream pipeline + +> - Retry from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. +> - Retry from graph view [generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. + +To retry a completed downstream pipeline, select **Retry** (**{retry}**): + +- From the downstream pipeline's details page. +- On the pipeline's card in the [pipeline graph view](index.md#view-full-pipeline-graph). + +### Cancel a downstream pipeline + +> - Retry from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. +> - Retry from graph view [generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. + +To cancel a downstream pipeline that is still running, select **Cancel** (**{cancel}**): + +- From the downstream pipeline's details page. +- On the pipeline's card in the [pipeline graph view](index.md#view-full-pipeline-graph). + +### Mirror the status of a downstream pipeline in the trigger job + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11238) in GitLab Premium 12.3. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. + +You can mirror the pipeline status from the triggered pipeline to the source trigger job +by using [`strategy: depend`](../yaml/index.md#triggerstrategy): + +::Tabs + +:::TabTitle Multi-project pipeline + +```yaml +trigger_job: + trigger: + project: my/project + strategy: depend +``` + +:::TabTitle Parent-child pipeline + +```yaml +trigger_job: + trigger: + include: + - local: path/to/child-pipeline.yml + strategy: depend +``` + +::EndTabs + +### View multi-project pipelines in pipeline graphs **(PREMIUM)** + +When you trigger a multi-project pipeline, the downstream pipeline displays +to the right of the [pipeline graph](index.md#visualize-pipelines). + + + +In [pipeline mini graphs](index.md#pipeline-mini-graphs), the downstream pipeline +displays to the right of the mini graph. + + + +## Pass artifacts to a downstream pipeline + +You can pass artifacts to a downstream pipeline by using [`needs:project`](../yaml/index.md#needsproject). + +1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword. +1. Trigger the downstream pipeline with a trigger job: + + ```yaml + build_artifacts: + stage: build + script: + - echo "This is a test artifact!" >> artifact.txt + artifacts: + paths: + - artifact.txt + + deploy: + stage: deploy + trigger: my/downstream_project + ``` + +1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline + by using `needs:project`. Set `job` to the job in the upstream pipeline to fetch artifacts from, + `ref` to the branch, and `artifacts: true`. + + ```yaml + test: + stage: test + script: + - cat artifact.txt + needs: + - project: my/upstream_project + job: build_artifacts + ref: main + artifacts: true + ``` + +### Pass artifacts from a Merge Request pipeline + +When you use `needs:project` to [pass artifacts to a downstream pipeline](#pass-artifacts-to-a-downstream-pipeline), +the `ref` value is usually a branch name, like `main` or `development`. + +For merge request pipelines, the `ref` value is in the form of `refs/merge-requests/<id>/head`, +where `id` is the merge request ID. You can retrieve this ref with the [`CI_MERGE_REQUEST_REF_PATH`](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) +CI/CD variable. Do not use a branch name as the `ref` with merge request pipelines, +because the downstream pipeline attempts to fetch artifacts from the latest branch pipeline. + +To fetch the artifacts from the upstream `merge request` pipeline instead of the `branch` pipeline, +pass this variable to the downstream pipeline using variable inheritance: + +1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword. +1. In the job that triggers the downstream pipeline, pass the `$CI_MERGE_REQUEST_REF_PATH` variable by using + [variable inheritance](#pass-yaml-defined-cicd-variables): + + ```yaml + build_artifacts: + stage: build + script: + - echo "This is a test artifact!" >> artifact.txt + artifacts: + paths: + - artifact.txt + + upstream_job: + variables: + UPSTREAM_REF: $CI_MERGE_REQUEST_REF_PATH + trigger: + project: my/downstream_project + branch: my-branch + ``` + +1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline + by using `needs:project`. Set the `ref` to the `UPSTREAM_REF` variable, and `job` + to the job in the upstream pipeline to fetch artifacts from: + + ```yaml + test: + stage: test + script: + - cat artifact.txt + needs: + - project: my/upstream_project + job: build_artifacts + ref: $UPSTREAM_REF + artifacts: true + ``` + +This method works for fetching artifacts from a regular merge request parent pipeline, +but fetching artifacts from [merge results](merged_results_pipelines.md) pipelines is not supported. + +## Pass CI/CD variables to a downstream pipeline + +You can pass CI/CD variables to a downstream pipeline with a few different methods, +based on where the variable is created or defined. + +### Pass YAML-defined CI/CD variables + +You can use the `variables` keyword to pass CI/CD variables to a downstream pipeline, +just like you would for any other job. + +For example, in a [multi-project pipeline](#multi-project-pipelines): + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + variables: + ENVIRONMENT: staging + stage: deploy + trigger: my/deployment +``` + +The `ENVIRONMENT` variable is passed to every job defined in a downstream +pipeline. It is available as a variable when GitLab Runner picks a job. + +In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline +that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` +job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline. + +```yaml +variables: + MY_VARIABLE: my-value + +trigger-downstream: + variables: + ENVIRONMENT: something + trigger: my/project +``` + +### Prevent global variables from being passed + +You can stop global variables from reaching the downstream pipeline by using the [`inherit:variables` keyword](../yaml/index.md#inheritvariables). +For example, in a [multi-project pipeline](#multi-project-pipelines): + +```yaml +variables: + MY_GLOBAL_VAR: value + +trigger-downstream: + inherit: + variables: false + variables: + MY_LOCAL_VAR: value + trigger: my/project +``` + +In this example, the `MY_GLOBAL_VAR` variable is not available in the triggered pipeline. + +### Pass a predefined variable + +You might want to pass some information about the upstream pipeline using predefined variables. +To do that, you can use interpolation to pass any variable. For example, +in a [multi-project pipeline](#multi-project-pipelines): + +```yaml +downstream-job: + variables: + UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME + trigger: my/project +``` + +In this scenario, the `UPSTREAM_BRANCH` variable with the value of the upstream pipeline's +`$CI_COMMIT_REF_NAME` is passed to `downstream-job`. It is available in the +context of all downstream builds. + +You cannot use this method to forward [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables) +to a downstream pipeline, as they are not available in trigger jobs. + +Upstream pipelines take precedence over downstream ones. If there are two +variables with the same name defined in both upstream and downstream projects, +the ones defined in the upstream project take precedence. + +### Pass dotenv variables created in a job **(PREMIUM)** + +You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](../variables/index.md#pass-an-environment-variable-to-another-job) +and [`needs:project`](../yaml/index.md#needsproject). + +For example, in a [multi-project pipeline](#multi-project-pipelines): + +1. Save the variables in a `.env` file. +1. Save the `.env` file as a `dotenv` report. +1. Trigger the downstream pipeline. + + ```yaml + build_vars: + stage: build + script: + - echo "BUILD_VERSION=hello" >> build.env + artifacts: + reports: + dotenv: build.env + + deploy: + stage: deploy + trigger: my/downstream_project + ``` + +1. Set the `test` job in the downstream pipeline to inherit the variables from the `build_vars` + job in the upstream project with `needs`. The `test` job inherits the variables in the + `dotenv` report and it can access `BUILD_VERSION` in the script: + + ```yaml + test: + stage: test + script: + - echo $BUILD_VERSION + needs: + - project: my/upstream_project + job: build_vars + ref: master + artifacts: true + ``` diff --git a/doc/ci/pipelines/img/downstream_pipeline_actions.png b/doc/ci/pipelines/img/downstream_pipeline_actions.png Binary files differdeleted file mode 100644 index 4c4384bab57..00000000000 --- a/doc/ci/pipelines/img/downstream_pipeline_actions.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 08264170d52..2696d3adabd 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -57,44 +57,10 @@ Pipelines can be configured in many different ways: already been merged into the target branch. - [Merge trains](../pipelines/merge_trains.md) use merged results pipelines to queue merges one after the other. -- [Parent-child pipelines](parent_child_pipelines.md) break down complex pipelines +- [Parent-child pipelines](downstream_pipelines.md#parent-child-pipelines) break down complex pipelines into one parent pipeline that can trigger multiple child sub-pipelines, which all run in the same project and with the same SHA. This pipeline architecture is commonly used for mono-repos. -- [Multi-project pipelines](multi_project_pipelines.md) combine pipelines for different projects together. - -### How parent-child pipelines compare to multi-project pipelines - -Parent-child pipelines and multi-project pipelines can sometimes be used for similar -purposes, but there are some key differences: - -Parent-child pipelines: - -- Run under the same project, ref, and commit SHA as the parent pipeline. -- Affect the overall status of the ref the pipeline runs against. For example, - if a pipeline fails for the main branch, it's common to say that "main is broken". - The status of child pipelines don't directly affect the status of the ref, unless the child - pipeline is triggered with [`strategy:depend`](../yaml/index.md#triggerstrategy). -- Are automatically canceled if the pipeline is configured with [`interruptible`](../yaml/index.md#interruptible) - when a new pipeline is created for the same ref. -- Display only the parent pipelines in the pipeline index page. Child pipelines are - visible when visiting their parent pipeline's page. -- Are limited to 2 levels of nesting. A parent pipeline can trigger multiple child pipelines, - and those child pipeline can trigger multiple child pipelines (`A -> B -> C`). - -Multi-project pipelines: - -- Are triggered from another pipeline, but the upstream (triggering) pipeline does - not have much control over the downstream (triggered) pipeline. However, it can - choose the ref of the downstream pipeline, and pass CI/CD variables to it. -- Affect the overall status of the ref of the project it runs in, but does not - affect the status of the triggering pipeline's ref, unless it was triggered with - [`strategy:depend`](../yaml/index.md#triggerstrategy). -- Are not automatically canceled in the downstream project when using [`interruptible`](../yaml/index.md#interruptible) - if a new pipeline runs for the same ref in the upstream pipeline. They can be - automatically canceled if a new pipeline is triggered for the same ref on the downstream project. -- Multi-project pipelines are standalone pipelines because they are normal pipelines - that happened to be triggered by an external project. They are all visible on the pipeline index page. -- Are independent, so there are no nesting limits. +- [Multi-project pipelines](downstream_pipelines.md#multi-project-pipelines) combine pipelines for different projects together. ## Configure a pipeline @@ -175,7 +141,7 @@ operation of the pipeline. To execute a pipeline manually: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. Select **Run pipeline**. 1. In the **Run for branch name or tag** field, select the branch or tag to run the pipeline for. @@ -288,7 +254,7 @@ page, then selecting **Delete**.  Deleting a pipeline does not automatically delete its -[child pipelines](parent_child_pipelines.md). +[child pipelines](downstream_pipelines.md#parent-child-pipelines). See the [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/39503) for details. @@ -323,6 +289,34 @@ preserving deployment keys and other credentials from being unintentionally accessed. To ensure that jobs intended to be executed on protected runners do not use regular runners, they must be tagged accordingly. +## Trigger a pipeline when an upstream project is rebuilt **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9045) in GitLab 12.8. + +You can trigger a pipeline in your project whenever a pipeline finishes for a new +tag in a different project. + +Prerequisites: + +- The upstream project must be [public](../../user/public_access.md). +- The user must have the Developer role + in the upstream project. + +To trigger the pipeline when the upstream project is rebuilt: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Pipeline subscriptions**. +1. Enter the project you want to subscribe to, in the format `<namespace>/<project>`. + For example, if the project is `https://gitlab.com/gitlab-org/gitlab`, use `gitlab-org/gitlab`. +1. Select **Subscribe**. + +Any pipelines that complete successfully for new tags in the subscribed project +now trigger a pipeline on the current project's default branch. The maximum +number of upstream pipeline subscriptions is 2 by default, for both the upstream and +downstream projects. On self-managed instances, an administrator can change this +[limit](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project). + ### How pipeline duration is calculated Total running time for a given pipeline excludes retries and pending @@ -388,8 +382,8 @@ You can group the jobs by: - [Job dependencies](#view-job-dependencies-in-the-pipeline-graph), which arranges jobs based on their [`needs`](../yaml/index.md#needs) dependencies. -[Multi-project pipeline graphs](multi_project_pipelines.md#multi-project-pipeline-visualization) help -you visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)** +[Multi-project pipeline graphs](downstream_pipelines.md#view-multi-project-pipelines-in-pipeline-graphs) help +you visualize the entire pipeline, including all cross-project inter-dependencies. If a stage contains more than 100 jobs, only the first 100 jobs are listed in the pipeline graph. The remaining jobs still run as normal. To see the jobs: @@ -403,8 +397,9 @@ pipeline graph. The remaining jobs still run as normal. To see the jobs: > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/328538) in GitLab 14.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/328538) in GitLab 14.2. -You can arrange jobs in the pipeline graph based on their [`needs`](../yaml/index.md#needs) -dependencies. +To arrange jobs in the pipeline graph based on their [`needs`](../yaml/index.md#needs) +dependencies, select **Job dependencies** in the **Group jobs by** section. This option +is available for pipelines with 3 or more jobs with `needs` job dependencies. Jobs in the leftmost column run first, and jobs that depend on them are grouped in the next columns. @@ -455,25 +450,6 @@ Pipeline analytics are available on the [**CI/CD Analytics** page](../../user/an Pipeline status and test coverage report badges are available and configurable for each project. For information on adding pipeline badges to projects, see [Pipeline badges](settings.md#pipeline-badges). -### Downstream pipelines - -In the pipeline graph view, downstream pipelines ([Multi-project pipelines](multi_project_pipelines.md) -and [Parent-child pipelines](parent_child_pipelines.md)) display as a list of cards -on the right of the graph. - -#### Cancel or retry downstream pipelines from the graph view - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. -> - [Generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. - -To cancel a downstream pipeline that is still running, select **Cancel** (**{cancel}**) -on the pipeline's card. - -To retry a failed downstream pipeline, select **Retry** (**{retry}**) -on the pipeline's card. - - - ## Pipelines API GitLab provides API endpoints to: diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index c8babe3320d..f30ae32efb2 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -305,7 +305,7 @@ the artifact. ## How searching for job artifacts works In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201784), artifacts -for [parent and child pipelines](parent_child_pipelines.md) are searched in hierarchical +for [parent and child pipelines](downstream_pipelines.md#parent-child-pipelines) are searched in hierarchical order from parent to child. For example, if both parent and child pipelines have a job with the same name, the job artifact from the parent pipeline is returned. @@ -388,7 +388,7 @@ Keeping the latest artifacts can use a large amount of storage space in projects with a lot of jobs or large artifacts. If the latest artifacts are not needed in a project, you can disable this behavior to save space: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Artifacts**. 1. Clear the **Keep artifacts from most recent successful jobs** checkbox. @@ -451,3 +451,63 @@ test-job: reports: dotenv: build.env ``` + +### Job artifacts are not expired + +If some job artifacts are not expiring as expected, check if the +[**Keep artifacts from most recent successful jobs**](#keep-artifacts-from-most-recent-successful-jobs) +setting is enabled. + +When this setting is enabled, job artifacts from the latest successful pipeline +of each ref do not expire and are not deleted. + +### Error message `This job could not start because it could not retrieve the needed artifacts.` + +A job configured with [`needs:artifacts`](../yaml/index.md#needsartifacts) keyword +fails to start and returns this error message if: + +- The job's dependencies cannot be found. +- The job cannot access the relevant resources due to insufficient permissions. + +The troubleshooting steps to follow are determined by the syntax used in the job configuration. + +#### Job configured with `needs:project` + +The `could not retrieve the needed artifacts.` error can happen for a job using +[`needs:project`](../yaml/index.md#needsproject), with a configuration similar to: + +```yaml +rspec: + needs: + - project: org/another-project + job: dependency-job + ref: master + artifacts: true +``` + +To troubleshoot this job, verify that: + +- Project `org/another-project` is in a group with a Premium subscription plan. +- The user running the job has permissions to access resources in `org/another-project`. +- The `project`, `job`, and `ref` combination exists and results in the desired dependency. +- Any variables in use evaluate to the correct values. + +#### Job configured with `needs:pipeline:job` + +The `could not retrieve the needed artifacts.` error can happen for a job using +[`needs:pipeline:job`](../yaml/index.md#needspipelinejob), with a configuration similar to: + +```yaml +rspec: + needs: + - pipeline: $UPSTREAM_PIPELINE_ID + job: dependency-job + artifacts: true +``` + +To troubleshoot this job, verify that: + +- The `$UPSTREAM_PIPELINE_ID` CI/CD variable is available in the current pipeline's + parent-child pipeline hierarchy. +- The `pipeline` and `job` combination exists and resolves to an existing pipeline. +- `dependency-job` has run and finished successfully. diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 5ba489c9830..f6c93356046 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -218,3 +218,28 @@ is not considered successful if: When using the [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) feature and both pipelines types are present, the merge request pipelines are checked, not the branch pipelines. + +### `An error occurred while trying to run a new pipeline for this merge request.` + +This error can happen when you select **Run pipeline** in a merge request, but the +project does not have merge request pipelines enabled anymore. + +Some possible reasons for this error message: + +- The project does not have merge request pipelines enabled, has no pipelines listed + in the **Pipelines** tab, and you select **Run pipelines**. +- The project used to have merge request pipelines enabled, but the configuration + was removed. For example: + + 1. The project has merge request pipelines enabled in the `.gitlab-ci.yml` configuration + file when the merge request is created. + 1. The **Run pipeline** options is available in the merge request's **Pipelines** tab, + and selecting **Run pipeline** at this point likely does not cause any errors. + 1. The project's `.gitlab-ci.yml` file is changed to remove the merge request pipelines configuration. + 1. The branch is rebased to bring the updated configuration into the merge request. + 1. Now the pipeline configuration no longer supports merge request pipelines, + but you select **Run pipeline** to run a merge request pipeline. + +If **Run pipeline** is available, but the project does not have merge request pipelines +enabled, do not use this option. You can push a commit or rebase the branch to trigger +new branch pipelines. diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index 2882cd378aa..88ab6163f3c 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -59,7 +59,7 @@ changes that are included in the target branch, and the `C` changes that are fro the merge request already in the train. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -Watch this video for a demonstration on +Watch this video for a demonstration on [how parallel execution of merge trains can prevent commits from breaking the default branch](https://www.youtube.com/watch?v=D4qCqXgZkHQ). ## Prerequisites @@ -81,9 +81,8 @@ To enable merge trains for your project: 1. If you are on a self-managed GitLab instance, ensure the [feature flag](#merge-trains-feature-flag) is set correctly. 1. [Configure your CI/CD configuration file](merge_request_pipelines.md#prerequisites) so that the pipeline or individual jobs run for merge requests. -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. 1. In the **Merge method** section, verify that **Merge commit** is selected. 1. In the **Merge options** section, select **Enable merged results pipelines** (if not already selected) and **Enable merge trains**. 1. Select **Save changes**. @@ -210,7 +209,7 @@ If it succeeds after a retry, the merge request is not removed from the merge tr Sometimes the **Start/Add to merge train** button is not available and the merge request says, "The pipeline for this merge request failed. Please retry the job or push a new commit to fix the failure." -This issue occurs when [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) +This issue occurs when [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) is enabled in **Settings > General > Merge requests**. This option requires that you run a new successful pipeline before you can re-add a merge request to a merge train. diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md index 777871a7c5f..7209a6b9a77 100644 --- a/doc/ci/pipelines/merged_results_pipelines.md +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -41,10 +41,9 @@ To use merged results pipelines: To enable merged results pipelines in a project, you must have at least the Maintainer role: -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. -1. Select **Enable merged results pipelines**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge options** section, select **Enable merged results pipelines**. 1. Select **Save changes**. WARNING: diff --git a/doc/ci/pipelines/multi_project_pipelines.md b/doc/ci/pipelines/multi_project_pipelines.md index a71af78f410..824f1445818 100644 --- a/doc/ci/pipelines/multi_project_pipelines.md +++ b/doc/ci/pipelines/multi_project_pipelines.md @@ -1,419 +1,11 @@ --- -stage: Verify -group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference +redirect_to: 'downstream_pipelines.md' +remove_date: '2022-11-31' --- -# Multi-project pipelines **(FREE)** +This document was moved to [another location](downstream_pipelines.md). -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. - -You can set up [GitLab CI/CD](../index.md) across multiple projects, so that a pipeline -in one project can trigger a pipeline in another project. You can visualize the entire pipeline -in one place, including all cross-project interdependencies. - -For example, you might deploy your web application from three different projects in GitLab. -Each project has its own build, test, and deploy process. With multi-project pipelines you can -visualize the entire pipeline, including all build and test stages for all three projects. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see the [Multi-project pipelines demo](https://www.youtube.com/watch?v=g_PIwBM1J84). - -Multi-project pipelines are also useful for larger products that require cross-project interdependencies, like those -with a [microservices architecture](https://about.gitlab.com/blog/2016/08/16/trends-in-version-control-land-microservices/). -Learn more in the [Cross-project Pipeline Triggering and Visualization demo](https://about.gitlab.com/learn/) -at GitLab@learn, in the Continuous Integration section. - -If you trigger a pipeline in a downstream private project, on the upstream project's pipelines page, -you can view: - -- The name of the project. -- The status of the pipeline. - -If you have a public project that can trigger downstream pipelines in a private project, -make sure there are no confidentiality problems. - -## Create multi-project pipelines - -To create multi-project pipelines, you can: - -- [Define them in your `.gitlab-ci.yml` file](#define-multi-project-pipelines-in-your-gitlab-ciyml-file). -- [Use the API](#create-multi-project-pipelines-by-using-the-api). - -### Define multi-project pipelines in your `.gitlab-ci.yml` file - -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. - -When you use the [`trigger`](../yaml/index.md#trigger) keyword to create a multi-project -pipeline in your `.gitlab-ci.yml` file, you create what is called a *trigger job*. For example: - -```yaml -rspec: - stage: test - script: bundle exec rspec - -staging: - variables: - ENVIRONMENT: staging - stage: deploy - trigger: my/deployment -``` - -In this example, after the `rspec` job succeeds in the `test` stage, -the `staging` trigger job starts. The initial status of this -job is `pending`. - -GitLab then creates a downstream pipeline in the -`my/deployment` project and, as soon as the pipeline is created, the -`staging` job succeeds. The full path to the project is `my/deployment`. - -You can view the status for the pipeline, or you can display -[the downstream pipeline's status instead](#mirror-status-of-a-triggered-pipeline-in-the-trigger-job). - -The user that creates the upstream pipeline must be able to create pipelines in the -downstream project (`my/deployment`) too. If the downstream project is not found, -or the user does not have [permission](../../user/permissions.md) to create a pipeline there, -the `staging` job is marked as _failed_. - -#### Trigger job configuration limitations - -Trigger jobs can use only a limited set of the GitLab CI/CD [configuration keywords](../yaml/index.md). -The keywords available for use in trigger jobs are: - -- [`trigger`](../yaml/index.md#trigger) -- [`stage`](../yaml/index.md#stage) -- [`allow_failure`](../yaml/index.md#allow_failure) -- [`rules`](../yaml/index.md#rules) -- [`only` and `except`](../yaml/index.md#only--except) -- [`when`](../yaml/index.md#when) (only with a value of `on_success`, `on_failure`, or `always`) -- [`extends`](../yaml/index.md#extends) -- [`needs`](../yaml/index.md#needs), but not [`needs:project`](../yaml/index.md#needsproject) - -Trigger jobs cannot use [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables). - -#### Specify a downstream pipeline branch - -You can specify a branch name for the downstream pipeline to use. -GitLab uses the commit on the head of the branch to -create the downstream pipeline. - -```yaml -rspec: - stage: test - script: bundle exec rspec - -staging: - stage: deploy - trigger: - project: my/deployment - branch: stable-11-2 -``` - -Use: - -- The `project` keyword to specify the full path to a downstream project. - In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/367660), variable expansion is - supported. -- The `branch` keyword to specify the name of a branch in the project specified by `project`. - In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/10126), variable expansion is - supported. - -Pipelines triggered on a protected branch in a downstream project use the [role](../../user/permissions.md) -of the user that ran the trigger job in the upstream project. If the user does not -have permission to run CI/CD pipelines against the protected branch, the pipeline fails. See -[pipeline security for protected branches](index.md#pipeline-security-on-protected-branches). - -#### Pass CI/CD variables to a downstream pipeline by using the `variables` keyword - -Sometimes you might want to pass CI/CD variables to a downstream pipeline. -You can do that by using the `variables` keyword, just like you would for any other job. - -```yaml -rspec: - stage: test - script: bundle exec rspec - -staging: - variables: - ENVIRONMENT: staging - stage: deploy - trigger: my/deployment -``` - -The `ENVIRONMENT` variable is passed to every job defined in a downstream -pipeline. It is available as a variable when GitLab Runner picks a job. - -In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline -that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` -job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline. - -```yaml -variables: - MY_VARIABLE: my-value - -trigger-downstream: - variables: - ENVIRONMENT: something - trigger: my/project -``` - -You can stop global variables from reaching the downstream pipeline by using the [`inherit:variables` keyword](../yaml/index.md#inheritvariables). -In this example, the `MY_GLOBAL_VAR` variable is not available in the triggered pipeline: - -```yaml -variables: - MY_GLOBAL_VAR: value - -trigger-downstream: - inherit: - variables: false - variables: - MY_LOCAL_VAR: value - trigger: my/project -``` - -You might want to pass some information about the upstream pipeline using, for -example, predefined variables. In order to do that, you can use interpolation -to pass any variable. For example: - -```yaml -downstream-job: - variables: - UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME - trigger: my/project -``` - -In this scenario, the `UPSTREAM_BRANCH` variable with the value of the upstream pipeline's -`$CI_COMMIT_REF_NAME` is passed to `downstream-job`. It is available in the -context of all downstream builds. - -You cannot use this method to forward [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables) -to a downstream pipeline, as they are not available in trigger jobs. - -Upstream pipelines take precedence over downstream ones. If there are two -variables with the same name defined in both upstream and downstream projects, -the ones defined in the upstream project take precedence. - -#### Pass CI/CD variables to a downstream pipeline by using variable inheritance **(PREMIUM)** - -You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](../variables/index.md#pass-an-environment-variable-to-another-job) and [`needs:project`](../yaml/index.md#needsproject). - -In the upstream pipeline: - -1. Save the variables in a `.env` file. -1. Save the `.env` file as a `dotenv` report. -1. Trigger the downstream pipeline. - - ```yaml - build_vars: - stage: build - script: - - echo "BUILD_VERSION=hello" >> build.env - artifacts: - reports: - dotenv: build.env - - deploy: - stage: deploy - trigger: my/downstream_project - ``` - -1. Set the `test` job in the downstream pipeline to inherit the variables from the `build_vars` - job in the upstream project with `needs`. The `test` job inherits the variables in the - `dotenv` report and it can access `BUILD_VERSION` in the script: - - ```yaml - test: - stage: test - script: - - echo $BUILD_VERSION - needs: - - project: my/upstream_project - job: build_vars - ref: master - artifacts: true - ``` - -#### Pass artifacts to a downstream pipeline - -You can pass artifacts to a downstream pipeline by using [`needs:project`](../yaml/index.md#needsproject). - -1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword. -1. Trigger the downstream pipeline with a trigger job: - - ```yaml - build_artifacts: - stage: build - script: - - echo "This is a test artifact!" >> artifact.txt - artifacts: - paths: - - artifact.txt - - deploy: - stage: deploy - trigger: my/downstream_project - ``` - -1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline - by using `needs:project`. Set `job` to the job in the upstream pipeline to fetch artifacts from, - `ref` to the branch, and `artifacts: true`. - - ```yaml - test: - stage: test - script: - - cat artifact.txt - needs: - - project: my/upstream_project - job: build_artifacts - ref: main - artifacts: true - ``` - -#### Pass artifacts to a downstream pipeline from a Merge Request pipeline - -When you use `needs:project` to [pass artifacts to a downstream pipeline](#pass-artifacts-to-a-downstream-pipeline), -the `ref` value is usually a branch name, like `main` or `development`. - -For merge request pipelines, the `ref` value is in the form of `refs/merge-requests/<id>/head`, -where `id` is the merge request ID. You can retrieve this ref with the [`CI_MERGE_REQUEST_REF_PATH`](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) -CI/CD variable. Do not use a branch name as the `ref` with merge request pipelines, -because the downstream pipeline attempts to fetch artifacts from the latest branch pipeline. - -To fetch the artifacts from the upstream `merge request` pipeline instead of the `branch` pipeline, -pass this variable to the downstream pipeline using variable inheritance: - -1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword. -1. In the job that triggers the downstream pipeline, pass the `$CI_MERGE_REQUEST_REF_PATH` variable by using - [variable inheritance](#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword): - - ```yaml - build_artifacts: - stage: build - script: - - echo "This is a test artifact!" >> artifact.txt - artifacts: - paths: - - artifact.txt - - upstream_job: - variables: - UPSTREAM_REF: $CI_MERGE_REQUEST_REF_PATH - trigger: - project: my/downstream_project - branch: my-branch - ``` - -1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline - by using `needs:project`. Set the `ref` to the `UPSTREAM_REF` variable, and `job` - to the job in the upstream pipeline to fetch artifacts from: - - ```yaml - test: - stage: test - script: - - cat artifact.txt - needs: - - project: my/upstream_project - job: build_artifacts - ref: UPSTREAM_REF - artifacts: true - ``` - -This method works for fetching artifacts from a regular merge request parent pipeline, -but fetching artifacts from [merge results](merged_results_pipelines.md) pipelines is not supported. - -#### Use `rules` or `only`/`except` with multi-project pipelines - -You can use CI/CD variables or the [`rules`](../yaml/index.md#rulesif) keyword to -[control job behavior](../jobs/job_control.md) for multi-project pipelines. When a -downstream pipeline is triggered with the [`trigger`](../yaml/index.md#trigger) keyword, -the value of the [`$CI_PIPELINE_SOURCE` predefined variable](../variables/predefined_variables.md) -is `pipeline` for all its jobs. - -If you use [`only/except`](../yaml/index.md#only--except) to control job behavior, use the -[`pipelines`](../yaml/index.md#onlyrefs--exceptrefs) keyword. - -#### Mirror status of a triggered pipeline in the trigger job - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11238) in GitLab Premium 12.3. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. - -You can mirror the pipeline status from the triggered pipeline to the source trigger job -by using [`strategy: depend`](../yaml/index.md#triggerstrategy). For example: - -```yaml -trigger_job: - trigger: - project: my/project - strategy: depend -``` - -### Create multi-project pipelines by using the API - -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/31573) to GitLab Free in 12.4. - -When you use the [`CI_JOB_TOKEN` to trigger pipelines](../jobs/ci_job_token.md), -GitLab recognizes the source of the job token. The pipelines become related, -so you can visualize their relationships on pipeline graphs. - -These relationships are displayed in the pipeline graph by showing inbound and -outbound connections for upstream and downstream pipeline dependencies. - -When using: - -- CI/CD variables or [`rules`](../yaml/index.md#rulesif) to control job behavior, the value of - the [`$CI_PIPELINE_SOURCE` predefined variable](../variables/predefined_variables.md) is - `pipeline` for multi-project pipeline triggered through the API with `CI_JOB_TOKEN`. -- [`only/except`](../yaml/index.md#only--except) to control job behavior, use the - `pipelines` keyword. - -## Trigger a pipeline when an upstream project is rebuilt **(PREMIUM)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9045) in GitLab 12.8. - -You can trigger a pipeline in your project whenever a pipeline finishes for a new -tag in a different project. - -Prerequisites: - -- The upstream project must be [public](../../user/public_access.md). -- The user must have the Developer role - in the upstream project. - -To trigger the pipeline when the upstream project is rebuilt: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. -1. Expand **Pipeline subscriptions**. -1. Enter the project you want to subscribe to, in the format `<namespace>/<project>`. - For example, if the project is `https://gitlab.com/gitlab-org/gitlab`, use `gitlab-org/gitlab`. -1. Select **Subscribe**. - -Any pipelines that complete successfully for new tags in the subscribed project -now trigger a pipeline on the current project's default branch. The maximum -number of upstream pipeline subscriptions is 2 by default, for both the upstream and -downstream projects. On self-managed instances, an administrator can change this -[limit](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project). - -## Multi-project pipeline visualization **(PREMIUM)** - -When your pipeline triggers a downstream pipeline, the downstream pipeline displays -to the right of the [pipeline graph](index.md#visualize-pipelines). - - - -In [pipeline mini graphs](index.md#pipeline-mini-graphs), the downstream pipeline -displays to the right of the mini graph. - - - -## Retry or cancel multi-project pipelines - -If you have permission to trigger pipelines in the downstream project, you can -retry or cancel multi-project pipelines: - -- [In the main graph view](index.md#downstream-pipelines). -- From the downstream pipeline's details page. +<!-- This redirect file can be deleted after <2022-11-31>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/pipelines/parent_child_pipelines.md b/doc/ci/pipelines/parent_child_pipelines.md index 3fd739087ec..be8ed8ba6d7 100644 --- a/doc/ci/pipelines/parent_child_pipelines.md +++ b/doc/ci/pipelines/parent_child_pipelines.md @@ -1,228 +1,11 @@ --- -stage: Verify -group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference +redirect_to: 'downstream_pipelines.md' +remove_date: '2022-12-05' --- -# Parent-child pipelines **(FREE)** +This document was moved to [another location](downstream_pipelines.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16094) in GitLab 12.7. - -As pipelines grow more complex, a few related problems start to emerge: - -- The staged structure, where all steps in a stage must be completed before the first - job in next stage begins, causes arbitrary waits, slowing things down. -- Configuration for the single global pipeline becomes very long and complicated, - making it hard to manage. -- Imports with [`include`](../yaml/index.md#include) increase the complexity of the configuration, and create the potential - for namespace collisions where jobs are unintentionally duplicated. -- Pipeline UX can become unwieldy with so many jobs and stages to work with. - -Additionally, sometimes the behavior of a pipeline needs to be more dynamic. The ability -to choose to start sub-pipelines (or not) is a powerful ability, especially if the -YAML is dynamically generated. - - - -Similarly to [multi-project pipelines](multi_project_pipelines.md), a pipeline can trigger a -set of concurrently running child pipelines, but within the same project: - -- Child pipelines still execute each of their jobs according to a stage sequence, but - would be free to continue forward through their stages without waiting for unrelated - jobs in the parent pipeline to finish. -- The configuration is split up into smaller child pipeline configurations. Each child pipeline contains only relevant steps which are - easier to understand. This reduces the cognitive load to understand the overall configuration. -- Imports are done at the child pipeline level, reducing the likelihood of collisions. - -Child pipelines work well with other GitLab CI/CD features: - -- Use [`rules: changes`](../yaml/index.md#ruleschanges) to trigger pipelines only when - certain files change. This is useful for monorepos, for example. -- Since the parent pipeline in `.gitlab-ci.yml` and the child pipeline run as normal - pipelines, they can have their own behaviors and sequencing in relation to triggers. - -See the [`trigger`](../yaml/index.md#trigger) keyword documentation for full details on how to -include the child pipeline configuration. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Parent-Child Pipelines feature demo](https://youtu.be/n8KpBSqZNbk). - -NOTE: -The artifact containing the generated YAML file must not be larger than 5MB. - -## Examples - -The simplest case is [triggering a child pipeline](../yaml/index.md#trigger) using a -local YAML file to define the pipeline configuration. In this case, the parent pipeline -triggers the child pipeline, and continues without waiting: - -```yaml -microservice_a: - trigger: - include: path/to/microservice_a.yml -``` - -You can include multiple files when defining a child pipeline. The child pipeline's -configuration is composed of all configuration files merged together: - -```yaml -microservice_a: - trigger: - include: - - local: path/to/microservice_a.yml - - template: Security/SAST.gitlab-ci.yml -``` - -In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/205157), -you can use [`include:file`](../yaml/index.md#includefile) to trigger child pipelines -with a configuration file in a different project: - -```yaml -microservice_a: - trigger: - include: - - project: 'my-group/my-pipeline-library' - ref: 'main' - file: '/path/to/child-pipeline.yml' -``` - -The maximum number of entries that are accepted for `trigger:include` is three. - -Similar to [multi-project pipelines](multi_project_pipelines.md#mirror-status-of-a-triggered-pipeline-in-the-trigger-job), -we can set the parent pipeline to depend on the status of the child pipeline upon completion: - -```yaml -microservice_a: - trigger: - include: - - local: path/to/microservice_a.yml - - template: Security/SAST.gitlab-ci.yml - strategy: depend -``` - -## Merge request child pipelines - -To trigger a child pipeline as a [merge request pipeline](merge_request_pipelines.md) we need to: - -- Set the trigger job to run on merge requests: - -```yaml -# parent .gitlab-ci.yml -microservice_a: - trigger: - include: path/to/microservice_a.yml - rules: - - if: $CI_MERGE_REQUEST_ID -``` - -- Configure the child pipeline by either: - - - Setting all jobs in the child pipeline to evaluate in the context of a merge request: - - ```yaml - # child path/to/microservice_a.yml - workflow: - rules: - - if: $CI_MERGE_REQUEST_ID - - job1: - script: ... - - job2: - script: ... - ``` - - - Alternatively, setting the rule per job. For example, to create only `job1` in - the context of merge request pipelines: - - ```yaml - # child path/to/microservice_a.yml - job1: - script: ... - rules: - - if: $CI_MERGE_REQUEST_ID - - job2: - script: ... - ``` - -## Dynamic child pipelines - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9. - -Instead of running a child pipeline from a static YAML file, you can define a job that runs -your own script to generate a YAML file, which is then used to trigger a child pipeline. - -This technique can be very powerful in generating pipelines targeting content that changed or to -build a matrix of targets and architectures. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Create child pipelines using dynamically generated configurations](https://youtu.be/nMdfus2JWHM). - -We also have an example project using -[Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet) -which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. -You could use a similar process for other templating languages like -[Dhall](https://dhall-lang.org/) or [ytt](https://get-ytt.io/). - -The artifact path is parsed by GitLab, not the runner, so the path must match the -syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows -runner for testing, the path separator for the trigger job would be `/`. Other CI/CD -configuration for jobs, like scripts, that use the Windows runner would use `\`. - -In GitLab 12.9, the child pipeline could fail to be created in certain cases, causing the parent pipeline to fail. -This is [resolved](https://gitlab.com/gitlab-org/gitlab/-/issues/209070) in GitLab 12.10. - -### Dynamic child pipeline example - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9. - -You can trigger a child pipeline from a [dynamically generated configuration file](../pipelines/parent_child_pipelines.md#dynamic-child-pipelines): - -```yaml -generate-config: - stage: build - script: generate-ci-config > generated-config.yml - artifacts: - paths: - - generated-config.yml - -child-pipeline: - stage: test - trigger: - include: - - artifact: generated-config.yml - job: generate-config -``` - -The `generated-config.yml` is extracted from the artifacts and used as the configuration -for triggering the child pipeline. - -## Nested child pipelines - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29651) in GitLab 13.4. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243747) in GitLab 13.5. - -Parent and child pipelines were introduced with a maximum depth of one level of child -pipelines, which was later increased to two. A parent pipeline can trigger many child -pipelines, and these child pipelines can trigger their own child pipelines. It's not -possible to trigger another level of child pipelines. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Nested Dynamic Pipelines](https://youtu.be/C5j3ju9je2M). - -## Pass CI/CD variables to a child pipeline - -You can pass CI/CD variables to a downstream pipeline using the same methods as -multi-project pipelines: - -- [By using the `variable` keyword](multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword). -- [By using variable inheritance](multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance). - -## Retry or cancel child pipelines - -You can retry or cancel child pipelines: - -- [In the main graph view](index.md#downstream-pipelines). -- In the child pipeline's details page. +<!-- This redirect file can be deleted after <2022-12-05>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/pipelines/pipeline_architectures.md b/doc/ci/pipelines/pipeline_architectures.md index 3ff22a16900..a02ac7ba067 100644 --- a/doc/ci/pipelines/pipeline_architectures.md +++ b/doc/ci/pipelines/pipeline_architectures.md @@ -84,12 +84,14 @@ deploy_a: script: - echo "This job deploys something. It will only run when all jobs in the" - echo "test stage complete." + environment: production deploy_b: stage: deploy script: - echo "This job deploys something else. It will only run when all jobs in the" - echo "test stage complete. It will start at about the same time as deploy_a." + environment: production ``` ## Directed Acyclic Graph Pipelines @@ -151,18 +153,20 @@ deploy_a: script: - echo "Since build_a and test_a run quickly, this deploy job can run much earlier." - echo "It does not need to wait for build_b or test_b." + environment: production deploy_b: stage: deploy needs: [test_b] script: - echo "Since build_b and test_b run slowly, this deploy job will run much later." + environment: production ``` ## Child / Parent Pipelines In the examples above, it's clear we've got two types of things that could be built independently. -This is an ideal case for using [Child / Parent Pipelines](parent_child_pipelines.md)) via +This is an ideal case for using [Child / Parent Pipelines](downstream_pipelines.md#parent-child-pipelines)) via the [`trigger` keyword](../yaml/index.md#trigger). It separates out the configuration into multiple files, keeping things very simple. You can also combine this with: @@ -237,6 +241,7 @@ deploy_a: needs: [test_a] script: - echo "This job deploys something." + environment: production ``` Example child `b` pipeline configuration, located in `/b/.gitlab-ci.yml`, making @@ -266,6 +271,7 @@ deploy_b: needs: [test_b] script: - echo "This job deploys something else." + environment: production ``` It's also possible to set jobs to run before or after triggering child pipelines, diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md index ad43895d7ef..72711f9b9dd 100644 --- a/doc/ci/pipelines/pipeline_efficiency.md +++ b/doc/ci/pipelines/pipeline_efficiency.md @@ -187,7 +187,7 @@ shouldn't run, saving pipeline resources. In a basic configuration, jobs always wait for all other jobs in earlier stages to complete before running. This is the simplest configuration, but it's also the slowest in most cases. [Directed Acyclic Graphs](../directed_acyclic_graph/index.md) and -[parent/child pipelines](parent_child_pipelines.md) are more flexible and can +[parent/child pipelines](downstream_pipelines.md#parent-child-pipelines) are more flexible and can be more efficient, but can also make pipelines harder to understand and analyze. ### Caching diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 8ab80e3798a..897caa340f9 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -27,7 +27,7 @@ Otherwise, the pipeline is not created. No error message is displayed. To add a pipeline schedule: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Schedules**. 1. Select **New schedule** and fill in the form. - **Interval Pattern**: Select one of the preconfigured intervals, or enter a custom @@ -45,7 +45,7 @@ To add a pipeline schedule: The owner of a pipeline schedule can edit it: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. In the left sidebar, select **CI/CD > Schedules**. 1. Next to the schedule, select **Edit** (**{pencil}**) and fill in the form. @@ -58,7 +58,7 @@ of the schedule. To trigger a pipeline schedule manually, so that it runs immediately instead of the next scheduled time: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Schedules**. 1. On the right of the list, for the pipeline you want to run, select **Play** (**{play}**). @@ -74,7 +74,7 @@ including [protected environments](../environments/protected_environments.md) an To take ownership of a pipeline created by a different user: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Schedules**. 1. On the right of the list, for the pipeline you want to become owner of, select **Take ownership**. diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 34eae9828dd..d663dea7de8 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -25,7 +25,7 @@ For public and internal projects, you can change who can see your: To change the visibility of your pipelines and related features: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Select or clear the **Public pipelines** checkbox. @@ -57,7 +57,7 @@ This setting has no effect when: To change the pipeline visibility for non-project members: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. For **CI/CD**, choose: @@ -73,7 +73,7 @@ is selected. You can set pending or running pipelines to cancel automatically when a new pipeline runs on the same branch. You can enable this in the project settings: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General Pipelines**. 1. Select the **Auto-cancel redundant pipelines** checkbox. @@ -94,7 +94,7 @@ newer one, which may not be what you want. To avoid this scenario: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Select the **Skip outdated deployment jobs** checkbox. @@ -130,7 +130,7 @@ directory. However, you can specify an alternate filename path, including locati To customize the path: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **CI/CD configuration file** field, enter the filename. If the file: @@ -179,7 +179,7 @@ able to edit it. You can choose how your repository is fetched from GitLab when a job runs. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Under **Git strategy**, select an option: @@ -200,7 +200,7 @@ in the `.gitlab-ci.yml` file. You can limit the number of changes that GitLab CI/CD fetches when it clones a repository. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Under **Git strategy**, under **Git shallow clone**, enter a value. @@ -217,7 +217,7 @@ in the `.gitlab-ci.yml` file. You can define how long a job can run before it times out. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **Timeout** field, enter the number of minutes, or a human-readable value like `2 hours`. @@ -282,7 +282,7 @@ You can verify correct syntax using the [pipeline editor](../pipeline_editor/ind To migrate from the project coverage setting to the `coverage` keyword, use the regular expression displayed in the settings. Available in GitLab 14.10 and earlier: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. @@ -326,7 +326,7 @@ Use this regex for commonly used test tools. To see the evolution of your project code coverage over time, you can view a graph or download a CSV file with this data. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Repository**. The historic data for each job is listed in the dropdown above the graph. @@ -392,7 +392,7 @@ Support for [`semver`](https://semver.org/) sorting is tracked [in this issue](h You can view the exact link for your badges. Then you can embed the badge in your HTML or Markdown pages. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **Pipeline status**, **Coverage report**, or **Latest release** sections, view the URLs for the images. diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index 0369824c92e..2b44cf3b898 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -109,6 +109,7 @@ To create a `.gitlab-ci.yml` file: stage: deploy script: - echo "This job deploys something from the $CI_COMMIT_BRANCH branch." + environment: production ``` `$GITLAB_USER_LOGIN` and `$CI_COMMIT_BRANCH` are diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md index e76c4621a0c..dff52a742a8 100644 --- a/doc/ci/resource_groups/index.md +++ b/doc/ci/resource_groups/index.md @@ -171,6 +171,7 @@ deploy: include: deploy.gitlab-ci.yml strategy: depend resource_group: AWS-production + environment: production ``` ```yaml @@ -187,6 +188,7 @@ provision: deployment: stage: deploy script: echo "Deploying..." + environment: production ``` You must define [`strategy: depend`](../yaml/index.md#triggerstrategy) @@ -208,7 +210,7 @@ Read more how you can use GitLab for [safe deployments](../environments/deployme Because [`oldest_first` process mode](#process-modes) enforces the jobs to be executed in a pipeline order, there is a case that it doesn't work well with the other CI features. -For example, when you run [a child pipeline](../pipelines/parent_child_pipelines.md) +For example, when you run [a child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines) that requires the same resource group with the parent pipeline, a dead lock could happen. Here is an example of a _bad_ setup: @@ -224,6 +226,7 @@ deploy: stage: deploy script: echo resource_group: production + environment: production ``` In a parent pipeline, it runs the `test` job that subsequently runs a child pipeline, @@ -250,4 +253,5 @@ deploy: stage: deploy script: echo resource_group: production + environment: production ``` diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 6dd03033926..9bafb69482b 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -76,7 +76,7 @@ Prerequisite: To use the Review Apps template: -1. On the top bar, select **Menu > Projects** and find the project you want to create a Review App job for. +1. On the top bar, select **Main menu > Projects** and find the project you want to create a Review App job for. 1. On the left sidebar, select **Deployments > Environments**. 1. Select **Enable Review Apps**. 1. Copy the provided code snippet and paste it into your diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index 3efa697bf2f..9d26ec63f96 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -151,7 +151,7 @@ different places. To view the IP address of a shared runner you must have administrator access to the GitLab instance. To determine this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Runners**. 1. Find the runner in the table and view the **IP Address** column. @@ -859,7 +859,7 @@ You can clean up group runners that have been inactive for more than three month Group runners are those that were created at the group level. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn on the **Enable stale runner cleanup** toggle. @@ -903,8 +903,8 @@ The version of GitLab Runner used by your runners should be To determine which runners need to be upgraded: 1. View the list of runners: - - For a group, on the top bar, select **Menu > Groups** and on the left sidebar, select **CI/CD > Runners**. - - For the instance, select **Menu > Admin** and on the left sidebar, select **Runners**. + - For a group, on the top bar, select **Main menu > Groups**, find your group, and on the left sidebar select **CI/CD > Runners**. + - For the instance, select **Main menu > Admin** and on the left sidebar, select **Runners**. 1. Above the list of runners, view the status: - **Outdated - recommended**: The runner does not have the latest `PATCH` version, which may make it vulnerable @@ -912,3 +912,43 @@ To determine which runners need to be upgraded: - **Outdated - available**: Newer versions are available but upgrading is not critical. 1. Filter the list by status to view which individual runners need to be upgraded. + +## Authentication token security + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 15.3 [with a flag](../../administration/feature_flags.md) named `enforce_runner_token_expires_at`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to +[enable the feature flag](../../administration/feature_flags.md) named `enforce_runner_token_expires_at`. +On GitLab.com, this feature is not available. + +Each runner has an [authentication token](../../api/runners.md#registration-and-authentication-tokens) +to connect with the GitLab instance. + +To help prevent the token from being compromised, you can have the +token rotate automatically at specified intervals. When the tokens are rotated, +they are updated for each runner, regardless of the runner's status (`online` or `offline`). + +No manual intervention should be required, and no running jobs should be affected. + +If you need to manually update the authentication token, you can run a +command to [reset the token](https://docs.gitlab.com/runner/commands/#gitlab-runner-reset-token). + +### Automatically rotate authentication tokens + +You can specify an interval for authentication tokens to rotate. +This rotation helps ensure the security of the tokens assigned to your runners. + +Prerequisites: + +- Ensure your runners are using [GitLab Runner 15.3 or later](https://docs.gitlab.com/runner/#gitlab-runner-versions). + +To automatically rotate runner authentication tokens: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Continuous Integration and Deployment** +1. Set a **Runners expiration** time for runners, leave empty for no expiration. +1. Select **Save**. + +Before the interval expires, runners automatically request a new authentication token. diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index f69d1f0f730..1de24efa8aa 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -7,8 +7,8 @@ type: reference # Runner SaaS **(FREE SAAS)** -If you use GitLab SaaS (GitLab.com), your CI jobs automatically run on runners provided by GitLab. -No configuration is required. Your jobs can run on: +If you use GitLab SaaS (GitLab.com), your [untagged](../../ci/runners/configure_runners.md#use-tags-to-control-which-jobs-a-runner-can-run) CI jobs automatically run in containers on the Linux Runners. +As long as shared runners are enabled for your project, no configuration is required. Your jobs can run on: - [Linux runners](saas/linux_saas_runner.md). - [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index 9bd0b52f423..f8a33ea6861 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -28,7 +28,7 @@ If you are using a self-managed instance of GitLab: going to your project's **Settings > CI/CD**, expanding **Runners**, and selecting **Show runner installation instructions**. These instructions are also available [in the documentation](https://docs.gitlab.com/runner/install/index.html). -- The administrator can also configure a maximum number of shared runner +- The administrator can also configure a maximum number of shared runner [CI/CD minutes for each group](../pipelines/cicd_minutes.md#set-the-quota-of-cicd-minutes-for-a-specific-namespace). If you are using GitLab.com: @@ -51,7 +51,7 @@ For existing projects, an administrator must To enable shared runners for a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn on the **Enable shared runners for this project** toggle. @@ -60,7 +60,7 @@ To enable shared runners for a project: To enable shared runners for a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn on the **Enable shared runners for this group** toggle. @@ -73,7 +73,7 @@ or group. To disable shared runners for a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. In the **Shared runners** area, turn off the **Enable shared runners for this project** toggle. @@ -87,7 +87,7 @@ Shared runners are automatically disabled for a project: To disable shared runners for a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn off the **Enable shared runners for this group** toggle. @@ -170,7 +170,7 @@ You must have the Owner role for the group. To create a group runner: 1. [Install GitLab Runner](https://docs.gitlab.com/runner/install/). -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **CI/CD > Runners**. 1. Note the URL and token. 1. [Register the runner](https://docs.gitlab.com/runner/register/). @@ -183,7 +183,7 @@ You can view and manage all runners for a group, its subgroups, and projects. You can do this for your self-managed GitLab instance or for GitLab.com. You must have the Owner role for the group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **CI/CD > Runners**. From this page, you can edit, pause, and remove runners from the group, its subgroups, and projects. @@ -193,7 +193,7 @@ From this page, you can edit, pause, and remove runners from the group, its subg You can pause or remove a group runner for your self-managed GitLab instance or for GitLab.com. You must have the Owner role for the group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **CI/CD > Runners**. 1. Select **Pause** or **Remove runner**. - If you pause a group runner that is used by multiple projects, the runner pauses for all projects. @@ -229,7 +229,7 @@ Prerequisite: To create a specific runner: 1. [Install GitLab Runner](https://docs.gitlab.com/runner/install/). -1. On the top bar, select **Menu > Projects** and find the project where you want to use the runner. +1. On the top bar, select **Main menu > Projects** and find the project where you want to use the runner. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. In the **Specific runners** section, note the URL and token. @@ -250,7 +250,7 @@ You must have at least the Maintainer role for: To enable a specific runner for a project: -1. On the top bar, select **Menu > Projects** and find the project where you want to enable the runner. +1. On the top bar, select **Main menu > Projects** and find the project where you want to enable the runner. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. In the **Specific runners** area, by the runner you want, select **Enable for this project**. @@ -269,7 +269,7 @@ but can also be changed later. To lock or unlock a specific runner: -1. On the top bar, select **Menu > Projects** and find the project where you want to enable the runner. +1. On the top bar, select **Main menu > Projects** and find the project where you want to enable the runner. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Find the specific runner you want to lock or unlock. Make sure it's enabled. You cannot lock shared or group runners. diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md index e96e89b47e5..a7d1b8722a5 100644 --- a/doc/ci/runners/saas/linux_saas_runner.md +++ b/doc/ci/runners/saas/linux_saas_runner.md @@ -4,43 +4,97 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# SaaS runners on Linux **(FREE SAAS)** +# SaaS runners on Linux -SaaS runners on Linux are autoscaled ephemeral Google Cloud Platform virtual machines. +When you run jobs on SaaS runners on Linux, the runners are on auto-scaled ephemeral virtual machine (VM) instances. +Each VM uses the Google Container-Optimized OS (COS) and the latest version of Docker Engine. +The default region for the VMs is `us-east1`. -Autoscaling means reduced queue times to spin up CI/CD jobs, and isolated VMs for each job, thus maximizing security. These shared runners are available on GitLab.com. +## Machine types available for private projects (x86-64) -GitLab offers Ultimate tier capabilities and included CI/CD minutes per group per month for our [Open Source](https://about.gitlab.com/solutions/open-source/join/), [Education](https://about.gitlab.com/solutions/education/), and [Startups](https://about.gitlab.com/solutions/startups/) programs. For private projects, GitLab offers various [plans](https://about.gitlab.com/pricing/), starting with a Free tier. +For the SaaS runners on Linux, GitLab offers a range of machine types for use in private projects. +For Free, Premium, and Ultimate plan customers, jobs on these instances consume the CI/CD minutes allocated to your namespace. -All your CI/CD jobs run on [n1-standard-1 instances](https://cloud.google.com/compute/docs/machine-types) with 3.75GB of RAM, Google COS and the latest Docker Engine -installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default -region of the VMs is US East1. -Each instance is used only for one job. This ensures that any sensitive data left on the system can't be accessed by other people's CI/CD jobs. +| | Small | Medium | Large | +|-------------------|---------------------------|---------------------------|--------------------------| +| Specs | 1 vCPU, 3.75GB RAM | 2 vCPUs, 8GB RAM | 4 vCPUs, 16GB RAM | +| GitLab CI/CD tags | `saas-linux-small-amd64` | `saas-linux-medium-amd64` | `saas-linux-large-amd64` | +| Subscription | Free, Premium, Ultimate | Free, Premium, Ultimate | Premium, Ultimate | -NOTE: -The final disk space your jobs can use will be less than 25GB. Some disk space allocated to the instance will be occupied by the operating system, the Docker image, and a copy of your cloned repository. +The `small` machine type is the default. Your job runs on this machine type if you don't specify +a [tags:](../../yaml/index.md#tags) keyword in your `.gitlab-ci.yml` file. + +CI/CD jobs that run on `medium` and `large` machine types **will** consume CI minutes at a different rate than CI/CD jobs on the `small` machine type. + +Refer to the CI/CD minutes [cost factor](../../../ci/pipelines/cicd_minutes.md#cost-factor) for the cost factor applied to the machine type based on size. + +## Example of how to tag a job -The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They don't run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times. +To use a machine type other than `small`, add a `tags:` keyword to your job. +For example: -Jobs handled by shared runners on GitLab.com (`shared-runners-manager-X.gitlab.com`) -**time out after 3 hours**, regardless of the timeout configured in a -project. Check issue [#4010](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4010) and [#4070](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4070) for the reference. +```yaml +stages: + - Prebuild + - Build + - Unit Test -Jobs handled by shared runners on Windows and macOS on GitLab.com **time out after 1 hour** while this service is in the [Beta](../../../policy/alpha-beta-support.md#beta-features) stage. +job_001: + stage: Prebuild + script: + - echo "this job runs on the default (small) instance" -Below are the runners' settings. +job_002: + tags: [ saas-linux-medium-amd64 ] + stage: Build + script: + - echo "this job runs on the medium instance" + + +job_003: + tags: [ saas-linux-large-amd64 ] + stage: Unit Test + script: + - echo "this job runs on the large instance" + +``` -| Setting | GitLab.com | Default | -| ----------- | ----------------- | ---------- | -| Executor | `docker+machine` | - | -| Default Docker image | `ruby:2.5` | - | -| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | +## SaaS runners for GitLab projects -These runners share a [distributed cache](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) through use of a Google Cloud Storage (GCS) bucket. Cache contents not updated within the last 14 days are automatically removed through use of an [object lifecycle management policy](https://cloud.google.com/storage/docs/lifecycle). +The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for +GitLab projects and related community forks. These runners are backed by a Google Compute +`n1-standard-2` machine type and do not run untagged jobs. Unlike the machine types used +for private projects, each virtual machine is re-used up to 40 times. + +## SaaS runners on Linux settings + +Below are the settings for SaaS runners on Linux. + +| Setting | GitLab.com | Default | +|-------------------------------------------------------------------------|------------------|---------| +| Executor | `docker+machine` | - | +| Default Docker image | `ruby:2.5` | - | +| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | + +- **Cache**: These runners share a + [distributed cache](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) + that's stored in a Google Cloud Storage (GCS) bucket. Cache contents not updated within + the last 14 days are automatically removed, based on the + [object lifecycle management policy](https://cloud.google.com/storage/docs/lifecycle). + +- **Timeout settings**: Jobs handled by the SaaS Runners on Linux + **time out after 3 hours**, regardless of the timeout configured in a + project. For details, see issues [#4010](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4010) + and [#4070](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4070). + +NOTE: +The final disk space your jobs can use will be less than 25GB. Some disk space +allocated to the instance will be occupied by the operating system, the Docker image, +and a copy of your cloned repository. ## Pre-clone script -With SaaS runners on Linux, you can run commands in a CI +With SaaS runners on Linux, you can run commands in a CI/CD job before the runner attempts to run `git init` and `git fetch` to download a GitLab repository. The [`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) @@ -55,12 +109,13 @@ To use this feature, define a [CI/CD variable](../../../ci/variables/index.md#cu `CI_PRE_CLONE_SCRIPT` that contains a bash script. NOTE: -The `CI_PRE_CLONE_SCRIPT` variable does not work on GitLab SaaS Windows or macOS Runners. +The `CI_PRE_CLONE_SCRIPT` variable does not work on GitLab SaaS Windows or macOS runners. ### Pre-clone script example This example was used in the `gitlab-org/gitlab` project until November 2021. -The project no longer uses this optimization because the [pack-objects cache](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache) +The project no longer uses this optimization because the +[pack-objects cache](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache) lets Gitaly serve the full CI/CD fetch traffic. See [Git fetch caching](../../../development/pipelines.md#git-fetch-caching). The `CI_PRE_CLONE_SCRIPT` was defined as a project CI/CD variable: diff --git a/doc/ci/secure_files/index.md b/doc/ci/secure_files/index.md index 8e141c62ef6..ff5c29cdb06 100644 --- a/doc/ci/secure_files/index.md +++ b/doc/ci/secure_files/index.md @@ -23,7 +23,7 @@ plain text and binary file types. You can manage secure files in the project settings, or with the [secure files API](../../api/secure_files.md). Secure files can be [downloaded and used by CI/CD jobs](#use-secure-files-in-cicd-jobs) -by using the [load-secure-files](https://gitlab.com/gitlab-org/incubation-engineering/devops-for-mobile-apps/load-secure-files) +by using the [load-secure-files](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/load-secure-files) tool. NOTE: @@ -34,7 +34,7 @@ Additional features and capabilities are planned. To add a secure file to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. In the **Secure Files** section, select **Expand**. 1. Select **Upload File**. @@ -43,7 +43,7 @@ To add a secure file to a project: ## Use secure files in CI/CD jobs -To use your secure files in a CI/CD job, you must use the [`load-secure-files`](https://gitlab.com/gitlab-org/incubation-engineering/devops-for-mobile-apps/load-secure-files) +To use your secure files in a CI/CD job, you must use the [`load-secure-files`](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/load-secure-files) tool to download the files in the job. After they are downloaded, you can use them with your other script commands. @@ -59,5 +59,5 @@ test: variables: SECURE_FILES_DOWNLOAD_PATH: './where/files/should/go/' script: - - curl --silent "https://gitlab.com/gitlab-org/incubation-engineering/devops-for-mobile-apps/load-secure-files/-/raw/main/installer" | bash + - curl --silent "https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/load-secure-files/-/raw/main/installer" | bash ``` diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index ba3f806c96e..9b2bcc39b3e 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -8,9 +8,12 @@ type: index # Services **(FREE)** -The `services` keyword defines a Docker image that runs during a `job` -linked to the Docker image that the image keyword defines. This allows -you to access the service image during build time. +When you configure CI/CD, you specify an image, which is used to create the container +where your jobs will run. To specify this image, you use the `image` keyword. + +You can specify an additional image by using the `services` keyword. This additional +image is used to create another container, which is available to the first container. +The two containers have access to one another and can communicate when running the job. The service image can run any application, but the most common use case is to run a database container, for example: diff --git a/doc/ci/testing/unit_test_reports.md b/doc/ci/testing/unit_test_reports.md index 6294996c703..28356a62c99 100644 --- a/doc/ci/testing/unit_test_reports.md +++ b/doc/ci/testing/unit_test_reports.md @@ -156,7 +156,7 @@ If parsing JUnit report XML results in an error, an indicator is shown next to t  -For test case parsing limits, see [Max test cases per unit test report](../../user/gitlab_com/#gitlab-cicd). +For test case parsing limits, see [Max test cases per unit test report](../../user/gitlab_com/index.md#gitlab-cicd). GitLab does not parse very [large nodes](https://nokogiri.org/tutorials/parsing_an_html_xml_document.html#parse-options) of JUnit reports. There is [an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/268035) open to make this optional. diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index 2c916ba092c..36d136727b0 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -13,7 +13,7 @@ to the [pipeline triggers API endpoint](../../api/pipeline_triggers.md). When authenticating with the API, you can use: - A [trigger token](#create-a-trigger-token) to trigger a branch or tag pipeline. -- A [CI/CD job token](../jobs/ci_job_token.md) to trigger a [multi-project pipeline](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api). +- A [CI/CD job token](../jobs/ci_job_token.md) to [trigger a multi-project pipeline](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api). ## Create a trigger token @@ -26,7 +26,7 @@ Prerequisite: To create a trigger token: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Pipeline triggers**. 1. Enter a description and select **Add trigger**. @@ -88,6 +88,7 @@ trigger_pipeline: - 'curl --fail --request POST --form token=$MY_TRIGGER_TOKEN --form ref=main "https://gitlab.example.com/api/v4/projects/123456/trigger/pipeline"' rules: - if: $CI_COMMIT_TAG + environment: production ``` In this example: @@ -152,7 +153,7 @@ users with the Owner and Maintainer role can view the values. To revoke a trigger token: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Pipeline triggers**. 1. To the left of the trigger token you want to revoke, select **Revoke** (**{remove}**). @@ -169,7 +170,7 @@ To [configure when to run jobs](../jobs/job_control.md) in triggered pipelines: | `$CI_PIPELINE_SOURCE` value | `only`/`except` keywords | Trigger method | |-----------------------------|--------------------------|---------------------| | `trigger` | `triggers` | In pipelines triggered with the [pipeline triggers API](../../api/pipeline_triggers.md) by using a [trigger token](#create-a-trigger-token). | -| `pipeline` | `pipelines` | In [multi-project pipelines](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api) triggered with the [pipeline triggers API](../../api/pipeline_triggers.md) by using the [`$CI_JOB_TOKEN`](../jobs/ci_job_token.md), or by using the [`trigger`](../yaml/index.md#trigger) keyword in the CI/CD configuration file. | +| `pipeline` | `pipelines` | In [multi-project pipelines](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api) triggered with the [pipeline triggers API](../../api/pipeline_triggers.md) by using the [`$CI_JOB_TOKEN`](../jobs/ci_job_token.md), or by using the [`trigger`](../yaml/index.md#trigger) keyword in the CI/CD configuration file. | Additionally, the `$CI_PIPELINE_TRIGGERED` predefined CI/CD variable is set to `true` in pipelines triggered with a trigger token. diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 0230aaf7113..8a03ec0b56f 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -40,15 +40,26 @@ is at: https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/editor/schema/ci.json. ``` -The schema rules for custom YAML tags like `!reference` will not work until the -custom tags are set in the editor settings. For example, in VS Code, you can set -`vscode-yaml` to parse `customTags`: - -```json -"yaml.customTags": [ - "!reference sequence" -] -``` +The schema rules for custom YAML tags like `!reference` are treated as invalid by your editor until the custom tags are +set in your editor settings. For example: + +- In VS Code, you can set `vscode-yaml` to parse `customTags` in your `settings.json` file: + + ```json + "yaml.customTags": [ + "!reference sequence" + ] + ``` + +- In Sublime Text, if you are using the `LSP-yaml` package, you can set `customTags` in your `LSP-yaml` user settings: + + ```json + { + "settings": { + "yaml.customTags": ["!reference sequence"] + } + } + ``` To see the full list of custom tags covered by the CI/CD schema, check the latest version of the schema, linked above. @@ -87,11 +98,11 @@ and [templates](examples/index.md#cicd-templates). Some pipeline types have their own detailed usage guides that you should read if you are using that type: -- [Multi-project pipelines](pipelines/multi_project_pipelines.md): Have your pipeline trigger +- [Multi-project pipelines](pipelines/downstream_pipelines.md#multi-project-pipelines): Have your pipeline trigger a pipeline in a different project. -- [Parent/child pipelines](pipelines/parent_child_pipelines.md): Have your main pipeline trigger +- [Parent/child pipelines](pipelines/downstream_pipelines.md#parent-child-pipelines): Have your main pipeline trigger and run separate pipelines in the same project. You can also - [dynamically generate the child pipeline's configuration](pipelines/parent_child_pipelines.md#dynamic-child-pipelines) + [dynamically generate the child pipeline's configuration](pipelines/downstream_pipelines.md#dynamic-child-pipelines) at runtime. - [Merge request pipelines](pipelines/merge_request_pipelines.md): Run a pipeline in the context of a merge request. @@ -259,7 +270,7 @@ are enabled, the button is either **Add to merge train** or **Add to merge train #### "A CI/CD pipeline must run and be successful before merge" message -This message is shown if the [Pipelines must succeed](../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) +This message is shown if the [Pipelines must succeed](../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) setting is enabled in the project and a pipeline has not yet run successfully. This also applies if the pipeline has not been created yet, or if you are waiting for an external CI service. If you don't use pipelines for your project, then you @@ -316,11 +327,16 @@ To reduce the configuration size, you can: [merged YAML](pipeline_editor/index.md#view-expanded-configuration) tab. Look for duplicated configuration that can be removed or simplified. - Move long or repeated `script` sections into standalone scripts in the project. -- Use [parent and child pipelines](pipelines/parent_child_pipelines.md) to move some +- Use [parent and child pipelines](pipelines/downstream_pipelines.md#parent-child-pipelines) to move some work to jobs in an independent child pipeline. On a self-managed instance, you can [increase the size limits](../administration/instance_limits.md#maximum-size-and-depth-of-cicd-configuration-yaml-files). +### Error 500 when editing the `.gitlab-ci.yml` file + +A [loop of included configuration files](pipeline_editor/index.md#configuration-validation-currently-not-available-message) +can cause a `500` error when editing the `.gitlab-ci.yml` file with the [web editor](../user/project/repository/web_editor.md). + ## Pipeline warnings Pipeline configuration warnings are shown when you: diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 72df8d56815..25178903c9a 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -239,7 +239,7 @@ You can define instance variables via the UI or [API](../../api/instance_level_c To add an instance variable: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD** and expand the **Variables** section. 1. Select the **Add variable** button, and fill in the details: @@ -592,6 +592,7 @@ deploy: BUILD_VARIABLE: value_from_deploy_job script: - echo "$BUILD_VARIABLE" # Output is: 'value_from_build_job' due to precedence + environment: production ``` The [`dependencies`](../yaml/index.md#dependencies) or @@ -616,12 +617,19 @@ deploy_one: - echo "$BUILD_VERSION" # Output is: 'hello' dependencies: - build + environment: + name: customer1 + deployment_tier: production + deploy_two: stage: deploy script: - echo "$BUILD_VERSION" # Output is empty dependencies: [] + environment: + name: customer2 + deployment_tier: production deploy_three: stage: deploy @@ -629,6 +637,10 @@ deploy_three: - echo "$BUILD_VERSION" # Output is: 'hello' needs: - build + environment: + name: customer3 + deployment_tier: production + deploy_four: stage: deploy @@ -637,6 +649,9 @@ deploy_four: needs: job: build artifacts: true + environment: + name: customer4 + deployment_tier: production deploy_five: stage: deploy @@ -645,9 +660,12 @@ deploy_five: needs: job: build artifacts: false + environment: + name: customer5 + deployment_tier: production ``` -[Multi-project pipelines](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance) +[Multi-project pipelines](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job) can also inherit variables from their upstream pipelines. ## CI/CD variable precedence @@ -695,8 +713,8 @@ You can override the value of a variable when you: 1. Run a job manually in the UI. 1. Use [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd). 1. Trigger a pipeline by using [the API](../triggers/index.md#pass-cicd-variables-in-the-api-call). -1. Pass variables to a downstream pipeline [by using the `variable` keyword](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword) - or [by using variable inheritance](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance). +1. Pass variables to a downstream pipeline [by using the `variable` keyword](../pipelines/downstream_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline) + or [by using variable inheritance](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job). The pipeline variables declared in these events take [priority over other variables](#cicd-variable-precedence). diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index f8900501244..77005e1cec4 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -66,8 +66,8 @@ as it can cause the pipeline to behave unexpectedly. | `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the Docker image running the job. | | `CI_JOB_JWT` | 12.10 | all | A RS256 JSON web token to authenticate with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). | | `CI_JOB_JWT_V1` | 14.6 | all | The same value as `CI_JOB_JWT`. | -| `CI_JOB_JWT_V2` | 14.6 | all | [**alpha:**](../../policy/alpha-beta-support.md#alpha-features) A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. Format is subject to change. Be aware, the `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). | -| `CI_JOB_MANUAL` | 8.12 | all | `true` if a job was started manually. | +| `CI_JOB_JWT_V2` | 14.6 | all | A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. Format is subject to change. Be aware, the `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). **Note:** The `CI_JOB_JWT_V2` variable is available for testing, but the full feature is planned to be generally available when [issue 360657](https://gitlab.com/gitlab-org/gitlab/-/issues/360657) is complete.| +| `CI_JOB_MANUAL` | 8.12 | all | Only available if the job was started manually. `true` when available. | | `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job. | | `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the job's stage. | | `CI_JOB_STATUS` | all | 13.5 | The status of the job as each runner stage is executed. Use with [`after_script`](../yaml/index.md#after_script). Can be `success`, `failed`, or `canceled`. | @@ -117,6 +117,9 @@ as it can cause the pipeline to behave unexpectedly. | `CI_SERVER_PORT` | 12.8 | all | The port of the GitLab instance URL, without host or protocol. For example `8080`. | | `CI_SERVER_PROTOCOL` | 12.8 | all | The protocol of the GitLab instance URL, without host or port. For example `https`. | | `CI_SERVER_REVISION` | all | all | GitLab revision that schedules jobs. | +| `CI_SERVER_TLS_CA_FILE` | all | all | File containing the TLS CA certificate to verify the GitLab server when `tls-ca-file` set in [runner settings](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | +| `CI_SERVER_TLS_CERT_FILE` | all | all | File containing the TLS certificate to verify the GitLab server when `tls-cert-file` set in [runner settings](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | +| `CI_SERVER_TLS_KEY_FILE` | all | all | File containing the TLS key to verify the GitLab server when `tls-key-file` set in [runner settings](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | | `CI_SERVER_URL` | 12.7 | all | The base URL of the GitLab instance, including protocol and port. For example `https://gitlab.example.com:8080`. | | `CI_SERVER_VERSION_MAJOR` | 11.4 | all | The major version of the GitLab instance. For example, if the GitLab version is `13.6.1`, the `CI_SERVER_VERSION_MAJOR` is `13`. | | `CI_SERVER_VERSION_MINOR` | 11.4 | all | The minor version of the GitLab instance. For example, if the GitLab version is `13.6.1`, the `CI_SERVER_VERSION_MINOR` is `6`. | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index db83f2a14f3..6c1737f7c65 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -30,9 +30,10 @@ There are two places defined variables can be used. On the: | [`cache:key`](../yaml/index.md#cachekey) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | | [`environment:name`](../yaml/index.md#environmentname) | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | | [`environment:url`](../yaml/index.md#environmenturl) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | +| [`environment:auto_stop_in`](../yaml/index.md#environmentauto_stop_in)| yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/> The value of the variable being substituted should be a period of time in a human readable natural language form. See [possible inputs](../yaml/index.md#environmentauto_stop_in) for more information.| | [`except:variables`](../yaml/index.md#onlyvariables--exceptvariables) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | | [`image`](../yaml/index.md#image) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | -| [`include`](../yaml/index.md#include) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>Predefined project variables are supported: `GITLAB_FEATURES`, `CI_DEFAULT_BRANCH`, and all variables that start with `CI_PROJECT_` (for example `CI_PROJECT_NAME`). | +| [`include`](../yaml/index.md#include) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>See [Use variables with include](../yaml/includes.md#use-variables-with-include) for more information on supported variables. | | [`only:variables`](../yaml/index.md#onlyvariables--exceptvariables) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | | [`resource_group`](../yaml/index.md#resource_group) | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/>- `CI_ENVIRONMENT_URL`<br/>- [Persisted variables](#persisted-variables). | | [`rules:if`](../yaml/index.md#rulesif) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index 61ef8bbfab7..56a9da7cb84 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -80,20 +80,6 @@ GitLab can display the results of one or more reports in: - The [security dashboard](../../user/application_security/security_dashboard/index.md). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). -<!--- start_remove The following content will be removed on remove_date: '2023-08-22' --> - -## `artifacts:reports:cobertura` (removed) - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3708) in GitLab 12.9. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.7. -> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/348980) in GitLab 15.0. - -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.7 and -[removed](https://gitlab.com/gitlab-org/gitlab/-/issues/348980) in GitLab 15.0. Use `artifacts:reports:coverage_report` -instead. - -<!--- end_remove --> - ## `artifacts:reports:coverage_report` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344533) in GitLab 14.10. @@ -113,7 +99,8 @@ artifacts: path: coverage/cobertura-coverage.xml ``` -The collected coverage report is uploaded to GitLab as an artifact. +The collected coverage report is uploaded to GitLab as an artifact. You can use +only one report per job. GitLab can display the results of coverage report in the merge request [diff annotations](../testing/test_coverage_visualization.md). @@ -160,6 +147,30 @@ GitLab can display the results of one or more reports in: - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). - The [security dashboard](../../user/application_security/security_dashboard/index.md). +## `artifacts:reports:cyclonedx` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360766) in GitLab 15.3 + +This report is a Software Bill of Materials describing the components of a project +following the [cyclonedx](https://cyclonedx.org/docs/1.4) protocol format. + +You can specify multiple cyclonedx reports per job. These can be either supplied +as a list of filenames, a filename pattern, or both: + +- List of filenames: `cyclonedx: [gl-sbom-npm-npm.cdx.json, gl-sbom-bundler-gem.cdx.json]`. +- A filename pattern: `cyclonedx: gl-sbom-*.json`. +- Combination of both of the above: `cyclonedx: [gl-sbom-*.json, my-cyclonedx.json]`. + +Below is an example of a job exposing cyclonedx artifacts: + +```yaml +artifacts: + reports: + cyclonedx: + - gl-sbom-npm-npm.cdx.json + - gl-sbom-bundler-gem.cdx.json +``` + ## `artifacts:reports:dast` **(ULTIMATE)** The `dast` report collects [DAST vulnerabilities](../../user/application_security/dast/index.md). The collected DAST @@ -183,7 +194,7 @@ GitLab can display the results of one or more reports in: - The pipeline [**Security** tab](../../user/application_security/vulnerability_report/pipeline.md#view-vulnerabilities-in-a-pipeline). - The [security dashboard](../../user/application_security/security_dashboard/index.md). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). -- The [dependency list](../../user/application_security/dependency_list/). +- The [dependency list](../../user/application_security/dependency_list/index.md). ## `artifacts:reports:dotenv` @@ -329,27 +340,3 @@ GitLab can display the results of one or more reports in the merge request [terraform widget](../../user/infrastructure/iac/mr_integration.md#output-terraform-plan-information-into-a-merge-request). For more information, see [Output `terraform plan` information into a merge request](../../user/infrastructure/iac/mr_integration.md). - -## `artifacts:reports:cyclonedx` - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360766) in GitLab 15.3 - -This report is a Software Bill of Materials describing the components of a project -following the [cyclonedx](https://cyclonedx.org/docs/1.4) protocol format. - -You can specify multiple cyclonedx reports per job. These can be either supplied -as a list of filenames, a filename pattern, or both: - -- List of filenames: `cyclonedx: [gl-sbom-npm-npm.cdx.json, gl-sbom-bundler-gem.cdx.json]`. -- A filename pattern: `cyclonedx: gl-sbom-*.json`. -- Combination of both of the above: `cyclonedx: [gl-sbom-*.json, my-cyclonedx.json]`. - -Below is an example of a job exposing cyclonedx artifacts: - -```yaml -artifacts: - reports: - cyclonedx: - - gl-sbom-npm-npm.cdx.json - - gl-sbom-bundler-gem.cdx.json -``` diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index 0b434e7013c..8aa3ebd4759 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -156,6 +156,12 @@ the time limit to resolve all files is 30 seconds. - You can override included configuration by having the same job name or global keyword in the `.gitlab-ci.yml` file. The two configurations are merged together, and the configuration in the `.gitlab-ci.yml` file takes precedence over the included configuration. +- If you rerun a: + - Job, the `include` files are not fetched again. All jobs in a pipeline use the configuration + fetched when the pipeline was created. Any changes to the source `include` files + do not affect job reruns. + - Pipeline, the `include` files are fetched again. If they changed after the last + pipeline run, the new pipeline uses the changed configuration. **Related topics**: @@ -607,6 +613,7 @@ job3: stage: deploy script: - deploy_to_staging + environment: staging ``` In this example, `job1` and `job2` run in parallel: @@ -1384,7 +1391,7 @@ In this example: for the coverage number. - If there are multiple coverage numbers found in the matched fragment, the first number is used. - Leading zeros are removed. -- Coverage output from [child pipelines](../pipelines/parent_child_pipelines.md) +- Coverage output from [child pipelines](../pipelines/downstream_pipelines.md#parent-child-pipelines) is not recorded or displayed. Check [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/280818) for more details. @@ -1478,6 +1485,7 @@ test linux: deploy: stage: deploy script: make deploy + environment: production ``` In this example, two jobs have artifacts: `build osx` and `build linux`. When `test osx` is executed, @@ -1632,6 +1640,7 @@ these are all equivalent: - `168 hours` - `7 days` - `one week` +- `never` **Example of `environment:auto_stop_in`**: @@ -1896,13 +1905,10 @@ image: #### `image:pull_policy` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21619) in GitLab 15.1 [with a flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) in GitLab 15.2. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) in GitLab 15.4. [Feature flag `ci_docker_image_pull_policy`](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) removed. > - Requires GitLab Runner 15.1 or later. -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. -The feature is not ready for production use. - The pull policy that the runner uses to fetch the Docker image. **Keyword type**: Job keyword. You can use it only as part of a job or in the [`default` section](#default). @@ -2123,6 +2129,7 @@ mac:rspec: production: stage: deploy script: echo "Running production..." + environment: production ``` This example creates four paths of execution: @@ -2286,14 +2293,14 @@ build_job: **Related topics**: -- To download artifacts between [parent-child pipelines](../pipelines/parent_child_pipelines.md), +- To download artifacts between [parent-child pipelines](../pipelines/downstream_pipelines.md#parent-child-pipelines), use [`needs:pipeline:job`](#needspipelinejob). #### `needs:pipeline:job` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/255983) in GitLab 13.7. -A [child pipeline](../pipelines/parent_child_pipelines.md) can download artifacts from a job in +A [child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines) can download artifacts from a job in its parent pipeline or another child pipeline in the same parent-child pipeline hierarchy. **Keyword type**: Job keyword. You can use it only as part of a job. @@ -2385,12 +2392,14 @@ deploy-job: - job: test-job2 optional: true - job: test-job1 + environment: production review-job: stage: deploy needs: - job: test-job2 optional: true + environment: review ``` In this example: @@ -2472,7 +2481,7 @@ when to add jobs to pipelines. | `external` | When you use CI services other than GitLab. | | `external_pull_requests` | When an external pull request on GitHub is created or updated (See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests)). | | `merge_requests` | For pipelines created when a merge request is created or updated. Enables [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/merged_results_pipelines.md), and [merge trains](../pipelines/merge_trains.md). | - | `pipelines` | For [multi-project pipelines](../pipelines/multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](#trigger) keyword. | + | `pipelines` | For [multi-project pipelines](../pipelines/downstream_pipelines.md#multi-project-pipelines) created by [using the API with `CI_JOB_TOKEN`](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api), or the [`trigger`](#trigger) keyword. | | `pushes` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedules` | For [scheduled pipelines](../pipelines/schedules.md). | | `tags` | When the Git reference for a pipeline is a tag. | @@ -2620,7 +2629,7 @@ docker build: **Related topics**: - [`only: changes` and `except: changes` examples](../jobs/job_control.md#onlychanges--exceptchanges-examples). -- If you use `changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds), +- If you use `changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge), you should [also use `only:merge_requests`](../jobs/job_control.md#use-onlychanges-with-merge-request-pipelines). - [Jobs or pipelines can run unexpectedly when using `only: changes`](../jobs/job_control.md#jobs-or-pipelines-run-unexpectedly-when-using-changes). @@ -2671,6 +2680,7 @@ pages: - public rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + environment: production ``` This example moves all files from the root of the project to the `public/` directory. @@ -2752,6 +2762,7 @@ deploystacks: STACK: [monitoring, backup, app] - PROVIDER: [gcp, vultr] STACK: [data, processing] + environment: $PROVIDER/$STACK ``` The example generates 10 parallel `deploystacks` jobs, each with different values @@ -3224,8 +3235,7 @@ job: - Unlike variables in [`script`](../variables/index.md#use-cicd-variables-in-job-scripts) sections, variables in rules expressions are always formatted as `$VARIABLE`. - You can use `rules:if` with `include` to [conditionally include other configuration files](includes.md#use-rules-with-include). -- In [GitLab 15.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/35438), - variables on the right side of `=~` and `!~` expressions are evaluated as regular expressions. +- CI/CD variables on the right side of `=~` and `!~` expressions are [evaluated as regular expressions](../jobs/job_control.md#store-the-regex-pattern-in-a-variable). **Related topics**: @@ -3643,12 +3653,9 @@ in that container. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21619) in GitLab 15.1 [with a flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) in GitLab 15.2. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) in GitLab 15.4. [Feature flag `ci_docker_image_pull_policy`](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) removed. > - Requires GitLab Runner 15.1 or later. -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, -ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. - The pull policy that the runner uses to fetch the Docker image. **Keyword type**: Job keyword. You can use it only as part of a job or in the [`default` section](#default). @@ -3725,6 +3732,7 @@ job4: stage: deploy script: - echo "This job deploys the code. It runs when the test stage completes." + environment: production ``` **Additional details**: @@ -3881,54 +3889,46 @@ test: ### `trigger` -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8997) in GitLab Premium 11.8. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. +Use `trigger` to declare that a job is a "trigger job" which starts a +[downstream pipeline](../pipelines/downstream_pipelines.md) that is either: + +- [A multi-project pipeline](../pipelines/downstream_pipelines.md#multi-project-pipelines). +- [A child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines). -Use `trigger` to start a downstream pipeline that is either: +Trigger jobs can use only a limited set of GitLab CI/CD configuration keywords. +The keywords available for use in trigger jobs are: -- [A multi-project pipeline](../pipelines/multi_project_pipelines.md). -- [A child pipeline](../pipelines/parent_child_pipelines.md). +- [`trigger`](#trigger). +- [`stage`](#stage). +- [`allow_failure`](#allow_failure). +- [`rules`](#rules). +- [`only` and `except`](#only--except). +- [`when`](#when) (only with a value of `on_success`, `on_failure`, or `always`). +- [`extends`](#extends). +- [`needs`](#needs), but not [`needs:project`](#needsproject). **Keyword type**: Job keyword. You can use it only as part of a job. **Possible inputs**: -- For multi-project pipelines, path to the downstream project. CI/CD variables - [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) - in GitLab 15.3 and later. -- For child pipelines, path to the child pipeline CI/CD configuration file. +- For multi-project pipelines, the path to the downstream project. CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) + in GitLab 15.3 and later, but not [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables). + Alternatively, use [`trigger:project](#triggerproject). +- For child pipelines, use [`trigger:include`](#triggerinclude). -**Example of `trigger` for multi-project pipeline**: +**Example of `trigger`**: ```yaml -rspec: - stage: test - script: bundle exec rspec - -staging: - stage: deploy - trigger: my/deployment -``` - -**Example of `trigger` for child pipelines**: - -```yaml -trigger_job: - trigger: - include: path/to/child-pipeline.yml +trigger-multi-project-pipeline: + trigger: my-group/my-project ``` **Additional details**: -- Jobs with `trigger` can only use a [limited set of keywords](../pipelines/multi_project_pipelines.md#define-multi-project-pipelines-in-your-gitlab-ciyml-file). - For example, you can't run commands with [`script`](#script), [`before_script`](#before_script), - or [`after_script`](#after_script). Also, [`environment`](#environment) is not supported with `trigger`. - You [cannot use the API to start `when:manual` trigger jobs](https://gitlab.com/gitlab-org/gitlab/-/issues/284086). - In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201938), you can use [`when:manual`](#when) in the same job as `trigger`. In GitLab 13.4 and earlier, using them together causes the error `jobs:#{job-name} when should be on_success, on_failure or always`. -- In [GitLab 13.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/197140/), you can - view which job triggered a downstream pipeline in the [pipeline graph](../pipelines/index.md#visualize-pipelines). - [Manual pipeline variables](../variables/index.md#override-a-defined-cicd-variable) and [scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule) are not passed to downstream pipelines by default. Use [trigger:forward](#triggerforward) @@ -3938,12 +3938,75 @@ trigger_job: **Related topics**: -- [Multi-project pipeline configuration examples](../pipelines/multi_project_pipelines.md#define-multi-project-pipelines-in-your-gitlab-ciyml-file). -- [Child pipeline configuration examples](../pipelines/parent_child_pipelines.md#examples). +- [Multi-project pipeline configuration examples](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-from-a-job-in-your-gitlab-ciyml-file). - To run a pipeline for a specific branch, tag, or commit, you can use a [trigger token](../triggers/index.md) to authenticate with the [pipeline triggers API](../../api/pipeline_triggers.md). The trigger token is different than the `trigger` keyword. +#### `trigger:include` + +Use `trigger:include` to declare that a job is a "trigger job" which starts a +[child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines). + +Use `trigger:include:artifact` to trigger a [dynamic child pipeline](../pipelines/downstream_pipelines.md#dynamic-child-pipelines). + +**Keyword type**: Job keyword. You can use it only as part of a job. + +**Possible inputs**: + +- The path to the child pipeline's configuration file. + +**Example of `trigger:include`**: + +```yaml +trigger-child-pipeline: + trigger: + include: path/to/child-pipeline.gitlab-ci.yml +``` + +**Related topics**: + +- [Child pipeline configuration examples](../pipelines/downstream_pipelines.md#trigger-a-parent-child-pipeline). + +#### `trigger:project` + +Use `trigger:project` to declare that a job is a "trigger job" which starts a +[multi-project pipeline](../pipelines/downstream_pipelines.md#multi-project-pipelines). + +By default, the multi-project pipeline triggers for the default branch. Use `trigger:branch` +to specify a different branch. + +**Keyword type**: Job keyword. You can use it only as part of a job. + +**Possible inputs**: + +- The path to the downstream project. CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) + in GitLab 15.3 and later, but not [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables). + +**Example of `trigger:project`**: + +```yaml +trigger-multi-project-pipeline: + trigger: + project: my-group/my-project +``` + +**Example of `trigger:project` for a different branch**: + +```yaml +trigger-multi-project-pipeline: + trigger: + project: my-group/my-project + branch: development +``` + +**Related topics**: + +- [Multi-project pipeline configuration examples](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-from-a-job-in-your-gitlab-ciyml-file). +- To run a pipeline for a specific branch, tag, or commit, you can also use a [trigger token](../triggers/index.md) + to authenticate with the [pipeline triggers API](../../api/pipeline_triggers.md). + The trigger token is different than the `trigger` keyword. + #### `trigger:strategy` Use `trigger:strategy` to force the `trigger` job to wait for the downstream pipeline to complete @@ -3984,8 +4047,8 @@ successfully complete before starting. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/355572) in GitLab 15.1. [Feature flag `ci_trigger_forward_variables`](https://gitlab.com/gitlab-org/gitlab/-/issues/355572) removed. Use `trigger:forward` to specify what to forward to the downstream pipeline. You can control -what is forwarded to both [parent-child pipelines](../pipelines/parent_child_pipelines.md) -and [multi-project pipelines](../pipelines/multi_project_pipelines.md). +what is forwarded to both [parent-child pipelines](../pipelines/downstream_pipelines.md#parent-child-pipelines) +and [multi-project pipelines](../pipelines/downstream_pipelines.md#multi-project-pipelines). **Possible inputs**: @@ -4062,6 +4125,7 @@ deploy_job: stage: deploy script: - deploy-script --url $DEPLOY_SITE --path "/" + environment: production deploy_review_job: stage: deploy @@ -4069,6 +4133,7 @@ deploy_review_job: REVIEW_PATH: "/review" script: - deploy-review-script --url $DEPLOY_SITE --path $REVIEW_PATH + environment: production ``` **Additional details**: @@ -4161,6 +4226,7 @@ deploy_job: script: - make deploy when: manual + environment: production cleanup_job: stage: cleanup @@ -4194,20 +4260,6 @@ In this example, the script: The following keywords are deprecated. -<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> - -### Globally-defined `types` (removed) - -The `types` keyword was deprecated in GitLab 9.0, and [removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). -Use [`stages`](#stages) instead. - -### Job-defined `type` (removed) - -The `type` keyword was deprecated in GitLab 9.0, and [removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). -Use [`stage`](#stage) instead. - -<!--- end_remove --> - ### Globally-defined `image`, `services`, `cache`, `before_script`, `after_script` Defining `image`, `services`, `cache`, `before_script`, and diff --git a/doc/ci/yaml/script.md b/doc/ci/yaml/script.md index f1cdcf57e64..bd8d7f02a17 100644 --- a/doc/ci/yaml/script.md +++ b/doc/ci/yaml/script.md @@ -244,6 +244,7 @@ pages-job: stage: deploy script: - curl --header 'PRIVATE-TOKEN: ${PRIVATE_TOKEN}' "https://gitlab.example.com/api/v4/projects" + environment: production ``` The YAML parser thinks the `:` defines a YAML keyword, and outputs the @@ -257,6 +258,7 @@ pages-job: stage: deploy script: - 'curl --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "https://gitlab.example.com/api/v4/projects"' + environment: production ``` ### Job does not fail when using `&&` in a script diff --git a/doc/ci/yaml/workflow.md b/doc/ci/yaml/workflow.md index 743a2639c0c..d3e815c742d 100644 --- a/doc/ci/yaml/workflow.md +++ b/doc/ci/yaml/workflow.md @@ -176,7 +176,7 @@ include: If a merge request displays `Checking pipeline status.`, but the message never goes away (the "spinner" never stops spinning), it might be due to `workflow:rules`. -This issue can happen if a project has [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) +This issue can happen if a project has [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) enabled, but the `workflow:rules` prevent a pipeline from running for the merge request. For example, with this workflow, merge requests cannot be merged, because no diff --git a/doc/cloud_seed/index.md b/doc/cloud_seed/index.md index 77b22609b12..4c56b6c4aaa 100644 --- a/doc/cloud_seed/index.md +++ b/doc/cloud_seed/index.md @@ -66,7 +66,7 @@ The generated service account has the following roles: - `roles/cloudbuild.builds.builder` - `roles/run.admin` - `roles/storage.admin` -- `roles/cloudsql.admin` +- `roles/cloudsql.client` - `roles/browser` You can enhance security by storing CI variables in secret managers. Learn more about [secret management with GitLab](../ci/secrets/index.md). @@ -106,6 +106,53 @@ This creates a new branch with the Cloud Run deployment pipeline (or injected in and creates an associated merge request where the changes and deployment pipeline execution can be reviewed and merged into the main branch. +## Provision Cloud SQL Databases + +Relational database instances can be provisioned from the `Project :: Infrastructure :: Google Cloud` page. Cloud SQL is +the underlying Google Cloud service that is used to provision the database instances. + +The following databases and versions are supported: + +- PostgreSQL: 14, 13, 12, 11, 10 and 9.6 +- MySQL: 8.0, 5.7 and 5.6 +- SQL Server + - 2019: Standard, Enterprise, Express and Web + - 2017: Standard, Enterprise, Express and Web + +Google Cloud pricing applies. Please refer to the [Cloud SQL pricing page](https://cloud.google.com/sql/pricing). + +1. [Create a database instance](#create-a-database-instance) +1. [Database setup through a background worker](#database-setup-through-a-background-worker) +1. [Connect to the database](#connect-to-the-database) +1. [Managing the database instance](#managing-the-database-instance) + +### Create a database instance + +From the `Project :: Infrastructure :: Google Cloud` page, select the **Database** tab. Here you will find three +buttons to create Postgres, MySQL, and SQL Server database instances. + +The database instance creation form has fields for GCP project, Git ref (branch or tag), database version and +machine type. Upon submission, the database instance is created and the database setup is queued as a background job. + +### Database setup through a background worker + +Successful creation of the database instance triggers a background worker to perform the following tasks: + +- Create a database user +- Create a database schema +- Store the database details in the project's CI/CD variables + +### Connect to the database + +Once the database instance setup is complete, the database connection details are available as project variables. These +can be managed through the `Project :: Settings :: CI` page and are made available to pipeline executing in the +appropriate environment. + +### Managing the database instance + +The list of instances in the `Project :: Infrastructure :: Google Cloud :: Databases` links back to the Google Cloud +Console. Select an instance to view the details and manage the instance. + ## Contribute to Cloud Seed There are several ways you can contribute to Cloud Seed: diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md index 0b36b9b2f2f..673ec692bf4 100644 --- a/doc/development/api_graphql_styleguide.md +++ b/doc/development/api_graphql_styleguide.md @@ -77,6 +77,63 @@ The complexity score of a query [can itself be queried for](../api/graphql/getti Requests time out at 30 seconds. +### Limit maximum field call count + +In some cases, you want to prevent the evaluation of a specific field on multiple parent nodes +because it results in an N+1 query problem and there is no optimal solution. This should be +considered an option of last resort, to be used only when methods such as +[lookahead to preload associations](#look-ahead), or [using batching](graphql_guide/batchloader.md) +have been considered. + +For example: + +```graphql +# This usage is expected. +query { + project { + environments + } +} + +# This usage is NOT expected. +# It results in N+1 query problem. EnvironmentsResolver can't use GraphQL batch loader in favor of GraphQL pagination. +query { + projects { + nodes { + environments + } + } +} +``` + +To prevent this, you can use the `Gitlab::Graphql::Limit::FieldCallCount` extension on the field: + +```ruby +# This allows maximum 1 call to the `environments` field. If the field is evaluated on more than one node, +# it raises an error. +field :environments do + extension(::Gitlab::Graphql::Limit::FieldCallCount, limit: 1) + end +``` + +or you can apply the extension in a resolver class: + +```ruby +module Resolvers + class EnvironmentsResolver < BaseResolver + extension(::Gitlab::Graphql::Limit::FieldCallCount, limit: 1) + # ... + end +end +``` + +When you add this limit, make sure that the affected field's `description` is also updated accordingly. For example, + +```ruby +field :environments, + description: 'Environments of the project. This field can only be resolved for one project in any single request.' +``` + ## Breaking changes The GitLab GraphQL API is [versionless](https://graphql.org/learn/best-practices/#versioning) which means @@ -484,7 +541,7 @@ You can also [change or remove Alpha items at any time](#breaking-change-exemptions) without needing to deprecate them. When the flag is removed, "release" the schema item by removing its Alpha property to make it public. -### Descriptions for feature flagged items +### Descriptions for feature-flagged items When using a feature flag to toggle the value or behavior of a schema item, the `description` of the item must: @@ -494,7 +551,11 @@ When using a feature flag to toggle the value or behavior of a schema item, the - State what the field returns, or behavior is, when the feature flag is disabled (or enabled, if more appropriate). -Example of a feature-flagged field: +### Examples of using feature flags + +#### Feature-flagged field + +A field value is toggled based on the feature flag state. A common use is to return `null` if the feature flag is disabled: ```ruby field :foo, GraphQL::Types::String, null: true, @@ -507,7 +568,10 @@ def foo end ``` -Example of a feature-flagged argument: +#### Feature-flagged argument + +An argument can be ignored, or have its value changed, based on the feature flag state. +A common use is to ignore the argument when a feature flag is disabled: ```ruby argument :foo, type: GraphQL::Types::String, required: false, @@ -521,11 +585,23 @@ def resolve(args) end ``` -### `feature_flag` property (deprecated) +#### Feature-flagged mutation -NOTE: -This property is deprecated and should no longer be used. The property -has been temporarily renamed to `_deprecated_feature_flag` and support for it will be removed in [#369202](https://gitlab.com/gitlab-org/gitlab/-/issues/369202). +A mutation that cannot be performed due to a feature flag state is handled as a +[non-recoverable mutation error](#failure-irrelevant-to-the-user). The error is returned at the top level: + +```ruby +description 'Mutates an object. Does not mutate the object if ' \ + '`my_feature_flag` feature flag is disabled.' + +def resolve(id: ) + object = authorized_find!(id: id) + + raise Gitlab::Graphql::Errors::ResourceNotAvailable, '`my_feature_flag` feature flag is disabled.' \ + if Feature.disabled?(:my_feature_flag, object) + # ... +end +``` ## Deprecating schema items @@ -1611,7 +1687,7 @@ correctly rendered to the clients. ### Errors in mutations -We encourage following the practice of +We encourage following the practice of [errors as data](https://graphql-ruby.org/mutations/mutation_errors) for mutations, which distinguishes errors by who they are relevant to, defined by who can deal with them. diff --git a/doc/development/api_styleguide.md b/doc/development/api_styleguide.md index b72ef1bffc4..7f7d78bb58e 100644 --- a/doc/development/api_styleguide.md +++ b/doc/development/api_styleguide.md @@ -110,14 +110,14 @@ Model.create(foo: params[:foo]) With Grape v1.3+, Array types must be defined with a `coerce_with` block, or parameters, fails to validate when passed a string from an -API request. See the +API request. See the [Grape upgrading documentation](https://github.com/ruby-grape/grape/blob/master/UPGRADING.md#ensure-that-array-types-have-explicit-coercions) for more details. ### Automatic coercion of nil inputs Prior to Grape v1.3.3, Array parameters with `nil` values would -automatically be coerced to an empty Array. However, due to +automatically be coerced to an empty Array. However, due to [this pull request in v1.3.3](https://github.com/ruby-grape/grape/pull/2040), this is no longer the case. For example, suppose you define a PUT `/test` request that has an optional parameter: @@ -259,7 +259,7 @@ In situations where the same model has multiple entities in the API discretion with applying this scope. It may be that you optimize for the most basic entity, with successive entities building upon that scope. -The `with_api_entity_associations` scope also +The `with_api_entity_associations` scope also [automatically preloads data](https://gitlab.com/gitlab-org/gitlab/-/blob/19f74903240e209736c7668132e6a5a735954e7c/app%2Fmodels%2Ftodo.rb#L34) for `Todo` _targets_ when returned in the [to-dos API](../api/todos.md). diff --git a/doc/development/application_secrets.md b/doc/development/application_secrets.md index 06a38eb238a..93e43856b34 100644 --- a/doc/development/application_secrets.md +++ b/doc/development/application_secrets.md @@ -17,6 +17,7 @@ This page is a development guide for application secrets. |`db_key_base` | The base key to encrypt the data for `attr_encrypted` columns | |`openid_connect_signing_key` | The signing key for OpenID Connect | | `encrypted_settings_key_base` | The base key to encrypt settings files with | +| `ci_jwt_signing_key` | The base key for encrypting the `CI_JOB_JWT` and `CI_JOB_JWT_V2` predefined CI/CD variables | ## Where the secrets are stored diff --git a/doc/development/application_slis/index.md b/doc/development/application_slis/index.md index 27e69ff3445..cb2eb9b8d90 100644 --- a/doc/development/application_slis/index.md +++ b/doc/development/application_slis/index.md @@ -25,6 +25,7 @@ to be emitted from the rails application: ## Existing SLIs 1. [`rails_request_apdex`](rails_request_apdex.md) +1. `global_search_apdex` ## Defining a new SLI @@ -45,8 +46,8 @@ for clarity, they define different metric names: As shown in this example, they can share a base name (`foo` in this example). We recommend this when they refer to the same operation. -Before the first scrape, it is important to have -[initialized the SLI with all possible label-combinations](https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics). +Before the first scrape, it is important to have +[initialized the SLI with all possible label-combinations](https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics). This avoid confusing results when using these counters in calculations. To initialize an SLI, use the `.initialize_sli` class method, for @@ -135,10 +136,7 @@ After that, add the following information: into the error budgets for stage groups. - `description`: a Markdown string explaining the SLI. It will be shown on dashboards and alerts. -- `kind`: the kind of indicator. Only `sliDefinition.apdexKind` is supported at the moment. - Reach out in - [this issue](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1395) - if you want to implement an SLI for success or error rates. +- `kind`: the kind of indicator. For example `sliDefinition.apdexKind`. When done, run `make generate` to generate recording rules for the new SLI. This command creates recordings for all services @@ -152,9 +150,9 @@ When these changes are merged, and the aggregations in the success ratio of the new aggregated metrics. For example: ```prometheus -sum by (environment, stage, type)(gitlab_sli_aggregation:rails_request_apdex:apdex:success:rate_1h) +sum by (environment, stage, type)(application_sli_aggregation:rails_request:apdex:success:rate_1h) / -sum by (environment, stage, type)(gitlab_sli_aggregation:rails_request_apdex:apdex:weight:rate_1h) +sum by (environment, stage, type)(application_sli_aggregation:rails_request:apdex:weight:score_1h) ``` This shows the success ratio, which can guide you to set an diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 10d6c0ae9c9..a813072a976 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -626,7 +626,7 @@ Mattermost is an open source, private cloud, Slack-alternative from <https://mat - Layer: Core Service (Data) - GitLab.com: [Storage Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#storage-architecture) -MinIO is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups, and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. +MinIO is an object storage server released under the GNU AGPL v3.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups, and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. #### NGINX diff --git a/doc/development/audit_event_guide/index.md b/doc/development/audit_event_guide/index.md index 0c66189a6f6..50d7eeed107 100644 --- a/doc/development/audit_event_guide/index.md +++ b/doc/development/audit_event_guide/index.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -31,7 +31,7 @@ To instrument an audit event, the following attributes should be provided: | Attribute | Type | Required? | Description | |:-------------|:---------------------|:----------|:------------------------------------------------------------------| -| `name` | String | false | Action name to be audited. Used for error tracking | +| `name` | String | false | Action name to be audited. Represents the [type of the event](#event-type-definitions). Used for error tracking | | `author` | User | true | User who authors the change | | `scope` | User, Project, Group | true | Scope which the audit event belongs to | | `target` | Object | true | Target object being audited | @@ -40,17 +40,15 @@ To instrument an audit event, the following attributes should be provided: ## How to instrument new Audit Events -There are three ways of instrumenting audit events: +1. Create a [YAML type definition](#add-a-new-audit-event-type) for the new audit event. +1. Call `Gitlab::Audit::Auditor.audit`, passing an action block. + +The following ways of instrumenting audit events are deprecated: - Create a new class in `ee/lib/ee/audit/` and extend `AuditEventService` - Call `AuditEventService` after a successful action -- Call `Gitlab::Audit::Auditor.audit` passing an action block - -This inconsistency leads to unexpected bugs, increases maintainer effort, and worsens the -developer experience. Therefore, we suggest you use `Gitlab::Audit::Auditor` to -instrument new audit events. -With new service, we can instrument audit events in two ways: +With `Gitlab::Audit::Auditor` service, we can instrument audit events in two ways: - Using block for multiple events. - Using standard method call for single events. @@ -197,6 +195,34 @@ deactivate B In addition to recording to the database, we also write these events to [a log file](../../administration/logs/index.md#audit_jsonlog). +## Event type definitions + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367847) in GitLab 15.4. + +All new audit events must have a type definition stored in `config/audit_events/types/` that contains a single source of truth for every auditable event in GitLab. + +### Add a new audit event type + +To add a new audit event type: + +1. Create a new file in `config/audit_events/types/` with the filename matching the name of the event type. For example, a definition for the event type triggered when a + user is added to a project might be stored in `config/audit_events/types/project_add_user.yml`. +1. Add contents to the file that conform to the [schema](#schema) defined in `config/audit_events/types/type_schema.json`. +1. Ensure that all calls to `Gitlab::Audit::Auditor` use the `name` defined in your file. + +### Schema + +| Field | Required | Description | +| ----- | -------- |--------------| +| `name` | yes | Unique, lowercase and underscored name describing the type of event. Must match the filename. | +| `description` | yes | Human-readable description of how this event is triggered | +| `group` | yes | Name of the group that introduced this audit event. For example, `manage::compliance` | +| `introduced_by_issue` | yes | Issue URL that proposed the addition of this type | +| `introduced_by_mr` | yes | MR URL that added this new type | +| `milestone` | yes | Milestone in which this type was added | +| `saved_to_database` | yes | Indicate whether to persist events to database and JSON logs | +| `streamed` | yes | Indicate that events should be streamed to external services (if configured) | + ## Event streaming All events where the entity is a `Group` or `Project` are recorded in the audit log, and also streamed to one or more diff --git a/doc/development/auto_devops.md b/doc/development/auto_devops.md index 55ab234cc68..b9b8770207e 100644 --- a/doc/development/auto_devops.md +++ b/doc/development/auto_devops.md @@ -20,7 +20,7 @@ based on your project contents. When Auto DevOps is enabled for a project, the user does not need to explicitly include any pipeline configuration through a [`.gitlab-ci.yml` file](../ci/yaml/index.md). -In the absence of a `.gitlab-ci.yml` file, the +In the absence of a `.gitlab-ci.yml` file, the [Auto DevOps CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) is used implicitly to configure the pipeline for the project. This template is a top-level template that includes other sub-templates, diff --git a/doc/development/backend/create_source_code_be/index.md b/doc/development/backend/create_source_code_be/index.md index e1ee78731de..a1322b3fa25 100644 --- a/doc/development/backend/create_source_code_be/index.md +++ b/doc/development/backend/create_source_code_be/index.md @@ -33,6 +33,9 @@ GitLab Shell handles Git SSH sessions for GitLab and modifies the list of author For more information, [refer to the README](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/README.md). for GitLab Shell. +To learn about the reasoning behind our creation of `gitlab-sshd`, read the blog post +[Why we implemented our own SSHD solution](https://about.gitlab.com/blog/2022/08/17/why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass/). + ## GitLab Rails ### Gitaly touch points diff --git a/doc/development/backend/ruby_style_guide.md b/doc/development/backend/ruby_style_guide.md index a9fee02a15a..6ba5b8dd2c5 100644 --- a/doc/development/backend/ruby_style_guide.md +++ b/doc/development/backend/ruby_style_guide.md @@ -20,6 +20,17 @@ See also [guidelines for reusing abstractions](../reusing_abstractions.md). Everything listed here can be [reopened for discussion](https://about.gitlab.com/handbook/values/#disagree-commit-and-disagree). +## String literals quoting + +Due to the sheer amount of work to rectify, we do not care whether string +literals are single, or double quoted. + +Previous discussions include: + +- <https://gitlab.com/gitlab-org/gitlab-foss/-/issues/44234> +- <https://gitlab.com/gitlab-org/gitlab-foss/-/issues/36076> +- <https://gitlab.com/gitlab-org/gitlab/-/issues/198046> + ## Instance variable access using `attr_reader` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52351) in GitLab 14.1. @@ -72,3 +83,106 @@ end Public attributes should only be used if they are accessed outside of the class. There is not a strong opinion on what strategy is used when attributes are only accessed internally, as long as there is consistency in related code. + +## Newlines style guide + +This style guide recommends best practices for newlines in Ruby code. + +### Rule: separate code with newlines only to group together related logic + +```ruby +# bad +def method + issue = Issue.new + + issue.save + + render json: issue +end +``` + +```ruby +# good +def method + issue = Issue.new + issue.save + + render json: issue +end +``` + +### Rule: separate code and block with newlines + +#### Newline before block + +```ruby +# bad +def method + issue = Issue.new + if issue.save + render json: issue + end +end +``` + +```ruby +# good +def method + issue = Issue.new + + if issue.save + render json: issue + end +end +``` + +### Rule: Newline after block + +```ruby +# bad +def method + if issue.save + issue.send_email + end + render json: issue +end +``` + +```ruby +# good +def method + if issue.save + issue.send_email + end + + render json: issue +end +``` + +#### Exception: no need for newline when code block starts or ends right inside another code block + +```ruby +# bad +def method + + if issue + + if issue.valid? + issue.save + end + + end + +end +``` + +```ruby +# good +def method + if issue + if issue.valid? + issue.save + end + end +end +``` diff --git a/doc/development/build_test_package.md b/doc/development/build_test_package.md index 4645bd02d9e..97dd24fc522 100644 --- a/doc/development/build_test_package.md +++ b/doc/development/build_test_package.md @@ -13,7 +13,7 @@ pipeline that can be used to trigger a pipeline in the Omnibus GitLab repository that will create: - A deb package for Ubuntu 16.04, available as a build artifact, and -- A Docker image, which is pushed to the +- A Docker image, which is pushed to the [Omnibus GitLab container registry](https://gitlab.com/gitlab-org/omnibus-gitlab/container_registry) (images titled `gitlab-ce` and `gitlab-ee` respectively and image tag is the commit which triggered the pipeline). diff --git a/doc/development/changelog.md b/doc/development/changelog.md index c5b234069e3..c0296a6d75e 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -190,7 +190,7 @@ editor. Once closed, Git presents you with a new text editor instance to edit the commit message of commit B. Add the trailer, then save and quit the editor. If all went well, commit B is now updated. -For more information about interactive rebases, take a look at +For more information about interactive rebases, take a look at [the Git documentation](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History). --- diff --git a/doc/development/cicd/index.md b/doc/development/cicd/index.md index e8e116037de..7a2dfa01d1e 100644 --- a/doc/development/cicd/index.md +++ b/doc/development/cicd/index.md @@ -37,7 +37,7 @@ On the left side we have the events that can trigger a pipeline based on various - When a [merge request is created or updated](../../ci/pipelines/merge_request_pipelines.md). - When an MR is added to a [Merge Train](../../ci/pipelines/merge_trains.md#merge-trains). - A [scheduled pipeline](../../ci/pipelines/schedules.md). -- When project is [subscribed to an upstream project](../../ci/pipelines/multi_project_pipelines.md#trigger-a-pipeline-when-an-upstream-project-is-rebuilt). +- When project is [subscribed to an upstream project](../../ci/pipelines/index.md#trigger-a-pipeline-when-an-upstream-project-is-rebuilt). - When [Auto DevOps](../../topics/autodevops/index.md) is enabled. - When GitHub integration is used with [external pull requests](../../ci/ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). - When an upstream pipeline contains a [bridge job](../../ci/yaml/index.md#trigger) which triggers a downstream pipeline. diff --git a/doc/development/cicd/pipeline_wizard.md b/doc/development/cicd/pipeline_wizard.md index 7a0b70bd8e8..227a49d85db 100644 --- a/doc/development/cicd/pipeline_wizard.md +++ b/doc/development/cicd/pipeline_wizard.md @@ -58,7 +58,7 @@ consists of 2-3 steps, for a total of 3-4 steps visible to the user. ```yaml # ~/pipeline_wizard/templates/my_template.yml - +id: gitlab/my-template title: Set up my specific tech pipeline description: Here's two or three introductory sentences that help the user understand what this wizard is going to set up. steps: @@ -156,12 +156,13 @@ Webpack does not parse it as an Object. In the root element of the template file, you can define the following properties: -| Name | Required | Type | Description | -|---------------|------------------------|--------|---------------------------------------------------------------------------------------| -| `title` | **{check-circle}** Yes | string | The page title as displayed to the user. It becomes an `h1` heading above the wizard. | -| `description` | **{check-circle}** Yes | string | The page description as displayed to the user. | -| `filename` | **{dotted-circle}** No | string | The name of the file that is being generated. Defaults to `.gitlab-ci.yml`. | -| `steps` | **{check-circle}** Yes | list | A list of [step definitions](#step-reference). | +| Name | Required | Type | Description | +|---------------|------------------------|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `id` | **{check-circle}** Yes | string | A unique template ID. This ID should follow a namespacing pattern, with a forward slash `/` as separator. Templates committed to GitLab source code should always begin with `gitlab`. For example: `gitlab/my-template` | +| `title` | **{check-circle}** Yes | string | The page title as displayed to the user. It becomes an `h1` heading above the wizard. | +| `description` | **{check-circle}** Yes | string | The page description as displayed to the user. | +| `filename` | **{dotted-circle}** No | string | The name of the file that is being generated. Defaults to `.gitlab-ci.yml`. | +| `steps` | **{check-circle}** Yes | list | A list of [step definitions](#step-reference). | ### `step` Reference diff --git a/doc/development/cicd/templates.md b/doc/development/cicd/templates.md index 4ea7a9d960c..d0c56fb18bc 100644 --- a/doc/development/cicd/templates.md +++ b/doc/development/cicd/templates.md @@ -465,7 +465,10 @@ To add a metric definition for a new template: - `name:` and `performance_indicator_type:`: Delete (not needed). - `introduced_by_url:`: The URL of the MR adding the template. - `data_source:`: Set to `redis_hll`. - - All other fields that have no values: Set to empty strings (`''`). + - `description`: Add a short description of what this metric counts, for example: `Count of pipelines using the latest Auto Deploy template` + - `product_*`: Set to [section, stage, group, and feature category](https://about.gitlab.com/handbook/product/categories/#devops-stages) + as per the [metrics dictionary guide](../service_ping/metrics_dictionary.md#metrics-definition-and-validation). + If you are unsure what to use for these keywords, you can ask for help in the merge request. - Add the following to the end of each file: ```yaml diff --git a/doc/development/code_intelligence/index.md b/doc/development/code_intelligence/index.md index a89730383e4..87697a5e252 100644 --- a/doc/development/code_intelligence/index.md +++ b/doc/development/code_intelligence/index.md @@ -35,7 +35,7 @@ sequenceDiagram Workhorse-->>-Runner: request results ``` -1. The CI/CD job generates a document in an LSIF format (usually `dump.lsif`) using +1. The CI/CD job generates a document in an LSIF format (usually `dump.lsif`) using [an indexer](https://lsif.dev) for the language of a project. The format [describes](https://github.com/sourcegraph/sourcegraph/blob/main/doc/code_intelligence/explanations/writing_an_indexer.md) interactions between a method or function and its definitions or references. The diff --git a/doc/development/code_review.md b/doc/development/code_review.md index e9e546c6f9b..c320540401f 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -46,19 +46,28 @@ Read more about [author responsibilities](#the-responsibility-of-the-merge-reque Domain experts are team members who have substantial experience with a specific technology, product feature, or area of the codebase. Team members are encouraged to self-identify as -domain experts and add it to their [team profiles](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/team_members/person/README.md). +domain experts and add it to their +[team profiles](https://about.gitlab.com/handbook/engineering/workflow/code-review/#how-to-self-identify-as-a-domain-expert). When self-identifying as a domain expert, it is recommended to assign the MR changing the `.yml` file to be merged by an already established Domain Expert or a corresponding Engineering Manager. We make the following assumption with regards to automatically being considered a domain expert: -- Team members working in a specific stage/group (for example, create: source code) are considered domain experts for that area of the app they work on -- Team members working on a specific feature (for example, search) are considered domain experts for that feature +- Team members working in a specific stage/group (for example, create: source code) are considered domain experts for that area of the app they work on. +- Team members working on a specific feature (for example, search) are considered domain experts for that feature. We default to assigning reviews to team members with domain expertise. When a suitable [domain expert](#domain-experts) isn't available, you can choose any team member to review the MR, or simply follow the [Reviewer roulette](#reviewer-roulette) recommendation. -Team members' domain expertise can be viewed on the [engineering projects](https://about.gitlab.com/handbook/engineering/projects/) page or on the [GitLab team page](https://about.gitlab.com/company/team/). +To find a domain expert: + +- View the list of team members who work in the [stage or group](https://about.gitlab.com/handbook/product/categories/#devops-stages) related to the merge request. +- View team members' domain expertise on the [engineering projects](https://about.gitlab.com/handbook/engineering/projects/) page or on the [GitLab team page](https://about.gitlab.com/company/team/). Domains are self-identified, so use your judgment to map the changes on your merge request to a domain. +- Look for team members who have contributed to the files in the merge request. View the logs by running `git log <file>`. +- Look for team members who have reviewed the files. You can find the relevant merge request by: + 1. Getting the commit SHA by using `git log <file>`. + 1. Navigating to `https://gitlab.com/gitlab-org/gitlab/-/commit/<SHA>`. + 1. Selecting the related merge request shown for the commit. ### Reviewer roulette @@ -92,6 +101,12 @@ page, with these behaviors: - 3️⃣ - `:three:` - 4️⃣ - `:four:` - 5️⃣ - `:five:` + + Review requests for merge requests that do not target the default branch of any + project under the [security group](https://gitlab.com/gitlab-org/security/) are + not counted. These MRs are usually backports, and maintainers or reviewers usually + do not need much time reviewing them. + - Team members whose Slack or [GitLab status](../user/profile/index.md#set-your-current-status) emoji is 🔵 `:large_blue_circle:` are more likely to be picked. This applies to both reviewers and trainee maintainers. - Reviewers with 🔵 `:large_blue_circle:` are two times as likely to be picked as other reviewers. @@ -117,7 +132,7 @@ As an experiment, we want to introduce a `local` reviewer status for database re focusing on work from a team/stage, but not outside of it. This helps to focus and build great domain knowledge. We are not introducing changes to the reviewer roulette till we evaluate the impact and feedback from this experiment. We ask to respect reviewers who decline reviews based on their focus on `local` reviews. For tracking purposes, -please use in your personal YAML file entry: `- reviewer database local` instead of `- reviewer database`. +please use in your personal YAML file entry: `- reviewer database local` instead of `- reviewer database`. ### Approval guidelines @@ -125,40 +140,26 @@ As described in the section on the responsibility of the maintainer below, you are recommended to get your merge request approved and merged by maintainers with [domain expertise](#domain-experts). -1. If your merge request includes `~backend` changes (*1*), it must be - **approved by a [backend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_backend)**. -1. If your merge request includes database migrations or changes to expensive queries (*2*), it must be - **approved by a [database maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_database)**. - Read the [database review guidelines](database_review.md) for more details. -1. If your merge request includes `~frontend` changes (*1*), it must be - **approved by a [frontend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_frontend)**. -1. If your merge request includes (`~UX`) user-facing changes (*3*), it must be - **approved by a [Product Designer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_UX)**. - See the [design and user interface guidelines](contributing/design.md) for details. -1. If your merge request includes adding a new JavaScript library (*1*)... - - If the library significantly increases the - [bundle size](https://gitlab.com/gitlab-org/frontend/playground/webpack-memory-metrics/-/blob/master/doc/report.md), it must - be **approved by a [frontend foundations member](https://about.gitlab.com/direction/ecosystem/foundations/)**. - - If the license used by the new library hasn't been approved for use in - GitLab, the license must be **approved by a [legal department member](https://about.gitlab.com/handbook/legal/)**. - More information about license compatibility can be found in our - [GitLab Licensing and Compatibility documentation](licensing.md). -1. If your merge request includes a new dependency or a file system change, it must be - **approved by a [Distribution team member](https://about.gitlab.com/company/team/)**. See how to work with the [Distribution team](https://about.gitlab.com/handbook/engineering/development/enablement/systems/distribution/#how-to-work-with-distribution) for more details. -1. If your merge request includes documentation changes, it must be **approved - by a [Technical writer](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments)**, - based on assignments in the appropriate [DevOps stage group](https://about.gitlab.com/handbook/product/categories/#devops-stages). -1. If your merge request includes changes to development guidelines, follow the [review process](development_processes.md#development-guidelines-review) and get the approvals accordingly. -1. If your merge request includes end-to-end **and** non-end-to-end changes (*4*), it must be **approved - by a [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors)**. -1. If your merge request only includes end-to-end changes (*4*) **or** if the MR author is a [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors), it must be **approved by a [Quality maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_qa)** -1. If your merge request includes a new or updated [application limit](https://about.gitlab.com/handbook/product/product-processes/#introducing-application-limits), it must be **approved by a [product manager](https://about.gitlab.com/company/team/)**. -1. If your merge request includes Product Intelligence (telemetry or analytics) changes, it should be reviewed and approved by a [Product Intelligence engineer](https://gitlab.com/gitlab-org/analytics-section/product-intelligence/engineers). -1. If your merge request includes an addition of, or changes to a [Feature spec](testing_guide/testing_levels.md#frontend-feature-tests), it must be **approved by a [Quality maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_qa) or [Quality reviewer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_qa)**. -1. If your merge request introduces a new service to GitLab (Puma, Sidekiq, Gitaly are examples), it must be **approved by a [product manager](https://about.gitlab.com/company/team/)**. See the [process for adding a service component to GitLab](adding_service_component.md) for details. -1. If your merge request includes changes related to authentication or authorization, it must be **approved by a [Manage:Authentication and Authorization team member](https://about.gitlab.com/company/team/)**. Check the [code review section on the group page](https://about.gitlab.com/handbook/engineering/development/dev/manage/authentication-and-authorization/#additional-considerations) for more details. Patterns for files known to require review from the team are listed in the in the `Authentication and Authorization` section of the [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS) file, and the team will be listed in the approvers section of all merge requests that modify these files. - -- (*1*): Specs other than JavaScript specs are considered `~backend` code. Haml markup is considered `~frontend` code. However, Ruby code within Haml templates is considered `~backend` code. +| If your merge request includes | It must be approved by a | +| ------------------------------- | ------------------------ | +| `~backend` changes (*1*) | [Backend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_backend). | +| `~database` migrations or changes to expensive queries (*2*) | [Database maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_database). Refer to the [database review guidelines](database_review.md) for more details. | +| `~workhorse` changes | [Workhorse maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_workhorse). | +| `~frontend` changes (*1*) | [Frontend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_frontend). | +| `~UX` user-facing changes (*3*) | [Product Designer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_UX). Refer to the [design and user interface guidelines](contributing/design.md) for details. | +| Adding a new JavaScript library (*1*) | - [Frontend foundations member](https://about.gitlab.com/direction/ecosystem/foundations/) if the library significantly increases the [bundle size](https://gitlab.com/gitlab-org/frontend/playground/webpack-memory-metrics/-/blob/master/doc/report.md).<br/>- A [legal department member](https://about.gitlab.com/handbook/legal/) if the license used by the new library hasn't been approved for use in GitLab.<br/><br/>More information about license compatibility can be found in our [GitLab Licensing and Compatibility documentation](licensing.md). | +| A new dependency or a file system change | - [Distribution team member](https://about.gitlab.com/company/team/). See how to work with the [Distribution team](https://about.gitlab.com/handbook/engineering/development/enablement/systems/distribution/#how-to-work-with-distribution) for more details.<br/>- For Rubygems, request an [AppSec review](gemfile.md#request-an-appsec-review). | +| `~documentation` changes | [Technical writer](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments) based on assignments in the appropriate [DevOps stage group](https://about.gitlab.com/handbook/product/categories/#devops-stages). | +| Changes to development guidelines | Follow the [review process](development_processes.md#development-guidelines-review) and get the approvals accordingly. | +| End-to-end **and** non-end-to-end changes (*4*) | [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors). | +| Only End-to-end changes (*4*) **or** if the MR author is a [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors) | [Quality maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_qa). | +| A new or updated [application limit](https://about.gitlab.com/handbook/product/product-processes/#introducing-application-limits) | [Product manager](https://about.gitlab.com/company/team/). | +| Product Intelligence (telemetry or analytics) changes | [Product Intelligence engineer](https://gitlab.com/gitlab-org/analytics-section/product-intelligence/engineers). | +| An addition of, or changes to a [Feature spec](testing_guide/testing_levels.md#frontend-feature-tests) | [Quality maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_qa) or [Quality reviewer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_qa). | +| A new service to GitLab (Puma, Sidekiq, Gitaly are examples) | [Product manager](https://about.gitlab.com/company/team/). See the [process for adding a service component to GitLab](adding_service_component.md) for details. | +| Changes related to authentication or authorization | [Manage:Authentication and Authorization team member](https://about.gitlab.com/company/team/). Check the [code review section on the group page](https://about.gitlab.com/handbook/engineering/development/dev/manage/authentication-and-authorization/#additional-considerations) for more details. Patterns for files known to require review from the team are listed in the in the `Authentication and Authorization` section of the [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS) file, and the team will be listed in the approvers section of all merge requests that modify these files. | + +- (*1*): Specs other than JavaScript specs are considered `~backend` code. Haml markup is considered `~frontend` code. However, Ruby code within Haml templates is considered `~backend` code. When in doubt, request both a frontend and backend review. - (*2*): We encourage you to seek guidance from a database maintainer if your merge request is potentially introducing expensive queries. It is most efficient to comment on the line of code in question with the SQL queries so they can give their advice. @@ -333,33 +334,29 @@ Because a maintainer's job only depends on their knowledge of the overall GitLab codebase, and not that of any specific domain, they can review, approve, and merge merge requests from any team and in any product area. +Maintainers are the DRI of assuring that the acceptance criteria of a merge request are reasonably met. +In general, [quality is everyone’s responsibility](https://about.gitlab.com/handbook/engineering/quality/), +but maintainers of an MR are held responsible for **ensuring** that an MR meets those general quality standards. + +If a maintainer feels that an MR is substantial enough, or requires a [domain expert](#domain-experts), +maintainers have the discretion to request a review from another reviewer, or maintainer. Here are some +examples of maintainers proactively doing this during review: + +- <https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82708#note_872325561> +- <https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38003#note_387981596> +- <https://gitlab.com/gitlab-org/gitlab/-/merge_requests/14017#note_178828088> + Maintainers do their best to also review the specifics of the chosen solution before merging, but as they are not necessarily [domain experts](#domain-experts), they may be poorly placed to do so without an unreasonable investment of time. In those cases, they defer to the judgment of the author and earlier reviewers, in favor of focusing on their primary responsibilities. -If a maintainer feels that an MR is substantial enough that it warrants a review from a [domain expert](#domain-experts), -and it is unclear whether a domain expert have been involved in the reviews to date, -they may request a [domain expert's](#domain-experts) review before merging the MR. - If a developer who happens to also be a maintainer was involved in a merge request as a reviewer, it is recommended that they are not also picked as the maintainer to ultimately approve and merge it. Maintainers should check before merging if the merge request is approved by the required approvers. If still awaiting further approvals from others, remove yourself as a reviewer then `@` mention the author and explain why in a comment. Stay as reviewer if you're merging the code. -Maintainers must check before merging if the merge request is introducing new -vulnerabilities, by inspecting the list in the merge request -[Security Widget](../user/application_security/index.md). -When in doubt, a [Security Engineer](https://about.gitlab.com/company/team/) can be involved. The list of detected -vulnerabilities must be either empty or containing: - -- dismissed vulnerabilities in case of false positives -- vulnerabilities converted to issues - -Maintainers should **never** dismiss vulnerabilities to "empty" the list, -without duly verifying them. - Note that certain merge requests may target a stable branch. These are rare events. These types of merge requests cannot be merged by the Maintainer. Instead, these should be sent to the [Release Manager](https://about.gitlab.com/community/release-managers/). @@ -418,7 +415,7 @@ first time. codebase. Thorough descriptions help all reviewers understand your request and test effectively. - If you know your change depends on another being merged first, note it in the - description and set a [merge request dependency](../user/project/merge_requests/merge_request_dependencies.md). + description and set a [merge request dependency](../user/project/merge_requests/dependencies.md). - Be grateful for the reviewer's suggestions. ("Good call. I'll make that change.") - Don't take it personally. The review is of the code, not of you. - Explain why the code exists. ("It's like that because of these reasons. Would @@ -489,7 +486,7 @@ experience, refactors the existing code). Then: optionally resolve within the merge request or follow-up at a later stage. - There's a [Chrome/Firefox add-on](https://gitlab.com/conventionalcomments/conventional-comments-button) which you can use to apply [Conventional Comment](https://conventionalcomments.org/) prefixes. - Ensure there are no open dependencies. Check [linked issues](../user/project/issues/related_issues.md) for blockers. Clarify with the authors -if necessary. If blocked by one or more open MRs, set an [MR dependency](../user/project/merge_requests/merge_request_dependencies.md). +if necessary. If blocked by one or more open MRs, set an [MR dependency](../user/project/merge_requests/dependencies.md). - After a round of line notes, it can be helpful to post a summary note such as "Looks good to me", or "Just a couple things to address." - Let the author know if changes are required following your review. @@ -507,24 +504,25 @@ Before taking the decision to merge: before merging. A comment must be posted if the MR is merged with any failed job. - If the MR contains both Quality and non-Quality-related changes, the MR should be merged by the relevant maintainer for user-facing changes (backend, frontend, or database) after the Quality related changes are approved by a Software Engineer in Test. -If a merge request is fundamentally ready, but needs only trivial fixes (such as -typos), consider demonstrating a [bias for action](https://about.gitlab.com/handbook/values/#bias-for-action) -by making those changes directly without going back to the author. You can do this by -using the [suggest changes](../user/project/merge_requests/reviews/suggestions.md) feature to apply -your own suggestions to the merge request. Note that: - -- If the changes are not straightforward, please prefer allowing the author to make the change. -- **Before applying suggestions**, edit the merge request to make sure - [squash and merge](../user/project/merge_requests/squash_and_merge.md#squash-and-merge) - is enabled, otherwise, the pipeline's Danger job fails. - - If a merge request does not have squash and merge enabled, and it - has more than one commit, then see the note below about rewriting - commit history. - -Authors are not authorized to merge their own merge requests and need to seek another maintainer to merge. +At least one maintainer must approve an MR before it can be merged. MR authors and +people who add commits to an MR are not authorized to approve the merge request, +so they must seek a maintainer who has not contributed to the MR to approve the MR before it can be merged. + This policy is in place to satisfy the CHG-04 control of the GitLab [Change Management Controls](https://about.gitlab.com/handbook/engineering/security/security-assurance/security-compliance/guidance/change-management.html). +To implement this policy in `gitlab-org/gitlab`, we have enabled the following +settings to ensure MRs get an approval from a top-level CODEOWNERS maintainer: + +- [Prevent approval by author](../user/project/merge_requests/approvals/settings.md#prevent-approval-by-author). +- [Prevent approvals by users who add commits](../user/project/merge_requests/approvals/settings.md#prevent-approvals-by-users-who-add-commits). +- [Prevent editing approval rules in merge requests](../user/project/merge_requests/approvals/settings.md#prevent-editing-approval-rules-in-merge-requests). +- [Remove all approvals when commits are added to the source branch](../user/project/merge_requests/approvals/settings.md#remove-all-approvals-when-commits-are-added-to-the-source-branch) + + There are scenarios such as rebasing locally or applying suggestions that are considered + the same as adding a commit and could reset existing approvals. Approvals are not removed + when rebasing from the UI or with the [`/rebase` quick action](../user/project/quick_actions.md). + When ready to merge: WARNING: diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md index 6999ffe810e..2ff51e765a3 100644 --- a/doc/development/contributing/index.md +++ b/doc/development/contributing/index.md @@ -99,10 +99,14 @@ If you would like to contribute to GitLab: could speed them up. - Consult the [Contribution Flow](#contribution-flow) section to learn the process. -If you have any questions or need help visit [Getting Help](https://about.gitlab.com/get-help/) to -learn how to communicate with GitLab. We have a [Gitter channel for contributors](https://gitter.im/gitlab/contributors), -however we favor -[asynchronous communication](https://about.gitlab.com/handbook/communication/#internal-communication) over real time communication. +### Communication channels + +If you have any questions or need help, visit [Getting Help](https://about.gitlab.com/get-help/) to learn how to +communicate with the GitLab community. GitLab prefers [asynchronous communication](https://about.gitlab.com/handbook/communication/#internal-communication) over real-time communication. + +We do encourage you to connect and hang out with us. GitLab has a Gitter room dedicated for [contributors](https://gitter.im/gitlab/contributors), which is bridged with our +internal Slack. We actively monitor this channel. There is also a community-run [Discord server](https://discord.gg/S4cwz9sR8u) where you can +find other contributors in the `#contributors` channel. Thanks for your contribution! diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md index faa1642d50a..0e9ac569558 100644 --- a/doc/development/contributing/merge_request_workflow.md +++ b/doc/development/contributing/merge_request_workflow.md @@ -35,12 +35,23 @@ and see the [Development section](../../index.md) for the required guidelines. ## Merge request guidelines for contributors -If you find an issue, please submit a merge request with a fix or improvement, if -you can, and include tests. If you don't know how to fix the issue but can write a test -that exposes the issue, we will accept that as well. In general, bug fixes that -include a regression test are merged quickly, while new features without proper -tests might be slower to receive feedback. The workflow to make a merge -request is as follows: +If you find an issue, please submit a merge request with a fix or improvement, +if you can, and include tests. + +If the change is non-trivial, we encourage you to +start a discussion with [a product manager or a member of the team](https://about.gitlab.com/handbook/product/categories/). +You can do +this by tagging them in an MR before submitting the code for review. Talking +to team members can be helpful when making design decisions. Communicating the +intent behind your changes can also help expedite merge request reviews. + +If +you don't know how to fix the issue but can write a test that exposes the +issue, we will accept that as well. In general, bug fixes that include a +regression test are merged quickly. New features without proper tests +might be slower to receive feedback. + +To create a merge request: 1. [Fork](../../user/project/repository/forking_workflow.md) the project into your personal namespace (or group) on GitLab.com. @@ -199,48 +210,27 @@ To reach the definition of done, the merge request must create no regressions an - Verified as working in production on GitLab.com. - Verified as working for self-managed instances. -If a regression occurs, we prefer you revert the change. We break the definition of done into two phases: [MR Merge](#mr-merge) and [Production use](#production-use). +If a regression occurs, we prefer you revert the change. Your contribution is *incomplete* until you have made sure it meets all of these requirements. -### MR Merge +### Functionality -1. Clear title and description explaining the relevancy of the contribution. 1. Working and clean code that is commented where needed. 1. The change is evaluated to [limit the impact of far-reaching work](https://about.gitlab.com/handbook/engineering/development/#reducing-the-impact-of-far-reaching-work). -1. Testing: - - - [Unit, integration, and system tests](../testing_guide/index.md) that all pass - on the CI server. - - Peer member testing is optional but recommended when the risk of a change is high. - This includes when the changes are [far-reaching](https://about.gitlab.com/handbook/engineering/development/#reducing-the-impact-of-far-reaching-work) - or are for [components critical for security](../code_review.md#security). - - Description includes any steps or setup required to ensure reviewers can view the changes you've made (for example, include any information about feature flags). - - Regressions and bugs are covered with tests that reduce the risk of the issue happening - again. - - For tests that use Capybara, read - [how to write reliable, asynchronous integration tests](https://thoughtbot.com/blog/write-reliable-asynchronous-integration-tests-with-capybara). - - [Black-box tests/end-to-end tests](../testing_guide/testing_levels.md#black-box-tests-at-the-system-level-aka-end-to-end-tests) - added if required. Please contact [the quality team](https://about.gitlab.com/handbook/engineering/quality/#teams) - with any questions. - - The change is tested in a review app where possible and if appropriate. -1. In case of UI changes: - - - Use available components from the GitLab Design System, - [Pajamas](https://design.gitlab.com/). - - The MR must include *Before* and *After* screenshots if UI changes are made. - - If the MR changes CSS classes, please include the list of affected pages, which - can be found by running `grep css-class ./app -R`. +1. [Performance guidelines](../merge_request_performance_guidelines.md) have been followed. +1. [Secure coding guidelines](https://gitlab.com/gitlab-com/gl-security/security-guidelines) have been followed. +1. [Application and rate limit guidelines](../merge_request_application_and_rate_limit_guidelines.md) have been followed. +1. [Documented](../documentation/index.md) in the `/doc` directory. 1. If your MR touches code that executes shell commands, reads or opens files, or handles paths to files on disk, make sure it adheres to the [shell command guidelines](../shell_commands.md) 1. [Code changes should include observability instrumentation](../code_review.md#observability-instrumentation). 1. If your code needs to handle file storage, see the [uploads documentation](../uploads/index.md). -1. If your merge request adds one or more migrations: - - Make sure to execute all migrations on a fresh database before the MR is reviewed. - If the review leads to large changes in the MR, execute the migrations again - after the review is complete. - - Write tests for more complex migrations. +1. If your merge request adds one or more migrations, make sure to execute all migrations on a fresh database + before the MR is reviewed. + If the review leads to large changes in the MR, execute the migrations again + after the review is complete. 1. If your merge request adds new validations to existing models, to make sure the data processing is backwards compatible: @@ -259,12 +249,37 @@ requirements. [self-managed instances](../../subscriptions/self_managed/index.md), so keep that in mind for any data implications with your merge request. +### Testing + +1. [Unit, integration, and system tests](../testing_guide/index.md) that all pass + on the CI server. +1. Peer member testing is optional but recommended when the risk of a change is high. + This includes when the changes are [far-reaching](https://about.gitlab.com/handbook/engineering/development/#reducing-the-impact-of-far-reaching-work) + or are for [components critical for security](../code_review.md#security). +1. Regressions and bugs are covered with tests that reduce the risk of the issue happening + again. +1. For tests that use Capybara, read + [how to write reliable, asynchronous integration tests](https://thoughtbot.com/blog/write-reliable-asynchronous-integration-tests-with-capybara). +1. [Black-box tests/end-to-end tests](../testing_guide/testing_levels.md#black-box-tests-at-the-system-level-aka-end-to-end-tests) + added if required. Please contact [the quality team](https://about.gitlab.com/handbook/engineering/quality/#teams) + with any questions. +1. The change is tested in a review app where possible and if appropriate. 1. Code affected by a feature flag is covered by [automated tests with the feature flag enabled and disabled](../feature_flags/index.md#feature-flags-in-tests), or both states are tested as part of peer member testing or as part of the rollout plan. -1. [Performance guidelines](../merge_request_performance_guidelines.md) have been followed. -1. [Secure coding guidelines](https://gitlab.com/gitlab-com/gl-security/security-guidelines) have been followed. -1. [Application and rate limit guidelines](../merge_request_application_and_rate_limit_guidelines.md) have been followed. -1. [Documented](../documentation/index.md) in the `/doc` directory. +1. If your merge request adds one or more migrations, write tests for more complex migrations. + +### UI changes + +1. Use available components from the GitLab Design System, + [Pajamas](https://design.gitlab.com/). +1. The MR must include *Before* and *After* screenshots if UI changes are made. +1. If the MR changes CSS classes, please include the list of affected pages, which + can be found by running `grep css-class ./app -R`. + +### Description of changes + +1. Clear title and description explaining the relevancy of the contribution. +1. Description includes any steps or setup required to ensure reviewers can view the changes you've made (for example, include any information about feature flags). 1. [Changelog entry added](../changelog.md), if necessary. 1. If your merge request introduces changes that require additional steps when installing GitLab from source, add them to `doc/install/installation.md` in @@ -274,10 +289,13 @@ requirements. `doc/update/upgrading_from_source.md` in the same merge request. If these instructions are specific to a version, add them to the "Version specific upgrading instructions" section. -1. Reviewed by relevant reviewers, and all concerns are addressed for Availability, Regressions, and Security. Documentation reviews should take place as soon as possible, but they should not block a merge request. + +### Approval + 1. The [MR acceptance checklist](../code_review.md#acceptance-checklist) has been checked as confirmed in the MR. 1. Create an issue in the [infrastructure issue tracker](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues) to inform the Infrastructure department when your contribution is changing default settings or introduces a new setting, if relevant. 1. An agreed-upon [rollout plan](https://about.gitlab.com/handbook/engineering/development/processes/rollout-plans/). +1. Reviewed by relevant reviewers, and all concerns are addressed for Availability, Regressions, and Security. Documentation reviews should take place as soon as possible, but they should not block a merge request. 1. Your merge request has at least 1 approval, but depending on your changes you might need additional approvals. Refer to the [Approval guidelines](../code_review.md#approval-guidelines). - You don't have to select any specific approvers, but you can if you really want @@ -286,6 +304,8 @@ requirements. ### Production use +The following items are checked after the merge request has been merged: + 1. Confirmed to be working in staging before implementing the change in production, where possible. 1. Confirmed to be working in the production with no new [Sentry](https://about.gitlab.com/handbook/engineering/monitoring/#sentry) errors after the contribution is deployed. 1. Confirmed that the [rollout plan](https://about.gitlab.com/handbook/engineering/development/processes/rollout-plans/) has been completed. diff --git a/doc/development/contributing/style_guides.md b/doc/development/contributing/style_guides.md index 7a4ebbdbadf..2e696cf517b 100644 --- a/doc/development/contributing/style_guides.md +++ b/doc/development/contributing/style_guides.md @@ -146,13 +146,20 @@ reduces the aforementioned [bike-shedding](https://en.wiktionary.org/wiki/bikesh To that end, we encourage creation of new RuboCop rules in the codebase. -We currently maintain Cops across several Ruby code bases, and not all of them are +We maintain Cops across several Ruby code bases, and not all of them are specific to the GitLab application. When creating a new cop that could be applied to multiple applications, we encourage you to add it to our [GitLab Styles](https://gitlab.com/gitlab-org/gitlab-styles) gem. If the Cop targets rules that only apply to the main GitLab application, it should be added to [GitLab](https://gitlab.com/gitlab-org/gitlab) instead. +#### RuboCop node pattern + +When creating [node patterns](https://docs.rubocop.org/rubocop-ast/node_pattern.html) to match +Ruby's AST, you can use [`scripts/rubocop-parse`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/rubocop-parse) +to display the AST of a Ruby expression, in order to help you create the matcher. +See also [!97024](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97024). + ### Resolving RuboCop exceptions When the number of RuboCop exceptions exceed the default [`exclude-limit` of 15](https://docs.rubocop.org/rubocop/1.2/usage/basic_usage.html#command-line-flags), @@ -222,6 +229,34 @@ We're following [Ciro Santilli's Markdown Style Guide](https://cirosantilli.com/ See the dedicated [Documentation Style Guide](../documentation/styleguide/index.md). +### Guidelines for good practices + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36576/) in GitLab 13.2 as GitLab Development documentation. + +*Good practice* examples demonstrate encouraged ways of writing code while +comparing with examples of practices to avoid. These examples are labeled as +*Bad* or *Good*. In GitLab development guidelines, when presenting the cases, +it's recommended to follow a *first-bad-then-good* strategy. First demonstrate +the *Bad* practice (how things *could* be done, which is often still working +code), and then how things *should* be done better, using a *Good* example. This +is typically an improved example of the same code. + +Consider the following guidelines when offering examples: + +- First, offer the *Bad* example, and then the *Good* one. +- When only one bad case and one good case is given, use the same code block. +- When more than one bad case or one good case is offered, use separated code + blocks for each. With many examples being presented, a clear separation helps + the reader to go directly to the good part. Consider offering an explanation + (for example, a comment, or a link to a resource) on why something is bad + practice. +- Better and best cases can be considered part of the good cases' code block. + In the same code block, precede each with comments: `# Better` and `# Best`. + +Although the bad-then-good approach is acceptable for the GitLab development +guidelines, do not use it for user documentation. For user documentation, use +*Do* and *Don't*. For examples, see the [Pajamas Design System](https://design.gitlab.com/content/punctuation/). + ## Python See the dedicated [Python Development Guidelines](../python_guide/index.md). diff --git a/doc/development/database/add_foreign_key_to_existing_column.md b/doc/development/database/add_foreign_key_to_existing_column.md index 8a8fe3c0a1e..4be3296b2bb 100644 --- a/doc/development/database/add_foreign_key_to_existing_column.md +++ b/doc/development/database/add_foreign_key_to_existing_column.md @@ -71,7 +71,7 @@ Migration file for adding `NOT VALID` foreign key: ```ruby class AddNotValidForeignKeyToEmailsUser < Gitlab::Database::Migration[2.0] def up - add_concurrent_foreign_key :emails, :users, on_delete: :cascade, validate: false + add_concurrent_foreign_key :emails, :users, column: :user_id, on_delete: :cascade, validate: false end def down diff --git a/doc/development/database/adding_database_indexes.md b/doc/development/database/adding_database_indexes.md index 8abd7c8298e..774703bd54f 100644 --- a/doc/development/database/adding_database_indexes.md +++ b/doc/development/database/adding_database_indexes.md @@ -328,8 +328,8 @@ asynchronously during weekend hours. Due to generally lower traffic and fewer de index destruction can proceed at a lower level of risk. 1. [Schedule the index to be removed](#schedule-the-index-to-be-removed). -1. [Verify the MR was deployed and the index exists in production](#verify-the-mr-was-deployed-and-the-index-exists-in-production). -1. [Add a migration to create the index synchronously](#add-a-migration-to-create-the-index-synchronously). +1. [Verify the MR was deployed and the index exists in production](#verify-the-mr-was-deployed-and-the-index-no-longer-exists-in-production). +1. [Add a migration to destroy the index synchronously](#add-a-migration-to-destroy-the-index-synchronously). ### Schedule the index to be removed @@ -357,21 +357,21 @@ to remove them. You must test the database index changes locally before creating a merge request. -### Verify the MR was deployed and the index exists in production +### Verify the MR was deployed and the index no longer exists in production You can verify if the MR was deployed to GitLab.com with `/chatops run auto_deploy status <merge_sha>`. To verify the existence of the index, you can: - Use a meta-command in `#database-lab`, for example: `\d <index_name>`. - - Make sure the index is not [`invalid`](https://www.postgresql.org/docs/12/sql-createindex.html#:~:text=The%20psql%20%5Cd%20command%20will%20report%20such%20an%20index%20as%20INVALID). +- Make sure the index no longer exists - Ask someone in `#database` to check if the index exists. - If you have access, you can verify directly on production or in a production clone. ### Add a migration to destroy the index synchronously -After you verify the index exists in the production database, create a second +After you verify the index no longer exists in the production database, create a second merge request that removes the index synchronously. The schema changes must be updated and committed to `structure.sql` in this second merge request. The synchronous migration results in a no-op on GitLab.com, but you should still add the @@ -379,7 +379,7 @@ migration as expected for other installations. For example, to create the second migration for the previous asynchronous example: **WARNING:** -Verify that the index no longer exist in production before merging a second migration with `remove_concurrent_index_by_name`. +Verify that the index no longer exists in production before merging a second migration with `remove_concurrent_index_by_name`. If the second migration is deployed before the index has been destroyed, the index is destroyed synchronously when the second migration executes. @@ -395,7 +395,7 @@ def up end def down - add_concurrent_index :ci_builds, :some_column, INDEX_NAME + add_concurrent_index :ci_builds, :some_column, name: INDEX_NAME end ``` @@ -403,7 +403,7 @@ end To test changes for removing an index, use the asynchronous index helpers on your local environment: -1. Enable the feature flags by running `Feature.enable(:database_async_index_destruction)` and `Feature.enable(:database_reindexing)` in the Rails console. +1. Enable the feature flags by running `Feature.enable(:database_reindexing)` in the Rails console. 1. Run `bundle exec rails db:migrate` which should create an entry in the `postgres_async_indexes` table. 1. Run `bundle exec rails gitlab:db:reindex` destroy the index asynchronously. 1. To verify the index, open the PostgreSQL console by using the [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/postgresql.md) diff --git a/doc/development/database/batched_background_migrations.md b/doc/development/database/batched_background_migrations.md index edb22fcf436..192cd0d3e49 100644 --- a/doc/development/database/batched_background_migrations.md +++ b/doc/development/database/batched_background_migrations.md @@ -233,7 +233,7 @@ class CopyColumnUsingBackgroundMigrationJob < BatchedMigrationJob end ``` -### Additional filters +## Additional filters By default, when creating background jobs to perform the migration, batched background migrations iterate over the full specified table. This iteration is done using the @@ -276,6 +276,10 @@ In the second (filtered) example, we know exactly 100 will be updated with each end ``` + NOTE: + For EE migrations that define `scope_to`, ensure the module extends `ActiveSupport::Concern`. + Otherwise, records are processed without taking the scope into consideration. + 1. In the post-deployment migration, enqueue the batched background migration: ```ruby diff --git a/doc/development/database/ci_mirrored_tables.md b/doc/development/database/ci_mirrored_tables.md index 06f0087fafe..1d285e607fa 100644 --- a/doc/development/database/ci_mirrored_tables.md +++ b/doc/development/database/ci_mirrored_tables.md @@ -10,9 +10,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w As part of the database [decomposition work](https://gitlab.com/groups/gitlab-org/-/epics/6168), which had the goal of splitting the single database GitLab is using, into two databases: `main` and -`ci`, came the big challenge of +`ci`, came the big challenge of [removing all joins between the `main` and the `ci` tables](multiple_databases.md#removing-joins-between-ci-and-non-ci-tables). -That is because PostgreSQL doesn't support joins between tables that belong to different databases. +That is because PostgreSQL doesn't support joins between tables that belong to different databases. However, some core application models in the main database are queried very often by the CI side. For example: diff --git a/doc/development/database/client_side_connection_pool.md b/doc/development/database/client_side_connection_pool.md index 3cd0e836a8d..3143391a553 100644 --- a/doc/development/database/client_side_connection_pool.md +++ b/doc/development/database/client_side_connection_pool.md @@ -10,7 +10,7 @@ Ruby processes accessing the database through ActiveRecord, automatically calculate the connection-pool size for the process based on the concurrency. -Because of the way [Ruby on Rails manages database connections](#connection-lifecycle), +Because of the way [Ruby on Rails manages database connections](#connection-lifecycle), it is important that we have at least as many connections as we have threads. While there is a 'pool' setting in [`database.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/database.yml.postgresql), it is not very practical because you need to @@ -28,7 +28,7 @@ because connections are instantiated lazily. ## Troubleshooting connection-pool issues -The connection-pool usage can be seen per environment in the +The connection-pool usage can be seen per environment in the [connection-pool saturation dashboard](https://dashboards.gitlab.net/d/alerts-sat_rails_db_connection_pool/alerts-rails_db_connection_pool-saturation-detail?orgId=1). If the connection-pool is too small, this would manifest in diff --git a/doc/development/database/database_debugging.md b/doc/development/database/database_debugging.md index 5921dc942f2..591e526cc96 100644 --- a/doc/development/database/database_debugging.md +++ b/doc/development/database/database_debugging.md @@ -4,7 +4,7 @@ group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Troubleshooting and Debugging Database +# Troubleshooting and debugging the database This section is to help give some copy-pasta you can use as a reference when you run into some head-banging database problems. diff --git a/doc/development/database/loose_foreign_keys.md b/doc/development/database/loose_foreign_keys.md index 8dbccf048d7..0af12939629 100644 --- a/doc/development/database/loose_foreign_keys.md +++ b/doc/development/database/loose_foreign_keys.md @@ -221,7 +221,7 @@ ON DELETE CASCADE; ``` The migration must run after the `DELETE` trigger is installed and the loose -foreign key definition is deployed. As such, it must be a +foreign key definition is deployed. As such, it must be a [post-deployment migration](post_deployment_migrations.md) dated after the migration for the trigger. If the foreign key is deleted earlier, there is a good chance of introducing data inconsistency which needs manual cleanup: diff --git a/doc/development/database/multiple_databases.md b/doc/development/database/multiple_databases.md index 31fc454f8a7..034a2c2e438 100644 --- a/doc/development/database/multiple_databases.md +++ b/doc/development/database/multiple_databases.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Multiple Databases To allow GitLab to scale further we -[decomposed the GitLab application database into multiple databases](https://gitlab.com/groups/gitlab-org/-/epics/6168). +[decomposed the GitLab application database into multiple databases](https://gitlab.com/groups/gitlab-org/-/epics/6168). The two databases are `main` and `ci`. GitLab supports being run with either one database or two databases. On GitLab.com we are using two separate databases. diff --git a/doc/development/database/not_null_constraints.md b/doc/development/database/not_null_constraints.md index 9b3d017b09f..cd2adc3ca28 100644 --- a/doc/development/database/not_null_constraints.md +++ b/doc/development/database/not_null_constraints.md @@ -53,8 +53,13 @@ end ## Add a `NOT NULL` constraint to an existing column -Adding `NOT NULL` to existing database columns requires multiple steps split into at least two -different releases: +Adding `NOT NULL` to existing database columns usually requires multiple steps split into at least two +different releases. If your table is small enough that you don't need to +use a background migration, you can include all these in the same merge +request. We recommend to use separate migrations to reduce +transaction durations. + +The steps required are: 1. Release `N.M` (current release) diff --git a/doc/development/database/ordering_table_columns.md b/doc/development/database/ordering_table_columns.md index 7cd3d4fb208..a16df6a4499 100644 --- a/doc/development/database/ordering_table_columns.md +++ b/doc/development/database/ordering_table_columns.md @@ -117,7 +117,7 @@ divided into fixed size chunks as follows: This means that excluding the variable sized data and tuple header, we need at least 8 * 6 = 48 bytes per row. -We can optimise this by using the following column order instead: +We can optimize this by using the following column order instead: | Column | Type | Size | |:--------------|:----------------------------|:---------| diff --git a/doc/development/database/strings_and_the_text_data_type.md b/doc/development/database/strings_and_the_text_data_type.md index e2e1191018b..4b5d1fc8f72 100644 --- a/doc/development/database/strings_and_the_text_data_type.md +++ b/doc/development/database/strings_and_the_text_data_type.md @@ -148,7 +148,7 @@ to update the `title_html` with a title that has more than 1024 characters, the a database error. Adding or removing a constraint to an existing attribute requires that any application changes are -deployed _first_, +deployed _first_, otherwise servers still in the old version of the application [may try to update the attribute with invalid values](../multi_version_compatibility.md#ci-artifact-uploads-were-failing). For these reasons, `add_text_limit` should run in a post-deployment migration. diff --git a/doc/development/database/understanding_explain_plans.md b/doc/development/database/understanding_explain_plans.md index 446a84d5232..c3cb408b35f 100644 --- a/doc/development/database/understanding_explain_plans.md +++ b/doc/development/database/understanding_explain_plans.md @@ -252,7 +252,7 @@ A scan on an index that required retrieving some data from the table. Bitmap scans fall between sequential scans and index scans. These are typically used when we would read too much data from an index scan, but too little to -perform a sequential scan. A bitmap scan uses what is known as a +perform a sequential scan. A bitmap scan uses what is known as a [bitmap index](https://en.wikipedia.org/wiki/Bitmap_index) to perform its work. The [source code of PostgreSQL](https://gitlab.com/postgres/postgres/blob/REL_11_STABLE/src/include/nodes/plannodes.h#L441) @@ -295,9 +295,9 @@ because the previous node produced 36 rows. This means that nested loops can quickly slow the query down if the various child nodes keep producing many rows. -## Optimising queries +## Optimizing queries -With that out of the way, let's see how we can optimise a query. Let's use the +With that out of the way, let's see how we can optimize a query. Let's use the following query as an example: ```sql @@ -453,7 +453,7 @@ this works is that now PostgreSQL no longer needs to apply a `Filter`, as the index only contains `twitter` values that are not empty. Keep in mind that you shouldn't just add partial indexes every time you want to -optimise a query. Every index has to be updated for every write, and they may +optimize a query. Every index has to be updated for every write, and they may require quite a bit of space, depending on the amount of indexed data. As a result, first check if there are any existing indexes you may be able to reuse. If there aren't any, check if you can perhaps slightly change an existing one to @@ -471,10 +471,10 @@ buffer numbers. [Database Lab Engine](#database-lab-engine) guarantees that the identical to production (and overall number of buffers is the same as on production), but difference in cache state and I/O speed may lead to different timings. -## Queries that can't be optimised +## Queries that can't be optimized -Now that we have seen how to optimise a query, let's look at another query that -we might not be able to optimise: +Now that we have seen how to optimize a query, let's look at another query that +we might not be able to optimize: ```sql EXPLAIN (ANALYZE, BUFFERS) @@ -546,7 +546,7 @@ improve this query, other than _not_ running it at all. What is important here is that while some may recommend to straight up add an index the moment you see a sequential scan, it is _much more important_ to first understand what your query does, how much data it retrieves, and so on. After -all, you can not optimise something you do not understand. +all, you can not optimize something you do not understand. ### Cardinality and selectivity @@ -567,7 +567,7 @@ using an index is not worth it, because it would produce almost no unique rows. ## Rewriting queries -So the above query can't really be optimised as-is, or at least not much. But +So the above query can't really be optimized as-is, or at least not much. But what if we slightly change the purpose of it? What if instead of retrieving all projects with `visibility_level` 0 or 20, we retrieve those that a user interacted with somehow? diff --git a/doc/development/database_review.md b/doc/development/database_review.md index 2decd304103..14d73437c36 100644 --- a/doc/development/database_review.md +++ b/doc/development/database_review.md @@ -116,9 +116,9 @@ the following preparations into account. - Ensure that the Database Dictionary is updated as [documented](database/database_dictionary.md). - Make migrations reversible by using the `change` method or include a `down` method when using `up`. - Include either a rollback procedure or describe how to rollback changes. -- Add the output of both migrating (`db:migrate`) and rolling back (`db:rollback`) for all migrations into the MR description. - - Ensure the down method reverts the changes in `db/structure.sql`. - - Update the migration output whenever you modify the migrations during the review process. +- Check that the [`db:check-migrations`](database/dbcheck-migrations-job.md) pipeline job has run successfully and the migration rollback behaves as expected. + - Ensure the `db:check-schema` job has run successfully and no unexpected schema changes are introduced in a rollback. This job may only trigger a warning if the schema was changed. + - Verify that the previously mentioned jobs continue to succeed whenever you modify the migrations during the review process. - Add tests for the migration in `spec/migrations` if necessary. See [Testing Rails migrations at GitLab](testing_guide/testing_migrations_guide.md) for more details. - When [high-traffic](https://gitlab.com/gitlab-org/gitlab/-/blob/master/rubocop/rubocop-migrations.yml#L3) tables are involved in the migration, use the [`enable_lock_retries`](migration_style_guide.md#retry-mechanism-when-acquiring-database-locks) method to enable lock-retries. Review the relevant [examples in our documentation](migration_style_guide.md#usage-with-transactional-migrations) for use cases and solutions. - Ensure RuboCop checks are not disabled unless there's a valid reason to. @@ -181,9 +181,11 @@ Include in the MR description: - When providing query plans, make sure it hits enough data: - You can use a GitLab production replica to test your queries on a large scale, through the `#database-lab` Slack channel or through [ChatOps](database/understanding_explain_plans.md#chatops). - - Usually, the `gitlab-org` namespace (`namespace_id = 9970`) and the - `gitlab-org/gitlab-foss` (`project_id = 13083`) or the `gitlab-org/gitlab` (`project_id = 278964`) - projects provide enough data to serve as a good example. + - To produce a query plan with enough data, you can use the IDs of: + - The `gitlab-org` namespace (`namespace_id = 9970`), for queries involving a group. + - The `gitlab-org/gitlab-foss` (`project_id = 13083`) or the `gitlab-org/gitlab` (`project_id = 278964`) projects, for queries involving a project. + - The `gitlab-qa` user (`user_id = 1614863`), for queries involving a user. + - Optionally, you can also use your own `user_id`, or the `user_id` of a user with a long history within the project or group being used to generate the query plan. - That means that no query plan should return 0 records or less records than the provided limit (if a limit is included). If a query is used in batching, a proper example batch with adequate included results should be identified and provided. - If your queries belong to a new feature in GitLab.com and thus they don't return data in production: - You may analyze the query and to provide the plan from a local environment. diff --git a/doc/development/deprecation_guidelines/index.md b/doc/development/deprecation_guidelines/index.md index f0364f60d38..1e9d3ebda77 100644 --- a/doc/development/deprecation_guidelines/index.md +++ b/doc/development/deprecation_guidelines/index.md @@ -11,16 +11,23 @@ to GitLab features. ## Terminology +<!-- +If updating these definitions, be sure to update them in the handbook as well: +https://about.gitlab.com/handbook/product/gitlab-the-product/#definitions +--> + **Deprecation**: +- Required before ending support for a feature or removing a feature. - Feature not recommended for use. - Development restricted to Priority 1 / Severity 1 bug fixes. - Will be removed in a future major release. -- Begins after a deprecation announcement outlining an end-of-support date. +- Begins after a deprecation announcement outlining an end-of-support or removal date. - Ends after the end-of-support date or removal date has passed. **End of Support**: +- Optional step before removal. - Feature usage strongly discouraged. - No support or fixes provided. - No longer tested internally. @@ -28,6 +35,10 @@ to GitLab features. - Begins after an end-of-support date has passed. - Ends after all relevant code has been removed. +[Announcing an End of Support period](https://about.gitlab.com/handbook/marketing/blog/release-posts/#announcing-an-end-of-support-period) +should only be used in special circumstances and is not recommended for general use. +Most features should be deprecated and then removed. + **Removal**: - Feature usage impossible. diff --git a/doc/development/development_processes.md b/doc/development/development_processes.md index e199aedd3f5..27ebe98bc63 100644 --- a/doc/development/development_processes.md +++ b/doc/development/development_processes.md @@ -67,7 +67,7 @@ Some changes affect more than one group. For example: - Changes to [code review guidelines](code_review.md). - Changes to [commit message guidelines](contributing/merge_request_workflow.md#commit-messages-guidelines). -- Changes to guidelines in [feature flags in development of GitLab](feature_flags/). +- Changes to guidelines in [feature flags in development of GitLab](feature_flags/index.md). - Changes to [feature flags documentation guidelines](documentation/feature_flags.md). In these cases, use the following workflow: diff --git a/doc/development/distributed_tracing.md b/doc/development/distributed_tracing.md index f49d024095d..9d62f2061ca 100644 --- a/doc/development/distributed_tracing.md +++ b/doc/development/distributed_tracing.md @@ -73,13 +73,13 @@ In this example, we have the following hypothetical values: - `driver`: the driver such a Jaeger. - `param_name`, `param_value`: these are driver specific configuration values. Configuration - parameters for Jaeger are documented [further on in this document](#2-configure-the-gitlab_tracing-environment-variable) + parameters for Jaeger are documented [further on in this document](#2-configure-the-gitlab_tracing-environment-variable) they should be URL encoded. Multiple values should be separated by `&` characters like a URL. ## Using Jaeger in the GitLab Development Kit -The first tracing implementation that GitLab supports is Jaeger, and the +The first tracing implementation that GitLab supports is Jaeger, and the [GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit/) supports distributed tracing with Jaeger out-of-the-box. @@ -116,7 +116,7 @@ Jaeger has many configuration options, but is very easy to start in an "all-in-o memory for trace storage (and is therefore non-persistent). The main advantage of "all-in-one" mode being ease of use. -For more detailed configuration options, refer to the +For more detailed configuration options, refer to the [Jaeger documentation](https://www.jaegertracing.io/docs/1.9/getting-started/). #### Using Docker @@ -201,7 +201,7 @@ If `GITLAB_TRACING` is not configured correctly, this issue is logged: ``` By default, GitLab ships with the Jaeger tracer, but other tracers can be included at compile time. -Details of how this can be done are included in the +Details of how this can be done are included in the [LabKit tracing documentation](https://pkg.go.dev/gitlab.com/gitlab-org/labkit/tracing). If no log messages about tracing are emitted, the `GITLAB_TRACING` environment variable is likely diff --git a/doc/development/documentation/feature_flags.md b/doc/development/documentation/feature_flags.md index 89e54183e50..87d2930dcb5 100644 --- a/doc/development/documentation/feature_flags.md +++ b/doc/development/documentation/feature_flags.md @@ -24,7 +24,8 @@ When you document feature flags, you must: ## Add version history text -When the state of a flag changes (for example, disabled by default to enabled by default), add the change to the version history. +When the state of a flag changes (for example, disabled by default to enabled by default), add the change to the +[version history](versions.md#add-a-version-history-item). Possible version history entries are: diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index ee439e93011..73c1874f09e 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w description: Learn how to contribute to GitLab Documentation. --- -# GitLab Documentation guidelines +# GitLab documentation The GitLab documentation is [intended as the single source of truth (SSOT)](https://about.gitlab.com/handbook/documentation/) for information about how to configure, use, and troubleshoot GitLab. The documentation contains use cases and usage instructions for every GitLab feature, organized by product area and subject. This includes topics and workflows that span multiple GitLab features and the use of GitLab with other applications. @@ -42,7 +42,7 @@ run only the jobs that match the type of contribution. If your contribution cont **only** documentation changes, then only documentation-related jobs run, and the pipeline completes much faster than a code contribution. -If you are submitting documentation-only changes to Omnibus or Charts, +If you are submitting documentation-only changes to Omnibus, Charts, or Operator, the fast pipeline is not determined automatically. Instead, create branches for docs-only merge requests using the following guide: @@ -240,9 +240,9 @@ Every GitLab instance includes documentation at `/help` (`https://gitlab.example that matches the version of the instance. For example, <https://gitlab.com/help>. The documentation available online at <https://docs.gitlab.com> is deployed every -four hours from the default branch of [GitLab, Omnibus, Runner, and Charts](#source-files-and-rendered-web-locations). +hour from the default branch of [GitLab, Omnibus, Runner, and Charts](#source-files-and-rendered-web-locations). After a merge request that updates documentation is merged, it is available online -in 4 hours or less. +in an hour or less. However, it's only available at `/help` on self-managed instances in the next released version. The date an update is merged can impact which self-managed release the update @@ -380,6 +380,47 @@ is made, Danger Bot leaves a comment with further instructions about the documen process. This is configured in the `Dangerfile` in the GitLab repository under [/danger/documentation/](https://gitlab.com/gitlab-org/gitlab/-/tree/master/danger/documentation). +## Help and feedback section + +This section ([introduced](https://gitlab.com/gitlab-org/gitlab-docs/-/merge_requests/319) in GitLab 11.4) +is displayed at the end of each document and can be omitted by adding a key into +the front matter: + +```yaml +--- +feedback: false +--- +``` + +The default is to leave it there. If you want to omit it from a document, you +must check with a technical writer before doing so. + +## Disqus + +We have integrated the docs site with Disqus (introduced by +[!151](https://gitlab.com/gitlab-org/gitlab-docs/-/merge_requests/151)), +allowing our users to post comments. + +To omit only the comments from the feedback section, use the following key in +the front matter: + +```yaml +--- +comments: false +--- +``` + +We're hiding comments only in main index pages, such as [the main documentation index](../../index.md), +since its content is too broad to comment on. Before omitting Disqus, you must +check with a technical writer. + +Note that after adding `feedback: false` to the front matter, it will omit +Disqus, therefore, don't add both keys to the same document. + +The click events in the feedback section are tracked with Google Tag Manager. +The conversions can be viewed on Google Analytics by navigating to +**Behavior > Events > Top events > docs**. + ## Automatic screenshot generator You can now set up an automatic screenshot generator to take and compress screenshots with the diff --git a/doc/development/documentation/redirects.md b/doc/development/documentation/redirects.md index 4c748924c67..1bc697f2878 100644 --- a/doc/development/documentation/redirects.md +++ b/doc/development/documentation/redirects.md @@ -16,12 +16,7 @@ description: Learn how to contribute to GitLab Documentation. # Redirects in GitLab documentation When you move, rename, or delete a page, you must add a redirect. Redirects reduce -how often users get 404s when visiting the documentation site from out-of-date links, like: - -- Bookmarks -- Links from external sites -- Links from old blog posts -- Links in the documentation site global navigation +how often users get 404s when they visit the documentation site from out-of-date links. Add a redirect to ensure: @@ -36,9 +31,11 @@ Add a redirect to ensure: Be sure to assign a technical writer to any merge request that moves, renames, or deletes a page. Technical Writers can help with any questions and can review your change. +## Types of redirects + There are two types of redirects: -- [Redirect added into the documentation files themselves](#add-a-redirect), for users who +- [Redirects added into the documentation files themselves](#redirect-to-a-page-that-already-exists), for users who view the docs in `/help` on self-managed instances. For example, [`/help` on GitLab.com](https://gitlab.com/help). These must be added in the same MR that renames or moves a doc. Redirects to internal pages expire after three months @@ -52,96 +49,109 @@ Expired redirect files are removed from the documentation projects by the [`clean_redirects` Rake task](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/doc/raketasks.md#clean-up-redirects), as part of the Technical Writing team's [monthly tasks](https://gitlab.com/gitlab-org/technical-writing/-/blob/main/.gitlab/issue_templates/tw-monthly-tasks.md). -## Add a redirect +## Redirect to a page that already exists + +To redirect a page to another page in the same repository: + +1. In the Markdown file that you want to direct to a new location: + + - Delete all of the content. + - Add this content: + + ```markdown + --- + redirect_to: '../newpath/to/file/index.md' + remove_date: 'YYYY-MM-DD' + --- + + This document was moved to [another location](../path/to/file/index.md). + + <!-- This redirect file can be deleted after <YYYY-MM-DD>. --> + <!-- Redirects that point to other docs in the same project expire in three months. --> + <!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> + <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> + ``` + + - Replace both instances of `../newpath/to/file/index.md` with the new file path. + - Replace both instances of `YYYY-MM-DD` with the expiration date, as explained in the template. + +1. If the page has Disqus comments, follow [the steps for pages with Disqus comments](#redirections-for-pages-with-disqus-comments). +1. If the page had images that aren't used on any other pages, delete them. -NOTE: -If the renamed page is new, you can sometimes skip the following steps and ask a -Technical Writer to manually add the redirect to [`redirects.yaml`](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/content/_data/redirects.yaml). -For example, if you add a new page and then rename it before it's added to a release -on the 18th. The old page is not in any version's `/help` section, so a technical writer -can jump straight to the [Pages redirect](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/doc/maintenance.md#pages-redirects). +After your changes are committed, search for and update all links that point to the old file: -To add a redirect: +- In <https://gitlab.com/gitlab-com/www-gitlab-com>, search for full URLs: -1. In the repository (`gitlab`, `gitlab-runner`, `omnibus-gitlab`, or `charts`), - create a new documentation file. Don't delete the old one. The easiest - way is to copy it. For example: + ```shell + grep -r "docs.gitlab.com/ee/path/to/file.html" . + ``` - ```shell - cp doc/user/search/old_file.md doc/api/new_file.md +- In <https://gitlab.com/gitlab-org/gitlab-docs/-/tree/master/content/_data>, + search the navigation bar configuration files for the path with `.html`: + + ```shell + grep -r "path/to/file.html" . ``` -1. Add the redirect code to the old documentation file by running the - following Rake task. The first argument is the path of the old file, - and the second argument is the path of the new file: +- In any of the four internal projects, search for links in the docs + and codebase. Search for all variations, including full URL and just the path. + For example, go to the root directory of the `gitlab` project and run: - - To redirect to a page in the same project, use relative paths and - the `.md` extension. Both old and new paths start from the same location. - In the following example, both paths are relative to `doc/`: + ```shell + grep -r "docs.gitlab.com/ee/path/to/file.html" . + grep -r "path/to/file.html" . + grep -r "path/to/file.md" . + grep -r "path/to/file" . + ``` - ```shell - bundle exec rake "gitlab:docs:redirect[doc/user/search/old_file.md, doc/api/new_file.md]" - ``` + You might need to try variations of relative links, such as `../path/to/file` or + `../file` to find every case. - - To redirect to a page in a different project or site, use the full URL (with `https://`) : +### Move a file's location - ```shell - bundle exec rake "gitlab:docs:redirect[doc/user/search/old_file.md, https://example.com]" - ``` +If you want to move a file from one location to another, you do not move it. +Instead, you duplicate the file, and add the redirect code to the old file. - - Alternatively, you can omit the arguments and be prompted to enter the values: +1. Create the new file. +1. Copy the contents of the old file to the new one. +1. In the old file, delete all the content. +1. In the old file, add the redirect code and follow the rest of the steps in + the [Redirect to a page that already exists](#redirect-to-a-page-that-already-exists) topic. - ```shell - bundle exec rake gitlab:docs:redirect - ``` +## Use code to add a redirect - If you don't want to use the Rake task, you can use the following template: +If you prefer to use a script to create the redirect: - ```markdown - --- - redirect_to: '../newpath/to/file/index.md' - remove_date: 'YYYY-MM-DD' - --- +Add the redirect code to the old documentation file by running the +following Rake task. The first argument is the path of the old file, +and the second argument is the path of the new file: - This document was moved to [another location](../path/to/file/index.md). +- To redirect to a page in the same project, use relative paths and + the `.md` extension. Both old and new paths start from the same location. + In the following example, both paths are relative to `doc/`: - <!-- This redirect file can be deleted after <YYYY-MM-DD>. --> - <!-- Redirects that point to other docs in the same project expire in three months. --> - <!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> - <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> - ``` + ```shell + bundle exec rake "gitlab:docs:redirect[doc/user/search/old_file.md, doc/api/new_file.md]" + ``` - - Replace both instances of `../newpath/to/file/index.md` with the new file path. - - Replace `YYYY-MM-DD` with the expiry date, as explained in the template. - -1. If the documentation page being moved has any Disqus comments, follow the steps - described in [Redirections for pages with Disqus comments](#redirections-for-pages-with-disqus-comments). -1. Open a merge request with your changes. If a documentation page - you're removing includes images that aren't used - with any other documentation pages, be sure to use your merge request to delete - those images from the repository. -1. Assign the merge request to a technical writer for review and merge. -1. Search for links to the old documentation file. You must find and update all - links that point to the old documentation file: - - - In <https://gitlab.com/gitlab-com/www-gitlab-com>, search for full URLs: - `grep -r "docs.gitlab.com/ee/path/to/file.html" .` - - In <https://gitlab.com/gitlab-org/gitlab-docs/-/tree/master/content/_data>, - search the navigation bar configuration files for the path with `.html`: - `grep -r "path/to/file.html" .` - - In any of the four internal projects, search for links in the docs - and codebase. Search for all variations, including full URL and just the path. - For example, go to the root directory of the `gitlab` project and run: - - ```shell - grep -r "docs.gitlab.com/ee/path/to/file.html" . - grep -r "path/to/file.html" . - grep -r "path/to/file.md" . - grep -r "path/to/file" . - ``` +- To redirect to a page in a different project or site, use the full URL (with `https://`) : + + ```shell + bundle exec rake "gitlab:docs:redirect[doc/user/search/old_file.md, https://example.com]" + ``` + +- Alternatively, you can omit the arguments and be prompted to enter the values: + + ```shell + bundle exec rake gitlab:docs:redirect + ``` + +## Redirecting a page created before the release + +If you create a new page and then rename it before it's added to a release on the 18th: - You may need to try variations of relative links, such as `../path/to/file` or - `../file` to find every case. +Instead of following that procedure, ask a Technical Writer to manually add the redirect +to [`redirects.yaml`](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/content/_data/redirects.yaml). ## Redirections for pages with Disqus comments diff --git a/doc/development/documentation/restful_api_styleguide.md b/doc/development/documentation/restful_api_styleguide.md index bf1461a810d..2991c1b3224 100644 --- a/doc/development/documentation/restful_api_styleguide.md +++ b/doc/development/documentation/restful_api_styleguide.md @@ -59,12 +59,12 @@ METHOD /endpoint Supported attributes: -| Attribute | Type | Required | Description | -|:-------------------------|:---------|:-----------------------|:----------------------| -| `attribute` | datatype | **{check-circle}** Yes | Detailed description. | -| `attribute` **(<tier>)** | datatype | **{dotted-circle}** No | Detailed description. | -| `attribute` | datatype | **{dotted-circle}** No | Detailed description. | -| `attribute` | datatype | **{dotted-circle}** No | Detailed description. | +| Attribute | Type | Required | Description | +|:-------------------------|:---------|:---------|:----------------------| +| `attribute` | datatype | Yes | Detailed description. | +| `attribute` **(<tier>)** | datatype | No | Detailed description. | +| `attribute` | datatype | No | Detailed description. | +| `attribute` | datatype | No | Detailed description. | If successful, returns [`<status_code>`](../../api/index.md#status-codes) and the following response attributes: @@ -123,13 +123,13 @@ To deprecate an attribute: 1. Add inline deprecation text to the description. ```markdown - | Attribute | Type | Required | Description | - |:--------------|:-------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------| - | `widget_name` | string | **{dotted-circle}** No | [Deprecated](<link-to-issue>) in GitLab 14.7 and is planned for removal in 15.4. Use `widget_id` instead. The name of the widget. | + | Attribute | Type | Required | Description | + |:--------------|:-------|:---------|:----------------------------------------------------------------------------------------------------------------------------------------------------| + | `widget_name` | string | No | [Deprecated](<link-to-issue>) in GitLab 14.7 and is planned for removal in 15.4. Use `widget_id` instead. The name of the widget. | ``` To widely announce a deprecation, or if it's a breaking change, -[update the deprecations and removals documentation](../deprecation_guidelines/#update-the-deprecations-and-removals-documentation). +[update the deprecations and removals documentation](../deprecation_guidelines/index.md#update-the-deprecations-and-removals-documentation). ## Method description @@ -139,20 +139,20 @@ always be in code blocks using backticks (`` ` ``). Sort the table by required attributes first, then alphabetically. ```markdown -| Attribute | Type | Required | Description | -|:-----------------------------|:--------------|:-----------------------|:----------------------------------------------------| -| `title` | string | **{check-circle}** Yes | Title of the issue. | -| `assignee_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | IDs of the users to assign the issue to. | -| `confidential` | boolean | **{dotted-circle}** No | Sets the issue to confidential. Default is `false`. | +| Attribute | Type | Required | Description | +|:-----------------------------|:--------------|:---------|:----------------------------------------------------| +| `title` | string | Yes | Title of the issue. | +| `assignee_ids` **(PREMIUM)** | integer array | No | IDs of the users to assign the issue to. | +| `confidential` | boolean | No | Sets the issue to confidential. Default is `false`. | ``` Rendered example: -| Attribute | Type | Required | Description | -|:-----------------------------|:--------------|:-----------------------|:----------------------------------------------------| -| `title` | string | **{check-circle}** Yes | Title of the issue. | -| `assignee_ids` **(PREMIUM)** | integer array | **{dotted-circle}** No | IDs of the users to assign the issue to. | -| `confidential` | boolean | **{dotted-circle}** No | Sets the issue to confidential. Default is `false`. | +| Attribute | Type | Required | Description | +|:-----------------------------|:--------------|:---------|:----------------------------------------------------| +| `title` | string | Yes | Title of the issue. | +| `assignee_ids` **(PREMIUM)** | integer array | No | IDs of the users to assign the issue to. | +| `confidential` | boolean | No | Sets the issue to confidential. Default is `false`. | For information about writing attribute descriptions, see the [GraphQL API description style guide](../api_graphql_styleguide.md#description-style-guide). diff --git a/doc/development/documentation/review_apps.md b/doc/development/documentation/review_apps.md index 8cb9e6437b8..a50efceb307 100644 --- a/doc/development/documentation/review_apps.md +++ b/doc/development/documentation/review_apps.md @@ -57,7 +57,7 @@ If you want to know the in-depth details, here's what's really happening: The following GitLab features are used among others: - [Manual jobs](../../ci/jobs/job_control.md#create-a-job-that-must-be-run-manually) -- [Multi project pipelines](../../ci/pipelines/multi_project_pipelines.md) +- [Multi project pipelines](../../ci/pipelines/downstream_pipelines.md#multi-project-pipelines) - [Review Apps](../../ci/review_apps/index.md) - [Artifacts](../../ci/yaml/index.md#artifacts) - [Specific runner](../../ci/runners/runners_scope.md#prevent-a-specific-runner-from-being-enabled-for-other-projects) diff --git a/doc/development/documentation/site_architecture/deployment_process.md b/doc/development/documentation/site_architecture/deployment_process.md index 5f6076f3195..8a9c2e1e8d7 100644 --- a/doc/development/documentation/site_architecture/deployment_process.md +++ b/doc/development/documentation/site_architecture/deployment_process.md @@ -144,18 +144,21 @@ graph LR ### Manually deploy to production -GitLab Docs is deployed to production whenever the `Build docs.gitlab.com every 4 hours` scheduled pipeline runs. By -default, this pipeline runs every four hours. +GitLab Docs is deployed to production whenever the `Build docs.gitlab.com every hour` scheduled pipeline runs. By +default, this pipeline runs every hour. Maintainers can [manually](../../../ci/pipelines/schedules.md#run-manually) run this pipeline to force a deployment to production: 1. Go to the [scheduled pipelines](https://gitlab.com/gitlab-org/gitlab-docs/-/pipeline_schedules) for `gitlab-docs`. -1. Next to `Build docs.gitlab.com every 4 hours`, select **Play** (**{play}**). +1. Next to `Build docs.gitlab.com every hour`, select **Play** (**{play}**). The updated documentation is available in production after the `pages` and `pages:deploy` jobs complete in the new pipeline. +If you do not have the Maintainer role to perform this task, ask for help in the +`#docs` Slack channel. + ## Docker files The [`dockerfiles` directory](https://gitlab.com/gitlab-org/gitlab-docs/blob/main/dockerfiles/) contains all needed diff --git a/doc/development/documentation/site_architecture/index.md b/doc/development/documentation/site_architecture/index.md index 2864bbe7404..44e4aac2756 100644 --- a/doc/development/documentation/site_architecture/index.md +++ b/doc/development/documentation/site_architecture/index.md @@ -33,7 +33,7 @@ Then you can use one of these approaches: of the documentation in the external repository. The landing page is indexed and searchable on <https://docs.gitlab.com>, but the rest of the documentation is not. For example, the [GitLab Workflow extension for VS Code](../../../user/project/repository/vscode.md). - We do not encourage the use of [pages with lists of links](../structure.md#topics-and-resources-pages), + We do not encourage the use of [pages with lists of links](../topic_types/index.md#topics-and-resources), so only use this option if the recommended options are not feasible. ## Monthly release process (versions) diff --git a/doc/development/documentation/structure.md b/doc/development/documentation/structure.md index a5d1290a17a..35a93f08f66 100644 --- a/doc/development/documentation/structure.md +++ b/doc/development/documentation/structure.md @@ -1,395 +1,11 @@ --- -stage: none -group: Style Guide -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -description: What to include in GitLab documentation pages. +redirect_to: 'topic_types/index.md' +remove_date: '2022-11-16' --- -# Documentation topic types (CTRT) +This document was moved to [another location](topic_types/index.md). -At GitLab, we have not traditionally used types for our content. However, we are starting to -move in this direction, and we now use four primary topic types (CTRT): - -- [Concept](#concept) -- [Task](#task) -- [Reference](#reference) -- [Troubleshooting](#troubleshooting) - -In general, each page in our docset contains multiple topics. (Each heading indicates a new topic.) -Each topic on a page should be a specific topic type. For example, -a page with the title `Pipelines` can include topics that are concepts and tasks. - -A page might also contain only one type of information. These pages are generally one of our -[other content types](#other-types-of-content). - -## Concept - -A concept introduces a single feature or concept. - -A concept should answer the questions: - -- What is this? -- Why would I use it? - -Think of everything someone might want to know if they've never heard of this concept before. - -Don't tell them **how** to do this thing. Tell them **what it is**. - -If you start describing another concept, start a new concept and link to it. - -Concepts should be in this format: - -```markdown -# Title (a noun, like "Widgets") - -A paragraph that explains what this thing is. - -Another paragraph that explains what this thing is. - -Remember, if you start to describe about another concept, stop yourself. -Each concept should be about one concept only. -``` - -### Concept headings - -For the heading text, use a noun. For example, `Widgets` or `GDK dependency management`. - -If a noun is ambiguous, you can add a gerund. For example, `Documenting versions` instead of `Versions`. - -Avoid these heading titles: - -- `Overview` or `Introduction`. Instead, use a more specific - noun or phrase that someone would search for. -- `Use cases`. Instead, incorporate the information as part of the concept. -- `How it works`. Instead, use a noun followed by `workflow`. For example, `Merge request workflow`. - -## Task - -A task gives instructions for how to complete a procedure. - -Tasks should be in this format: - -```markdown -# Title (starts with an active verb, like "Create a widget" or "Delete a widget") - -Do this task when you want to... - -Prerequisites (optional): - -- Thing 1 -- Thing 2 -- Thing 3 - -To do this task: - -1. Location then action. (Go to this menu, then select this item.) -1. Another step. -1. Another step. - -Task result (optional). Next steps (optional). -``` - -Here is an example. - -```markdown -# Create an issue - -Create an issue when you want to track bugs or future work. - -Prerequisites: - -- You must have at least the Developer role for the project. - -To create an issue: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Issues > List**. -1. In the top right corner, select **New issue**. -1. Complete the fields. (If you have reference content that lists each field, link to it here.) -1. Select **Create issue**. - -The issue is created. You can view it by going to **Issues > List**. -``` - -### Task headings - -For the heading text, use the structure `active verb` + `noun`. -For example, `Create an issue`. - -If you have several tasks on a page that share prerequisites, you can use the title -`Prerequisites` and link to it. - -## Reference - -Reference information should be in an easily-scannable format, -like a table or list. It's similar to a dictionary or encyclopedia entry. - -```markdown -# Title (a noun, like "Pipeline settings" or "Administrator options") - -Introductory sentence. - -| Setting | Description | -|---------|-------------| -| **Name** | Descriptive sentence about the setting. | -``` - -### Reference headings - -Reference headings are usually nouns. - -Avoid these heading titles: - -- `Important notes`. Instead, incorporate this information - closer to where it belongs. For example, this information might be a prerequisite - for a task, or information about a concept. -- `Limitations`. Instead, move the content near other similar information. - If you must, you can use the title `Known issues`. - -## Troubleshooting - -Troubleshooting topics should be the last topics on a page. - -Troubleshooting can be one of three categories: - -- **An introductory topic.** This topic introduces the troubleshooting section of a page. - For example: - - ```markdown - ## Troubleshooting - - When working with <x feature>, you might encounter the following issues. - ``` - -- **Troubleshooting task.** The title should be similar to a [standard task](#task). - For example, "Run debug tools" or "Verify syntax." - -- **Troubleshooting reference.** This information includes the error message. For example: - - ```markdown - ### The error message or a description of it - - You might get an error that states <error message>. - - This issue occurs when... - - The workaround is... - ``` - - If multiple causes or workarounds exist, consider putting them into a table format. - If you use the exact error message, surround it in backticks so it's styled as code. - -If a page has more than five troubleshooting topics, put the content on a separate page that has troubleshooting information exclusively. Name the page `Troubleshooting <featurename>`. - -### Troubleshooting headings - -For the heading of a **Troubleshooting reference** topic: - -- Consider including at least a partial error message. -- Use fewer than 70 characters. - -If you do not put the full error in the title, include it in the body text. - -### Related topics - -If inline links are not sufficient, you can create a topic called **Related topics** -and include an unordered list of related topics. This topic should be above the Troubleshooting section. - -```markdown -# Related topics - -- [Configure your pipeline](link-to-topic) -- [Trigger a pipeline manually](link-to-topic) -``` - -## General heading text guidelines - -In general, for heading text: - -- Be clear and direct. Make every word count. -- Use articles and prepositions. -- Follow [capitalization](styleguide/index.md#capitalization) guidelines. -- Do not repeat text from earlier headings. For example, if the page is about merge requests, - instead of `Troubleshooting merge requests`, use only `Troubleshooting`. - -See also [guidelines for headings in Markdown](styleguide/index.md#headings-in-markdown). - -## Other types of content - -There are other types of content in the GitLab documentation that don't -classify as one of the four primary [topic types](#documentation-topic-types-ctrt). -These include: - -- [Tutorials](#tutorials) -- [Get started pages](#get-started) -- [Topics and resources pages](#topics-and-resources-pages) - -In most cases, these pages are standalone. - -### Tutorials - -A tutorial is an end-to-end walkthrough of a complex workflow or scenario. -In general, you might consider using a tutorial when: - -- The workflow requires a number of sequential steps where each step consists - of sub-steps. -- The steps cover a variety of GitLab features or third-party tools. - -Tutorials are learning aids that complement our core documentation. -They do not introduce new features. -Always use the primary [topic types](#documentation-topic-types-ctrt) to document new features. - -Tutorials should be in this format: - -```markdown -# Title (starts with "Tutorial:" followed by an active verb, like "Tutorial: Create a website") - -A paragraph that explains what the tutorial does, and the expected outcome. - -To create a website: - -1. [Do the first task](#do-the-first-task) -1. [Do the second task](#do-the-second-task) - -Prerequisites (optional): - -- Thing 1 -- Thing 2 -- Thing 3 - -## Do the first task - -To do step 1: - -1. First step. -1. Another step. -1. Another step. - -## Do the second task - -Before you begin, make sure you have [done the first task](#do-the-first-task). - -To do step 2: - -1. First step. -1. Another step. -1. Another step. -``` - -### Get started - -A get started page is a set of steps to help a user get set up -quickly to use a single GitLab feature or tool. -It might consist of more than one task. - -Get started pages should be in this format: - -```markdown -# Title ("Get started with <feature>") - -Complete the following steps to ... . - -1. First step. -1. Another step. -1. Another step. - -If you need to add more than one task, -consider using subsections for each distinct task. -``` - -### Topics and resources pages - -This page has a list of links that point to important sections -of documentation for a specific GitLab feature or tool. - -We do not encourage the use of these types of pages. -Lists like this can get out of date quickly and offer little value to users. -We've included this type here because: - -- There are existing pages in the documentation that follow this format, - and they should be standardized. -- They can sometimes help navigate a complex section of the documentation. - -If you come across a page like this -or you have to create one, use this format: - -```markdown -# Title ("Topics and resources for <feature>") - -Brief sentence to describe the feature. - -Refer to these resources for more information about <this feature>: - -- Link 1 -- Link 2 -- Link 3 -``` - -## Help and feedback section - -This section ([introduced](https://gitlab.com/gitlab-org/gitlab-docs/-/merge_requests/319) in GitLab 11.4) -is displayed at the end of each document and can be omitted by adding a key into -the front matter: - -```yaml ---- -feedback: false ---- -``` - -The default is to leave it there. If you want to omit it from a document, you -must check with a technical writer before doing so. - -### Disqus - -We also have integrated the docs site with Disqus (introduced by -[!151](https://gitlab.com/gitlab-org/gitlab-docs/-/merge_requests/151)), -allowing our users to post comments. - -To omit only the comments from the feedback section, use the following key in -the front matter: - -```yaml ---- -comments: false ---- -``` - -We're hiding comments only in main index pages, such as [the main documentation index](../../index.md), -since its content is too broad to comment on. Before omitting Disqus, you must -check with a technical writer. - -Note that after adding `feedback: false` to the front matter, it will omit -Disqus, therefore, don't add both keys to the same document. - -The click events in the feedback section are tracked with Google Tag Manager. -The conversions can be viewed on Google Analytics by navigating to -**Behavior > Events > Top events > docs**. - -## Guidelines for good practices - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36576/) in GitLab 13.2 as GitLab Development documentation. - -*Good practice* examples demonstrate encouraged ways of writing code while -comparing with examples of practices to avoid. These examples are labeled as -*Bad* or *Good*. In GitLab development guidelines, when presenting the cases, -it's recommended to follow a *first-bad-then-good* strategy. First demonstrate -the *Bad* practice (how things *could* be done, which is often still working -code), and then how things *should* be done better, using a *Good* example. This -is typically an improved example of the same code. - -Consider the following guidelines when offering examples: - -- First, offer the *Bad* example, and then the *Good* one. -- When only one bad case and one good case is given, use the same code block. -- When more than one bad case or one good case is offered, use separated code - blocks for each. With many examples being presented, a clear separation helps - the reader to go directly to the good part. Consider offering an explanation - (for example, a comment, or a link to a resource) on why something is bad - practice. -- Better and best cases can be considered part of the good cases' code block. - In the same code block, precede each with comments: `# Better` and `# Best`. - -Although the bad-then-good approach is acceptable for the GitLab development -guidelines, do not use it for user documentation. For user documentation, use -*Do* and *Don't*. For examples, see the [Pajamas Design System](https://design.gitlab.com/content/punctuation/). +<!-- This redirect file can be deleted after <2022-11-16>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
\ No newline at end of file diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md index 709e6b2d0d9..bc79bf0fbe2 100644 --- a/doc/development/documentation/styleguide/index.md +++ b/doc/development/documentation/styleguide/index.md @@ -887,7 +887,7 @@ To be consistent, use these templates when you write navigation steps in a task To open project settings: ```markdown -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. ``` @@ -895,7 +895,7 @@ To open project settings: To open group settings: ```markdown -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. ``` @@ -903,7 +903,7 @@ To open group settings: To open the Admin Area: ```markdown -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. ``` To select your avatar: @@ -950,7 +950,7 @@ Use the phrase **Complete the fields**. For example: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Push rules**. 1. Complete the fields. @@ -1361,6 +1361,48 @@ It renders on the GitLab documentation site as: > - This is a list item > - Second item in the list +## Tabs + +On the docs site, you can format text so it's displayed as tabs. + +To create a set of tabs, follow this example: + +```plaintext +::Tabs + +:::TabTitle Tab One + +Here's some content in tab one. + +:::TabTitle Tab Two + +Here's some other content in tab two. + +::EndTabs +``` + +This code renders on the GitLab documentation site as: + +::Tabs + +:::TabTitle Tab One + +Here's some content in tab one. + +:::TabTitle Tab Two + +Here's some other content in tab two. + +::EndTabs + +For tab titles, be brief and consistent. Ensure they are parallel, and start each with a capital letter. +For example: + +- `Omnibus package`, `Helm chart`, `Source` +- `15.1 and earlier`, `15.2 and later` + +See [Pajamas](https://design.gitlab.com/components/tabs/#guidelines) for details. + ## Terms To maintain consistency through GitLab documentation, use these styles and terms. diff --git a/doc/development/documentation/styleguide/word_list.md b/doc/development/documentation/styleguide/word_list.md index 1976caefc8e..029c7389290 100644 --- a/doc/development/documentation/styleguide/word_list.md +++ b/doc/development/documentation/styleguide/word_list.md @@ -8,8 +8,12 @@ description: 'Writing styles, markup, formatting, and other standards for GitLab # Recommended word list To help ensure consistency in the documentation, the Technical Writing team -recommends these wording choices. The GitLab handbook also maintains a list of -[top misused terms](https://about.gitlab.com/handbook/communication/top-misused-terms/). +recommends these word choices. In addition: + +- The GitLab handbook contains a list of + [top misused terms](https://about.gitlab.com/handbook/communication/top-misused-terms/). +- The documentation [style guide](../styleguide#language) includes details + about language and capitalization. For guidance not on this page, we defer to these style guides: @@ -19,6 +23,10 @@ For guidance not on this page, we defer to these style guides: <!-- vale off --> <!-- markdownlint-disable --> +## `&` + +Do not use Latin abbreviations. Use **and** instead, unless you are documenting a UI element that uses an `&`. + ## `@mention` Try to avoid **`@mention`**. Say **mention** instead, and consider linking to the @@ -75,7 +83,7 @@ Instead of: ## Admin Area -Use title case **Admin Area** to refer to the area of the UI that you access when you select **Menu > Admin**. +Use title case **Admin Area** to refer to the area of the UI that you access when you select **Main menu > Admin**. This area of the UI says **Admin Area** at the top of the page and on the menu. ## agent @@ -139,6 +147,18 @@ Do not use **and so on**. Instead, be more specific. For details, see Use [**section**](#section) instead of **area**. The only exception is [the Admin Area](#admin-area). +## as + +Do not use **as** to mean **because**. + +Use: + +- Because none of the endpoints return an ID... + +Instead of: + +- As none of the endpoints return an ID... + ## associate Do not use **associate** when describing adding issues to epics, or users to issues, merge requests, @@ -265,6 +285,13 @@ Use title case for the GitLab Container Registry. Do not use **currently** when talking about the product or its features. The documentation describes the product as it is today. ([Vale](../testing.md#vale) rule: [`CurrentStatus.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/CurrentStatus.yml)) +## default branch + +Use **default branch** to refer generically to the primary branch in the repository. +Users can set the default branch by using a UI setting. + +For examples that use the default branch, use `main` instead of [`master`](#master). + ## Dependency Proxy Use title case for the GitLab Dependency Proxy. @@ -394,7 +421,7 @@ Information in FAQs belongs with other similar information, under an easily sear ## field -Use **box** instead of **field** or **text box**. +Use **text box** instead of **field** or **box**. Use: @@ -407,7 +434,7 @@ Instead of: However, you can make an exception when you are writing a task and you need to refer to all of the fields at once. For example: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Complete the fields. @@ -604,6 +631,10 @@ Instead of: - Buy a license. - Purchase a license. +## limitations + +Do not use **limitations**. Use **known issues** instead. + ## log in, log on Do not use **log in** or **log on**. Use [sign in](#sign-in) instead. If the user interface has **Log in**, you can use it. @@ -644,7 +675,8 @@ Do not use **manpower**. Use words like **workforce** or **GitLab team members** ## master -Do not use **master**. Options are **primary** or **main**. ([Vale](../testing.md#vale) rule: [`InclusionCultural.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/InclusionCultural.yml)) +Do not use `master`. Use `main` when you need a sample [default branch name](#default-branch). +([Vale](../testing.md#vale) rule: [`InclusionCultural.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/InclusionCultural.yml)) ## may, might diff --git a/doc/development/documentation/testing.md b/doc/development/documentation/testing.md index 428a57a11fb..59a078bdec0 100644 --- a/doc/development/documentation/testing.md +++ b/doc/development/documentation/testing.md @@ -190,7 +190,7 @@ To update the linting images: 1. In `gitlab-docs`, open a merge request to update `.gitlab-ci.yml` to use the new tooling version. ([Example MR](https://gitlab.com/gitlab-org/gitlab-docs/-/merge_requests/2571)) -1. When merged, start a `Build docs.gitlab.com every 4 hours` [scheduled pipeline](https://gitlab.com/gitlab-org/gitlab-docs/-/pipeline_schedules). +1. When merged, start a `Build docs.gitlab.com every hour` [scheduled pipeline](https://gitlab.com/gitlab-org/gitlab-docs/-/pipeline_schedules). 1. Go the pipeline you started, and manually run the relevant build-images job, for example, `image:docs-lint-markdown`. 1. In the job output, get the name of the new image. diff --git a/doc/development/documentation/topic_types/concept.md b/doc/development/documentation/topic_types/concept.md new file mode 100644 index 00000000000..a20bb93a97f --- /dev/null +++ b/doc/development/documentation/topic_types/concept.md @@ -0,0 +1,46 @@ +--- +stage: none +group: Style Guide +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Concept topic type + +A concept introduces a single feature or concept. + +A concept should answer the questions: + +- What is this? +- Why would I use it? + +Think of everything someone might want to know if they've never heard of this concept before. + +Don't tell them **how** to do this thing. Tell them **what it is**. + +If you start describing another concept, start a new concept and link to it. + +Concepts should be in this format: + +```markdown +# Title (a noun, like "Widgets") + +A paragraph that explains what this thing is. + +Another paragraph that explains what this thing is. + +Remember, if you start to describe about another concept, stop yourself. +Each concept should be about one concept only. +``` + +## Concept headings + +For the heading text, use a noun. For example, `Widgets` or `GDK dependency management`. + +If a noun is ambiguous, you can add a gerund. For example, `Documenting versions` instead of `Versions`. + +Avoid these heading titles: + +- `Overview` or `Introduction`. Instead, use a more specific + noun or phrase that someone would search for. +- `Use cases`. Instead, incorporate the information as part of the concept. +- `How it works`. Instead, use a noun followed by `workflow`. For example, `Merge request workflow`. diff --git a/doc/development/documentation/topic_types/index.md b/doc/development/documentation/topic_types/index.md new file mode 100644 index 00000000000..af3e66fe87a --- /dev/null +++ b/doc/development/documentation/topic_types/index.md @@ -0,0 +1,130 @@ +--- +stage: none +group: Style Guide +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Documentation topic types (CTRT) + +At GitLab, we have not traditionally used types for our content. However, we are starting to +move in this direction, and we now use four primary topic types: + +- [Concept](concept.md) +- [Task](task.md) +- [Reference](reference.md) +- [Troubleshooting](troubleshooting.md) + +The tech writing team sometimes uses the acronym `CTRT` to refer to our topic types. +The acronym refers to the first letter of each topic type. + +In general, each page in the GitLab documentation contains multiple topics. +Each topic on a page should be recognizable as a specific topic type. + +## Other topic types + +In addition to the four primary topic types, we have a few other types. + +### Related topics + +If inline links are not sufficient, you can create a topic called **Related topics** +and include an unordered list of related topics. This topic should be above the Troubleshooting section. + +```markdown +# Related topics + +- [Configure your pipeline](link-to-topic). +- [Trigger a pipeline manually](link-to-topic). +``` + +### Tutorials + +A tutorial is page that contains an end-to-end walkthrough of a complex workflow or scenario. +In general, you might consider using a tutorial when: + +- The workflow requires a number of sequential steps where each step consists + of sub-steps. +- The steps cover a variety of GitLab features or third-party tools. + +Tutorials are learning aids that complement our core documentation. +They do not introduce new features. +Always use the primary [topic types](#documentation-topic-types-ctrt) to document new features. + +Tutorials should be in this format: + +```markdown +# Title (starts with "Tutorial:" followed by an active verb, like "Tutorial: Create a website") + +A paragraph that explains what the tutorial does, and the expected outcome. + +To create a website: + +1. [Do the first task](#do-the-first-task) +1. [Do the second task](#do-the-second-task) + +Prerequisites (optional): + +- Thing 1 +- Thing 2 +- Thing 3 + +## Do the first task + +To do step 1: + +1. First step. +1. Another step. +1. Another step. + +## Do the second task + +Before you begin, make sure you have [done the first task](#do-the-first-task). + +To do step 2: + +1. First step. +1. Another step. +1. Another step. +``` + +### Get started + +A get started page is a set of steps to help a user get set up +quickly to use a single GitLab feature or tool. +It consists of more than one task. + +Get started pages should be in this format: + +```markdown +# Title ("Get started with <feature>") + +Complete the following steps to ... . + +1. First step. +1. Another step. +1. Another step. + +If you need to add more than one task, +consider using subsections for each distinct task. +``` + +In the left nav, use `Get started` as the text. On the page itself, spell out +the full name. For example, `Get started with application security`. + +### Topics and resources + +Some pages are solely a list of links to other documentation. + +We do not encourage this page type. Lists of links can get out-of-date quickly +and offer little value to users, who prefer to search to find information. + +## Heading text guidelines + +In general, for heading text: + +- Be clear and direct. Make every word count. +- Use articles and prepositions. +- Follow [capitalization](../styleguide/index.md#capitalization) guidelines. +- Do not repeat text from earlier headings. For example, if the page is about merge requests, + instead of `Troubleshooting merge requests`, use only `Troubleshooting`. + +See also [guidelines for headings in Markdown](../styleguide/index.md#headings-in-markdown). diff --git a/doc/development/documentation/topic_types/reference.md b/doc/development/documentation/topic_types/reference.md new file mode 100644 index 00000000000..42f4f5f6f94 --- /dev/null +++ b/doc/development/documentation/topic_types/reference.md @@ -0,0 +1,32 @@ +--- +stage: none +group: Style Guide +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Reference topic type + +Reference information should be in an easily-scannable format, +like a table or list. It's similar to a dictionary or encyclopedia entry. + +```markdown +# Title (a noun, like "Pipeline settings" or "Administrator options") + +Introductory sentence. + +| Setting | Description | +|---------|-------------| +| **Name** | Descriptive sentence about the setting. | +``` + +## Reference headings + +Reference headings are usually nouns. + +Avoid these heading titles: + +- `Important notes`. Instead, incorporate this information + closer to where it belongs. For example, this information might be a prerequisite + for a task, or information about a concept. +- `Limitations`. Instead, move the content near other similar information. + If you must, you can use the title `Known issues`. diff --git a/doc/development/documentation/topic_types/task.md b/doc/development/documentation/topic_types/task.md new file mode 100644 index 00000000000..3488cb90cf9 --- /dev/null +++ b/doc/development/documentation/topic_types/task.md @@ -0,0 +1,65 @@ +--- +stage: none +group: Style Guide +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Task topic type + +A task gives instructions for how to complete a procedure. + +Tasks should be in this format: + +```markdown +# Title (starts with an active verb, like "Create a widget" or "Delete a widget") + +Do this task when you want to... + +Prerequisites (optional): + +- Thing 1 +- Thing 2 +- Thing 3 + +To do this task: + +1. Location then action. (Go to this menu, then select this item.) +1. Another step. +1. Another step. + +Task result (optional). Next steps (optional). +``` + +Here is an example. + +```markdown +# Create an issue + +Create an issue when you want to track bugs or future work. + +Prerequisites: + +- You must have at least the Developer role for the project. + +To create an issue: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Issues > List**. +1. In the top right corner, select **New issue**. +1. Complete the fields. (If you have reference content that lists each field, link to it here.) +1. Select **Create issue**. + +The issue is created. You can view it by going to **Issues > List**. +``` + +## Task headings + +For the heading text, use the structure `active verb` + `noun`. +For example, `Create an issue`. + +If you have several tasks on a page that share prerequisites, you can use the title +`Prerequisites` and link to it. + +## Related topics + +- [View the format for writing task steps](../styleguide/index.md#navigation). diff --git a/doc/development/documentation/topic_types/troubleshooting.md b/doc/development/documentation/topic_types/troubleshooting.md new file mode 100644 index 00000000000..35187bd892e --- /dev/null +++ b/doc/development/documentation/topic_types/troubleshooting.md @@ -0,0 +1,56 @@ +--- +stage: none +group: Style Guide +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Troubleshooting topic type + +Troubleshooting topics should be the last topics on a page. + +If a page has more than five troubleshooting topics, put the content on a separate page that has troubleshooting information exclusively. Name the page `Troubleshooting <feature>` +and in the left nav, use the word `Troubleshoot` only. + +Troubleshooting can be one of three types. + +## An introductory topic + +This topic introduces the troubleshooting section of a page. +For example: + +```markdown +## Troubleshooting + +When working with <x feature>, you might encounter the following issues. +``` + +## Troubleshooting task + +The title should be similar to a [standard task](task.md). +For example, "Run debug tools" or "Verify syntax." + +## Troubleshooting reference + +This topic includes the error message. For example: + +```markdown +### The error message or a description of it + +You might get an error that states <error message>. + +This issue occurs when... + +The workaround is... +``` + +If multiple causes or workarounds exist, consider putting them into a table format. +If you use the exact error message, surround it in backticks so it's styled as code. + +## Troubleshooting headings + +For the heading of a **Troubleshooting reference** topic: + +- Consider including at least a partial error message. +- Use fewer than 70 characters. + +If you do not put the full error in the title, include it in the body text. diff --git a/doc/development/documentation/versions.md b/doc/development/documentation/versions.md index 3679c731a77..85733603cfe 100644 --- a/doc/development/documentation/versions.md +++ b/doc/development/documentation/versions.md @@ -69,6 +69,11 @@ If a feature is moved to another subscription tier, use `moved`: > - [Moved](<link-to-issue>) from GitLab Premium to GitLab Free in 12.0. ``` +#### Features introduced behind feature flags + +When features are introduced behind feature flags, you must add details about the feature flag to the documentation. +For more information, see [Document features deployed behind feature flags](feature_flags.md). + ### Inline version text If you're adding content to an existing topic, you can add version information @@ -205,7 +210,7 @@ You can say that we plan to remove a feature. ### Legal disclaimer for future features -If you **must** write about features we have not yet delivered, put this exact disclaimer near the content it applies to. +If you **must** write about features we have not yet delivered, put this exact disclaimer about forward-looking statements near the content it applies to. ```markdown DISCLAIMER: @@ -227,6 +232,6 @@ As with all projects, the items mentioned on this page are subject to change or The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc. -If all of the content on the page is not available, use the disclaimer once at the top of the page. +If all of the content on the page is not available, use the disclaimer about forward-looking statements once at the top of the page. If the content in a topic is not ready, use the disclaimer in the topic. diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index 777bc77875e..869cb0bab0a 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -20,7 +20,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Implement a new EE feature -If you're developing a GitLab Starter, GitLab Premium, or GitLab Ultimate licensed feature, use these steps to +If you're developing a GitLab Premium or GitLab Ultimate licensed feature, use these steps to add your new feature or extend it. GitLab license features are added to [`ee/app/models/gitlab_subscriptions/features.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/gitlab_subscriptions/features.rb). To determine how @@ -33,9 +33,9 @@ Use the following questions to guide you: must locate the existing feature identifier to [guard it](#guard-your-ee-feature). - If this is a new feature, decide on an identifier, such as `my_feature_name`, to add to the `features.rb` file. -1. Is this a **GitLab Starter**, **GitLab Premium**, or **GitLab Ultimate** feature? - - Based on the plan you choose to use the feature in, add the feature identifier to `STARTER_FEATURES`, - `PREMIUM_FEATURES`, or `ULTIMATE_FEATURES`. +1. Is this a **GitLab Premium** or **GitLab Ultimate** feature? + - Based on the plan you choose to use the feature in, add the feature identifier to `PREMIUM_FEATURES` + or `ULTIMATE_FEATURES`. 1. Will this feature be available globally (system-wide at the GitLab instance level)? - Features such as [Geo](../administration/geo/index.md) and [Database Load Balancing](../administration/postgresql/database_load_balancing.md) are used by the entire instance @@ -281,7 +281,7 @@ There are a few gotchas with it: overriding the method, because we can't know when the overridden method (that is, calling `super` in the overriding method) would want to stop early. In this case, we shouldn't just override it, but update the original method - to make it call the other method we want to extend, like a + to make it call the other method we want to extend, like a [template method pattern](https://en.wikipedia.org/wiki/Template_method_pattern). For example, given this base: @@ -1128,7 +1128,7 @@ EE licensed features that enhance existing functionality in the UI add new elements or interactions to your Vue application as components. To separate template differences, use a child EE component to separate Vue template differences. -You must import the EE component [asynchronously](https://vuejs.org/v2/guide/components-dynamic-async.html#Async-Components). +You must import the EE component [asynchronously](https://v2.vuejs.org/v2/guide/components-dynamic-async.html#Async-Components). This allows GitLab to load the correct component in EE, while in CE GitLab loads an empty component that renders nothing. This code **must** exist in the CE repository, in addition to the EE repository. diff --git a/doc/development/elasticsearch.md b/doc/development/elasticsearch.md index 47942817790..b3996e16fa1 100644 --- a/doc/development/elasticsearch.md +++ b/doc/development/elasticsearch.md @@ -277,7 +277,7 @@ These Advanced Search migrations, like any other GitLab changes, need to support Depending on the order of deployment, it's possible that the migration has started or finished and there's still a server running the application code from before the -migration. We need to take this into consideration until we can +migration. We need to take this into consideration until we can [ensure all Advanced Search migrations start after the deployment has finished](https://gitlab.com/gitlab-org/gitlab/-/issues/321619). ### Reverting a migration @@ -317,7 +317,7 @@ safely can. We choose to use GitLab major version upgrades as a safe time to remove backwards compatibility for indices that have not been fully migrated. We -[document this in our upgrade documentation](../update/index.md#upgrading-to-a-new-major-version). +[document this in our upgrade documentation](../update/index.md#upgrading-to-a-new-major-version). We also choose to replace the migration code with the halted migration and remove tests so that: @@ -399,7 +399,7 @@ that may contain information to help diagnose performance issues. ### Performance Bar -Elasticsearch requests will be displayed in the +Elasticsearch requests will be displayed in the [`Performance Bar`](../administration/monitoring/performance/performance_bar.md), which can be used both locally in development and on any deployed GitLab instance to diagnose poor search performance. This will show the exact queries being made, @@ -495,7 +495,7 @@ theoretically be used to figure out what needs to be replayed are: These updates can be replayed by triggering another `ElasticDeleteProjectWorker`. -With the above methods and taking regular +With the above methods and taking regular [Elasticsearch snapshots](https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html) we should be able to recover from different kinds of data loss issues in a relatively short period of time compared to indexing everything from diff --git a/doc/development/emails.md b/doc/development/emails.md index 1b3c9226dd8..c997916aa21 100644 --- a/doc/development/emails.md +++ b/doc/development/emails.md @@ -160,9 +160,9 @@ and Helm Chart configuration (see [example merge request](https://gitlab.com/git #### Rationale This was done because to avoid [thread deadlocks](https://github.com/ruby/net-imap/issues/14), `MailRoom` needs -an updated version of the `net-imap` gem. However, this -[version of the net-imap cannot be installed by an unprivileged user](https://github.com/ruby/net-imap/issues/14) due to -[an error installing the digest gem](https://github.com/ruby/digest/issues/14). +an updated version of the `net-imap` gem. However, this +[version of the net-imap cannot be installed by an unprivileged user](https://github.com/ruby/net-imap/issues/14) due to +[an error installing the digest gem](https://github.com/ruby/digest/issues/14). [This bug in the Ruby interpreter](https://bugs.ruby-lang.org/issues/17761) was fixed in Ruby 3.0.2. diff --git a/doc/development/experiment_guide/index.md b/doc/development/experiment_guide/index.md index e11e516485a..500a19fe1ad 100644 --- a/doc/development/experiment_guide/index.md +++ b/doc/development/experiment_guide/index.md @@ -10,7 +10,7 @@ Experiments can be conducted by any GitLab team, most often the teams from the [Growth Sub-department](https://about.gitlab.com/handbook/engineering/development/growth/). Experiments are not tied to releases because they primarily target GitLab.com. -Experiments are run as an A/B/n test, and are behind an [experiment feature flag](../feature_flags/#experiment-type) +Experiments are run as an A/B/n test, and are behind an [experiment feature flag](../feature_flags/index.md#experiment-type) to turn the test on or off. Based on the data the experiment generates, the team decides if the experiment had a positive impact and should be made the new default, or rolled back. diff --git a/doc/development/export_csv.md b/doc/development/export_csv.md index 0f50d1438fc..29e80f676da 100644 --- a/doc/development/export_csv.md +++ b/doc/development/export_csv.md @@ -11,10 +11,10 @@ This document lists the different implementations of CSV export in GitLab codeba | Export type | How it works | Advantages | Disadvantages | Existing examples | |---|---|---|---|---| | Streaming | - Query and yield data in batches to a response stream.<br>- Download starts immediately. | - Report available immediately. | - No progress indicator.<br>- Requires a reliable connection. | [Export Audit Event Log](../administration/audit_events.md#export-to-csv) | -| Downloading | - Query and write data in batches to a temporary file.<br>- Loads the file into memory.<br>- Sends the file to the client. | - Report available immediately. | - Large amount of data might cause request timeout.<br>- Memory intensive.<br>- Request expires when user navigates to a different page. | - [Export Chain of Custody Report](../user/compliance/compliance_report/#chain-of-custody-report)<br>- [Export License Usage File](../subscriptions/self_managed/index.md#export-your-license-usage) | +| Downloading | - Query and write data in batches to a temporary file.<br>- Loads the file into memory.<br>- Sends the file to the client. | - Report available immediately. | - Large amount of data might cause request timeout.<br>- Memory intensive.<br>- Request expires when user navigates to a different page. | - [Export Chain of Custody Report](../user/compliance/compliance_report/index.md#chain-of-custody-report)<br>- [Export License Usage File](../subscriptions/self_managed/index.md#export-your-license-usage) | | As email attachment | - Asynchronously process the query with background job.<br>- Email uses the export as an attachment. | - Asynchronous processing. | - Requires users use a different app (email) to download the CSV.<br>- Email providers may limit attachment size. | - [Export issues](../user/project/issues/csv_export.md)<br>- [Export merge requests](../user/project/merge_requests/csv_export.md) | | As downloadable link in email (*) | - Asynchronously process the query with background job.<br>- Email uses an export link. | - Asynchronous processing.<br>- Bypasses email provider attachment size limit. | - Requires users use a different app (email).<br>- Requires additional storage and cleanup. | [Export User Permissions](https://gitlab.com/gitlab-org/gitlab/-/issues/1772) | -| Polling (non-persistent state) | - Asynchronously processes the query with the background job.<br>- Frontend(FE) polls every few seconds to check if CSV file is ready. | - Asynchronous processing.<br>- Automatically downloads to local machine on completion.<br>- In-app solution. | - Non-persistable request - request expires when user navigates to a different page.<br>- API is processed for each polling request. | [Export Vulnerabilities](../user/application_security/vulnerability_report/#export-vulnerability-details) | +| Polling (non-persistent state) | - Asynchronously processes the query with the background job.<br>- Frontend(FE) polls every few seconds to check if CSV file is ready. | - Asynchronous processing.<br>- Automatically downloads to local machine on completion.<br>- In-app solution. | - Non-persistable request - request expires when user navigates to a different page.<br>- API is processed for each polling request. | [Export Vulnerabilities](../user/application_security/vulnerability_report/index.md#export-vulnerability-details) | | Polling (persistent state) (*) | - Asynchronously processes the query with background job.<br>- Backend (BE) maintains the export state<br>- FE polls every few seconds to check status.<br>- FE shows 'Download link' when export is ready.<br>- User can download or regenerate a new report. | - Asynchronous processing.<br>- No database calls made during the polling requests (HTTP 304 status is returned until export status changes).<br>- Does not require user to stay on page until export is complete.<br>- In-app solution.<br>- Can be expanded into a generic CSV feature (such as dashboard / CSV API). | - Requires to maintain export states in DB.<br>- Does not automatically download the CSV export to local machine, requires users to select 'Download'. | [Export Merge Commits Report](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/43055) | NOTE: diff --git a/doc/development/fe_guide/graphql.md b/doc/development/fe_guide/graphql.md index 442dda20d23..6dcc57b0ff5 100644 --- a/doc/development/fe_guide/graphql.md +++ b/doc/development/fe_guide/graphql.md @@ -729,8 +729,8 @@ In this case, we can either: - Skip passing a cursor. - Pass `null` explicitly to `after`. -After data is fetched, we can use the `update`-hook as an opportunity -[to customize the data that is set in the Vue component property](https://apollo.vuejs.org/api/smart-query.html#options). +After data is fetched, we can use the `update`-hook as an opportunity +[to customize the data that is set in the Vue component property](https://apollo.vuejs.org/api/smart-query.html#options). This allows us to get a hold of the `pageInfo` object among other data. In the `result`-hook, we can inspect the `pageInfo` object to see if we need to fetch diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 02086ec5f1b..d90c270153e 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -36,7 +36,8 @@ Sign in to BrowserStack with the credentials saved in the **Engineering** vault ## Initiatives -Current high-level frontend goals are listed on [Frontend Epics](https://gitlab.com/groups/gitlab-org/-/epics?label_name%5B%5D=frontend). +You can find current frontend initiatives with a cross-functional impact on epics +with the label [frontend-initiative](https://gitlab.com/groups/gitlab-org/-/epics?state=opened&page=1&sort=UPDATED_AT_DESC&label_name[]=frontend-initiative). ## Principles diff --git a/doc/development/fe_guide/storybook.md b/doc/development/fe_guide/storybook.md index 45342eb6d72..a3a1fa2160f 100644 --- a/doc/development/fe_guide/storybook.md +++ b/doc/development/fe_guide/storybook.md @@ -46,10 +46,12 @@ To add a story: 1. Write the story as per the [official Storybook instructions](https://storybook.js.org/docs/vue/writing-stories/introduction/) - Notes: - - Specify the `title` field of the story as the component's file path from the `javascripts/` directory. - - For example, if the component is located at `app/assets/javascripts/vue_shared/components/sidebar/todo_toggle/todo_button.vue`, specify the story `title` as `vue_shared/components/sidebar/todo_toggle/todo_button`. This will ensure the Storybook navigation maps closely to our internal directory structure. + NOTE: + Specify the `title` field of the story as the component's file path from the `javascripts/` directory, without the `/components` part. + For example, if the component is located at `app/assets/javascripts/vue_shared/components/sidebar/todo_toggle/todo_button.vue`, + specify the story `title` as `vue_shared/sidebar/todo_toggle/todo_button`. + If the component is located in the `ee/` directory, make sure to prefix the story's title with `ee/` as well. + This will ensure the Storybook navigation maps closely to our internal directory structure. ## Mock backend APIs diff --git a/doc/development/fe_guide/style/vue.md b/doc/development/fe_guide/style/vue.md index c9bd0e1b35a..39dc9cc9c4e 100644 --- a/doc/development/fe_guide/style/vue.md +++ b/doc/development/fe_guide/style/vue.md @@ -121,7 +121,7 @@ Check the [rules](https://github.com/vuejs/eslint-plugin-vue#bulb-rules) for mor 1. **Extensions**: Use `.vue` extension for Vue components. Do not use `.js` as file extension ([#34371](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/34371)). -1. **Reference Naming**: Use PascalCase for their instances: +1. **Reference Naming**: Use PascalCase for their default imports: ```javascript // bad @@ -431,7 +431,7 @@ must be unique. It's advised to use `kebab-case` namespaces. Useful links: -1. [`key`](https://vuejs.org/v2/guide/list.html#key) +1. [Maintaining State](https://v2.vuejs.org/v2/guide/list.html#Maintaining-State) 1. [Vue Style Guide: Keyed v-for](https://vuejs.org/v2/style-guide/#Keyed-v-for-essential) ## Vue testing @@ -448,10 +448,8 @@ Typically, when testing a Vue component, the component should be "re-mounted" in To achieve this: 1. Create a mutable `wrapper` variable inside the top-level `describe` block. -1. Mount the component using [`mount`](https://v1.test-utils.vuejs.org/api/#mount)/ -[`shallowMount`](https://v1.test-utils.vuejs.org/api/#shallowMount). -1. Reassign the resulting [`Wrapper`](https://v1.test-utils.vuejs.org/api/wrapper/#wrapper) -instance to our `wrapper` variable. +1. Mount the component using [`mount`](https://v1.test-utils.vuejs.org/api/#mount) or [`shallowMount`](https://v1.test-utils.vuejs.org/api/#shallowMount). +1. Reassign the resulting [`Wrapper`](https://v1.test-utils.vuejs.org/api/wrapper/#wrapper) instance to our `wrapper` variable. Creating a global, mutable wrapper provides a number of advantages, including the ability to: @@ -671,6 +669,6 @@ In the coming months you should fix that tech debt, with its priority to be dete not be rewritten. For example, jQuery tests rewritten to Vue tests. 1. You may choose to use VueX as a centralized state management. If you choose not to use VueX, you must use the *store pattern* which can be found in the -[Vue.js documentation](https://vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch). +[Vue.js documentation](https://v2.vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch). 1. Once you have chosen a centralized state-management solution you must use it for your entire application. Don't mix and match your state-management solutions. diff --git a/doc/development/fe_guide/tooling.md b/doc/development/fe_guide/tooling.md index 2bb6cbfaf7a..4b55580c33c 100644 --- a/doc/development/fe_guide/tooling.md +++ b/doc/development/fe_guide/tooling.md @@ -63,8 +63,8 @@ disabled due to legacy compatibility reasons but they are in the process of bein Do not disable specific ESLint rules. To avoid introducing technical debt, you may disable the following rules only if you are invoking/instantiating existing code modules. -- [`no-new`](https://eslint.org/docs/rules/no-new) -- [`class-method-use-this`](https://eslint.org/docs/rules/class-methods-use-this) +- [`no-new`](https://eslint.org/docs/latest/rules/no-new) +- [`class-method-use-this`](https://eslint.org/docs/latest/rules/class-methods-use-this) Disable these rules on a per-line basis. This makes it easier to refactor in the future. For example, use `eslint-disable-next-line` or `eslint-disable-line`. diff --git a/doc/development/fe_guide/troubleshooting.md b/doc/development/fe_guide/troubleshooting.md index c0894621ed1..ab10c5bf988 100644 --- a/doc/development/fe_guide/troubleshooting.md +++ b/doc/development/fe_guide/troubleshooting.md @@ -4,7 +4,7 @@ group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Troubleshooting +# Troubleshooting frontend development issues Running into a problem? Maybe this will help ¯\＿(ツ)＿/¯. diff --git a/doc/development/fe_guide/view_component.md b/doc/development/fe_guide/view_component.md index 2e373e6933b..7ddce609ee7 100644 --- a/doc/development/fe_guide/view_component.md +++ b/doc/development/fe_guide/view_component.md @@ -33,7 +33,7 @@ Consider this list a best effort. The full list can be found in [`app/components #### Alert -The `Pajamas::AlertComponent` follows the [Pajamas Alert](https://design.gitlab.com/components/alert) specification. +The `Pajamas::AlertComponent` follows the [Pajamas Alert](https://design.gitlab.com/components/alert/) specification. **Examples:** @@ -57,7 +57,7 @@ For the full list of options, see its #### Banner -The `Pajamas::BannerComponent` follows the [Pajamas Banner](https://design.gitlab.com/components/banner) specification. +The `Pajamas::BannerComponent` follows the [Pajamas Banner](https://design.gitlab.com/components/banner/) specification. **Examples:** @@ -88,7 +88,7 @@ For the full list of options, see its #### Button -The `Pajamas::ButtonComponent` follows the [Pajamas Button](https://design.gitlab.com/components/button) specification. +The `Pajamas::ButtonComponent` follows the [Pajamas Button](https://design.gitlab.com/components/button/) specification. **Examples:** @@ -125,7 +125,7 @@ For the full list of options, see its #### Card -The `Pajamas::CardComponent` follows the [Pajamas Card](https://design.gitlab.com/components/card) specification. +The `Pajamas::CardComponent` follows the [Pajamas Card](https://design.gitlab.com/components/card/) specification. **Examples:** @@ -188,7 +188,7 @@ For the full list of options, see its #### Toggle -The `Pajamas::ToggleComponent` follows the [Pajamas Toggle](https://design.gitlab.com/components/toggle) specification. +The `Pajamas::ToggleComponent` follows the [Pajamas Toggle](https://design.gitlab.com/components/toggle/) specification. ```haml = render Pajamas::ToggleComponent.new(classes: 'js-force-push-toggle', @@ -205,7 +205,34 @@ To actually initialize this component, make sure to call the `initToggle` helper For the full list of options, see its [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/components/pajamas/toggle_component.rb). -### Best practices +## Layouts + +Layout components can be used to create common layout patterns used in GitLab. + +### Available components + +#### Horizontal section + +Many of the settings pages use a layout where the title and description are on the left and the settings fields are on the right. The `Layouts::HorizontalSectionComponent` can be used to create this layout. + +**Example:** + +```haml += render ::Layouts::HorizontalSectionComponent.new(options: { class: 'gl-mb-6' }) do |c| + = c.title { _('Naming, visibility') } + = c.description do + = _('Update your group name, description, avatar, and visibility.') + = link_to _('Learn more about groups.'), help_page_path('user/group/index') + = c.body do + .form-group.gl-form-group + = f.label :name, _('New group name') + = f.text_field :name +``` + +For the full list of options, see its +[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/components/layouts/horizontal_section_component.rb). + +## Best practices - If you are about to create a new view in Haml, use the available components over creating plain Haml tags with CSS classes. diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index 27660c0f5f7..5cb461c8ca0 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Vue -To get started with Vue, read through [their documentation](https://vuejs.org/v2/guide/). +To get started with Vue, read through [their documentation](https://v2.vuejs.org/v2/guide/index.html). ## Examples @@ -23,8 +23,8 @@ The main goal we are trying to achieve is to have only one data flow, and only o To achieve this goal we use [Vuex](#vuex). You can also read about this architecture in Vue documentation about -[state management](https://vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch) -and about [one way data flow](https://vuejs.org/v2/guide/components.html#One-Way-Data-Flow). +[state management](https://v2.vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch) +and about [one way data flow](https://v2.vuejs.org/v2/guide/components-props.html#One-Way-Data-Flow). ### Components and Store @@ -322,7 +322,7 @@ For example, tables are used in a quite amount of places across GitLab, a table would be a good fit for a component. On the other hand, a table cell used only in one table would not be a good use of this pattern. -You can read more about components in Vue.js site, [Component System](https://vuejs.org/v2/guide/#Composing-with-Components). +You can read more about components in Vue.js site, [Component System](https://v2.vuejs.org/v2/guide/#Composing-with-Components). ### A folder for the Store @@ -348,8 +348,7 @@ recommended to observe objects with their own stateful behavior. Based on the Vue guidance: -- **Do not** use or create a JavaScript class in your [data function](https://v2.vuejs.org/v2/api/#data), -such as `user: new User()`. +- **Do not** use or create a JavaScript class in your [data function](https://v2.vuejs.org/v2/api/#data). - **Do not** add new JavaScript class implementations. - **Do** use [GraphQL](../api_graphql_styleguide.md), [Vuex](vuex.md) or a set of components if cannot use primitives or objects. @@ -531,8 +530,7 @@ Each Vue component has a unique output. This output is always present in the ren Although each method of a Vue component can be tested individually, our goal is to test the output of the render function, which represents the state at all times. -Visit the [Vue testing guide](https://v2.vuejs.org/v2/guide/testing.html#Unit-Testing) for help -testing the rendered output. +Visit the [Vue testing guide](https://v2.vuejs.org/v2/guide/testing.html#Unit-Testing) for help. Here's an example of a well structured unit test for [this Vue component](#appendix---vue-component-subject-under-test): @@ -671,7 +669,7 @@ it('should fire the click event', () => { }) ``` -When firing a Vue event, use [`emit`](https://vuejs.org/v2/guide/components-custom-events.html). +When firing a Vue event, use [`emit`](https://v2.vuejs.org/v2/guide/components-custom-events.html). ```javascript wrapper = shallowMount(DropdownItem); diff --git a/doc/development/fe_guide/vuex.md b/doc/development/fe_guide/vuex.md index 8bfb912161a..2d1569b7812 100644 --- a/doc/development/fe_guide/vuex.md +++ b/doc/development/fe_guide/vuex.md @@ -245,7 +245,7 @@ A mutation written like this is easier to maintain. In addition, we avoid errors ### `getters.js` Sometimes we may need to get derived state based on store state, like filtering for a specific prop. -Using a getter also caches the result based on dependencies due to [how computed props work](https://vuejs.org/v2/guide/computed.html#Computed-Caching-vs-Methods) +Using a getter also caches the result based on dependencies due to [how computed props work](https://v2.vuejs.org/v2/guide/computed.html#Computed-Caching-vs-Methods) This can be done through the `getters`: ```javascript @@ -364,7 +364,7 @@ export default initialState => ({ We made the conscious decision to avoid this pattern to improve the ability to discover and search our frontend codebase. The same applies -when [providing data to a Vue app](vue.md#providing-data-from-haml-to-javascript). The reasoning for this is described in +when [providing data to a Vue app](vue.md#providing-data-from-haml-to-javascript). The reasoning for this is described in [this discussion](https://gitlab.com/gitlab-org/frontend/rfcs/-/issues/56#note_302514865): > Consider a `someStateKey` is being used in the store state. You _may_ not be diff --git a/doc/development/fe_guide/widgets.md b/doc/development/fe_guide/widgets.md index b54f9add97d..c6bb89d1fe8 100644 --- a/doc/development/fe_guide/widgets.md +++ b/doc/development/fe_guide/widgets.md @@ -18,11 +18,11 @@ When building a widget, we should follow a few principles described below. All widgets should use the same stack (Vue + Apollo Client). To make it happen, we must add Vue Apollo to the application root (if we use a widget as a component) or provide it directly to a widget. For sidebar widgets, use the -[sidebar Apollo Client and Apollo Provider](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/sidebar/graphql.js): +[issuable Apollo Client and Apollo Provider](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/graphql_shared/issuable_client.js): ```javascript import SidebarConfidentialityWidget from '~/sidebar/components/confidential/sidebar_confidentiality_widget.vue'; -import { apolloProvider } from '~/sidebar/graphql'; +import { apolloProvider } from '~/graphql_shared/issuable_client'; function mountConfidentialComponent() { new Vue({ @@ -118,7 +118,7 @@ In this case, we can use a renderless component that imports a client and listen ```javascript import { fetchPolicies } from '~/lib/graphql'; import { confidentialityQueries } from '~/sidebar/constants'; -import { defaultClient as gqlClient } from '~/sidebar/graphql'; +import { defaultClient as gqlClient } from '~/graphql_shared/issuable_client'; created() { if (this.issuableType !== IssuableType.Issue) { diff --git a/doc/development/feature_development.md b/doc/development/feature_development.md index e50c1edd282..fd1c7f4afa5 100644 --- a/doc/development/feature_development.md +++ b/doc/development/feature_development.md @@ -174,6 +174,7 @@ See [database guidelines](database/index.md). ## Domain-specific guides - [CI/CD development documentation](cicd/index.md) +- [Sec Section development documentation](sec/index.md) ## Technical Reference by Group diff --git a/doc/development/feature_flags/controls.md b/doc/development/feature_flags/controls.md index 8a862e5f7cd..63dad3070c7 100644 --- a/doc/development/feature_flags/controls.md +++ b/doc/development/feature_flags/controls.md @@ -112,6 +112,8 @@ incidents or in-progress change issues, for example: 2021-06-29 Canary deployment failing QA tests ``` +Before enabling a feature flag, verify that you are not violating any [Production Change Lock periods](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/#production-change-lock-pcl) and are in compliance with the [Feature Flags and the Change Management Process](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/#feature-flags-and-the-change-management-process). + The following `/chatops` commands should be performed in the Slack `#production` channel. @@ -337,7 +339,7 @@ take one of the following actions: To remove a feature flag, open **one merge request** to make the changes. In the MR: -1. Add the ~"feature flag" label so release managers are aware the changes are hidden behind a feature flag. +1. Add the ~"feature flag" label so release managers are aware of the removal. 1. If the merge request has to be picked into a stable branch, add the appropriate `~"Pick into X.Y"` label, for example `~"Pick into 13.0"`. See [the feature flag process](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/#including-a-feature-behind-feature-flag-in-the-final-release) diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md index 502a028f089..444b53f9c8d 100644 --- a/doc/development/feature_flags/index.md +++ b/doc/development/feature_flags/index.md @@ -433,8 +433,8 @@ When using the percentage rollout of actors on multiple feature flags, the actor For example, the following feature flags are enabled for a certain percentage of actors: ```plaintext -/chatops run chatops feature set feature-set-1 25 --actors -/chatops run chatops feature set feature-set-2 25 --actors +/chatops run feature set feature-set-1 25 --actors +/chatops run feature set feature-set-2 25 --actors ``` If a project A has `:feature-set-1` enabled, there is no guarantee that project A also has `:feature-set-2` enabled. @@ -514,12 +514,12 @@ You can also enable a feature flag for a given gate: Feature.enable(:feature_flag_name, Project.find_by_full_path("root/my-project")) ``` -### Removing a feature flag locally (in development) +### Disabling a feature flag locally (in development) When manually enabling or disabling a feature flag from the Rails console, its default value gets overwritten. This can cause confusion when changing the flag's `default_enabled` attribute. -To reset the feature flag to the default status, you can remove it in the rails console (`rails c`) +To reset the feature flag to the default status, you can disable it in the rails console (`rails c`) as follows: ```ruby @@ -535,16 +535,18 @@ Feature.remove(:feature_flag_name) ```mermaid graph LR - A[flag: default off] -->|'added' / 'changed'| B(flag: default on) + A[flag: default off] -->|'added' / 'changed' / 'fixed' / '...'| B(flag: default on) B -->|'other'| C(remove flag, keep new code) B -->|'removed' / 'changed'| D(remove flag, keep old code) - A -->|'added' / 'changed'| C + A -->|'added' / 'changed' / 'fixed' / '...'| C A -->|no changelog| D ``` - Any change behind a feature flag that is **enabled** by default **should** have a changelog entry. - The changelog for a feature flag should describe the feature and not the flag, unless a default on feature flag is removed keeping the new code (`other` in the flowchart above). +- A feature flag can also be used for rolling out a bug fix or a maintenance work. In this scenario, the changelog + must be related to it, for example; `fixed` or `other`. ## Feature flags in tests diff --git a/doc/development/fips_compliance.md b/doc/development/fips_compliance.md index c690408ee60..1029ed88eac 100644 --- a/doc/development/fips_compliance.md +++ b/doc/development/fips_compliance.md @@ -65,7 +65,7 @@ listed here that also do not work properly in FIPS mode: - [Solutions for vulnerabilities](../user/application_security/vulnerabilities/index.md#resolve-a-vulnerability) for yarn projects. - [Static Application Security Testing (SAST)](../user/application_security/sast/index.md) - supports a reduced set of [analyzers](../user/application_security/sast/#fips-enabled-images) + supports a reduced set of [analyzers](../user/application_security/sast/index.md#fips-enabled-images) when operating in FIPS-compliant mode. - Advanced Search is currently not included in FIPS mode. It must not be enabled in order to be FIPS-compliant. - [Gravatar or Libravatar-based profile images](../administration/libravatar.md) are not FIPS-compliant. @@ -578,6 +578,6 @@ Merge requests that can trigger Package and QA, can trigger a FIPS package and a Reference Architecture test pipeline. The base image used for the trigger is Ubuntu 20.04 FIPS: -1. Trigger `package-and-qa`, if not already triggered. +1. Trigger `e2e:package-and-test` job, if not already triggered. 1. On the `gitlab-omnibus-mirror` child pipeline, manually trigger `Trigger:package:fips`. 1. When the package job is complete, manually trigger the `RAT:FIPS` job. diff --git a/doc/development/gemfile.md b/doc/development/gemfile.md index f9cf69020bb..87304a761ea 100644 --- a/doc/development/gemfile.md +++ b/doc/development/gemfile.md @@ -4,7 +4,7 @@ group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# `Gemfile` guidelines +# Gemfile guidelines When adding a new entry to `Gemfile` or upgrading an existing dependency pay attention to the following rules. @@ -15,6 +15,23 @@ We do not allow gems that are fetched from Git repositories. All gems have to be available in the RubyGems index. We want to minimize external build dependencies and build times. +## Request an Appsec review + +When adding a new gem to our `Gemfile` or even changing versions in +`Gemfile.lock` it is strongly recommended that you +[request a Security review](https://about.gitlab.com/handbook/engineering/security/#how-to-request-a-security-review). +New gems add an extra security risk for GitLab, and it is important to +evaluate this risk before we ship this to production. Technically, just adding +a new gem and pushing to a branch in our main `gitlab` project is a security +risk as it will run in CI using your GitLab.com credentials. As such you should +evaluate early on if you think this gem seems legitimate before you even +install it. + +Reviewers should also be aware of our related +[recommendations for reviewing community contributions](code_review.md#community-contributions) +and take care before running a pipeline for community contributions that +contains changes to `Gemfile` or `Gemfile.lock`. + ## License compliance Refer to [licensing guidelines](licensing.md) for ensuring license compliance. @@ -73,7 +90,7 @@ to a gem, go through these steps: apply if someone who currently works at GitLab wants to maintain the gem beyond their time working at GitLab. -When publishing a gem to RubyGems.org, also note the section on +When publishing a gem to RubyGems.org, also note the section on [gem owners](https://about.gitlab.com/handbook/developer-onboarding/#ruby-gems) in the handbook. @@ -132,7 +149,7 @@ that also relied on `thor` but had its version pinned to a vulnerable one. These changes are easy to miss in the `Gemfile.lock`. Pinning the version would result in a conflict that would need to be solved. -To avoid upgrading indirect dependencies, we can use +To avoid upgrading indirect dependencies, we can use [`bundle update --conservative`](https://bundler.io/man/bundle-update.1.html#OPTIONS). When submitting a merge request including a dependency update, diff --git a/doc/development/geo.md b/doc/development/geo.md index f042af42de5..1ae9d9ee32b 100644 --- a/doc/development/geo.md +++ b/doc/development/geo.md @@ -653,8 +653,7 @@ on, check out our [self-service framework](geo/framework.md). ### GET:Geo pipeline -As part of the [package-and-qa](testing_guide/end_to_end/index.md#using-the-package-and-qa-job) pipeline, there is an option to manually trigger a job named `GET:Geo`. This -pipeline uses [GET](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) to spin up a +As part of the [e2e:package-and-test](testing_guide/end_to_end/index.md#using-the-package-and-test-job) pipeline, there is an option to manually trigger a job named `GET:Geo`. This pipeline uses [GET](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) to spin up a [1k](../administration/reference_architectures/1k_users.md) Geo installation, and run the [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa) Geo scenario against the instance. When working on Geo features, it is a good idea to ensure the `qa-geo` job passes in a triggered `GET:Geo pipeline`. @@ -669,7 +668,7 @@ see the [QA documentation](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa The pipeline involves the interaction of multiple different projects: -- [GitLab](https://gitlab.com/gitlab-org/gitlab) - The [package-and-qa job](testing_guide/end_to_end/index.md#using-the-package-and-qa-job) is launched from merge requests in this project. +- [GitLab](https://gitlab.com/gitlab-org/gitlab) - The [`e2e:package-and-test` job](testing_guide/end_to_end/index.md#using-the-package-and-test-job) is launched from merge requests in this project. - [`omnibus-gitlab`](https://gitlab.com/gitlab-org/omnibus-gitlab) - Builds relevant artifacts containing the changes from the triggering merge request pipeline. - [GET-Configs/Geo](https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit-configs/Geo) - Coordinates the lifecycle of a short-lived Geo installation that can be evaluated. - [GET](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) - Contains the necessary logic for creating and destroying Geo installations. Used by `GET-Configs/Geo`. diff --git a/doc/development/geo/proxying.md b/doc/development/geo/proxying.md index 2f0226c489c..d4cb611e965 100644 --- a/doc/development/geo/proxying.md +++ b/doc/development/geo/proxying.md @@ -128,7 +128,7 @@ Secondary-->>Client: admin/geo/replication/projects logged in response (session ## Git pull -For historical reasons, the `push_from_secondary` path is used to forward a Git pull. There is +For historical reasons, the `push_from_secondary` path is used to forward a Git pull. There is [an issue proposing to rename this route](https://gitlab.com/gitlab-org/gitlab/-/issues/292690) to avoid confusion. ### Git pull over HTTP(s) diff --git a/doc/development/git_object_deduplication.md b/doc/development/git_object_deduplication.md index a6b359769f8..a20bdf633cd 100644 --- a/doc/development/git_object_deduplication.md +++ b/doc/development/git_object_deduplication.md @@ -18,7 +18,7 @@ GitLab implements Git object deduplication. ### Understanding Git alternates -At the Git level, we achieve deduplication by using +At the Git level, we achieve deduplication by using [Git alternates](https://git-scm.com/docs/gitrepository-layout#gitrepository-layout-objects). Git alternates is a mechanism that lets a repository borrow objects from another repository on the same machine. @@ -99,7 +99,7 @@ are as follows: ### Assumptions -- All repositories in a pool must use [hashed storage](../administration/repository_storage_types.md). +- All repositories in a pool must use [hashed storage](../administration/repository_storage_types.md). This is so that we don't have to ever worry about updating paths in `object/info/alternates` files. - All repositories in a pool must be on the same Gitaly storage shard. diff --git a/doc/development/github_importer.md b/doc/development/github_importer.md index 0aa1bad711d..9740ae04553 100644 --- a/doc/development/github_importer.md +++ b/doc/development/github_importer.md @@ -71,7 +71,7 @@ This worker imports all pull requests. For every pull request a job for the ### 5. Stage::ImportPullRequestsMergedByWorker -This worker imports the pull requests' _merged-by_ user information. The +This worker imports the pull requests' _merged-by_ user information. The [_List pull requests_](https://docs.github.com/en/rest/pulls#list-pull-requests) API doesn't provide this information. Therefore, this stage must fetch each merged pull request individually to import this information. A @@ -101,7 +101,20 @@ label links in the same worker removes the need for performing a separate crawl through the API data, reducing the number of API calls necessary to import a project. -### 8. Stage::ImportNotesWorker +### 8. Stage::ImportIssueEventsWorker + +This worker imports all issues and pull request events. For every event, we +schedule a job for the `Gitlab::GithubImport::ImportIssueEventWorker` worker. + +We can import both issues and pull request events by single stage because of a specific aspect of the GitHub API. It looks like that under the hood, issues and pull requests +GitHub are stored in a single table. Therefore, they have globally-unique IDs and so: + +- Every pull request is an issue. +- Issues aren't pull requests. + +Therefore, both issues and pull requests have a common API for most related things. + +### 9. Stage::ImportNotesWorker This worker imports regular comments for both issues and pull requests. For every comment, we schedule a job for the @@ -112,7 +125,28 @@ returns comments for both issues and pull requests. This means we have to wait for all issues and pull requests to be imported before we can import regular comments. -### 9. Stage::FinishImportWorker +### 10. Stage::ImportAttachmentsWorker + +This worker imports release notes attachments that are linked inside Markdown. +For every release of the project, we schedule a job of +`Gitlab::GithubImport::ImportReleaseAttachmentsWorker` for every comment. + +Each job: + +1. Iterates over all attachment links inside of a specific release note. +1. Downloads the attachment. +1. Replaces the old link with a newly-generated link to GitLab. + +### 11. Stage::ImportProtectedBranchesWorker + +This worker imports protected branch rules. +For every rule that exists on GitHub, we schedule a job of +`Gitlab::GithubImport::ImportProtectedBranchWorker`. + +Each job compares the branch protection rules from GitHub and GitLab and applies +the strictest of the rules to the branches in GitLab. + +### 12. Stage::FinishImportWorker This worker completes the import process by performing some housekeeping (such as flushing any caches) and by marking the import as completed. diff --git a/doc/development/gitlab_flavored_markdown/specification_guide/index.md b/doc/development/gitlab_flavored_markdown/specification_guide/index.md index 756b87cd407..c1227e5d33f 100644 --- a/doc/development/gitlab_flavored_markdown/specification_guide/index.md +++ b/doc/development/gitlab_flavored_markdown/specification_guide/index.md @@ -345,8 +345,50 @@ For the [Markdown snapshot testing](#markdown-snapshot-testing) to work properly, you must account for these differences in a way that ensures the tests are reliable, and always behave the same across different test runs or environments. -To account for these differences, there is a process called **_normalization_**. Normalization -allows custom regular expressions with +To account for these differences, there is a process called **_normalization_**. Several ways to approach normalization exist: + +1. Fixture-based normalization +1. Environment-variable-based normalization +1. Regex-based normalization + +#### Fixture-based normalization + +Fixture-based normalization should be used whenever possible, because it is simpler and easier to +understand than regex-based normalization. + +The [Markdown snapshot testing](#markdown-snapshot-testing) uses RSpec to generate the +[example snapshot files](#example-snapshot-files). RSpec enables you to: + +- Use the same powerful fixture support and helpers as all the rest of the GitLab RSpec suite. +- Use fixtures to control the state of the database when the example snapshots are generated. +- Extract this fixture setup to an RSpec shared context. This shared context is used to ensure + the same database state exists wherever the snapshot tests are run, either by the CI suite, or + locally via [`run-snapshot-tests.sh`](#run-snapshot-testssh-script). + +You can see the RSpec shared context containing these fixtures in +[`spec/support/shared_contexts/glfm/example_snapshot_fixtures.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/support/shared_contexts/glfm/example_snapshot_fixtures.rb). + +#### Environment-variable-based normalization + +In some cases, fixtures may not be usable, because they do not provide control over the varying +values. In these cases, we can introduce support for a environment variable into the production +code, which allows us to override the randommness in our test environment when we are +generating the HTML for footnote examples. Even though it is in the production code path, it has +no effect unless it is explicitly set, therefore it is innocuous. It allows us to avoid +the more-complex regex-based normalization described below. + +The current example of this is when normally random footnote IDs are overridden to be deterministic +by setting `GITLAB_TEST_FOOTNOTE_ID`. It is set along with the fixtures setup in the +[`spec/support/shared_contexts/glfm/example_snapshot_fixtures.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/support/shared_contexts/glfm/example_snapshot_fixtures.rb) +shared context. + +#### Regex-based normalization + +If neither fixture-based nor environment-variable-based normalization can be used, use regex-based +normalization. It is powerful, but more complex, and requires more maintenance. +It requires referring to specific examples by name, and crafting the proper regexes. + +Regex-based normalization allows custom regular expressions with [_capturing groups_](https://ruby-doc.org/core-3.1.2/Regexp.html#class-Regexp-label-Capturing) to be applied to two different versions of HTML or JSON for a given Markdown example, and the contents of the captured groups can be replaced with the same fixed values. @@ -653,10 +695,16 @@ is the manually updated canonical Markdown+HTML examples for GLFM extensions. - It contains examples in the [standard backtick-delimited `spec.txt` format](#various-markdown-specifications), each of which contain a Markdown example and the corresponding canonical HTML. +- For all GitLab examples, the "extension" annotation after the backticks should consist of only + `example gitlab`. It does not currently include any additional extension annotations describing + the specific Markdown, unlike the GitHub Flavored Markdown examples, which do include + these additional annotations (such as `example strikethrough`). - The `update-specification.rb` script inserts it as new sections before the appendix of generated `spec.txt`. -- It should consist of `H1` header sections, with all examples nested exactly 2 levels deep within `H2` - header sections. +- It should consist of `H1` header sections, with all examples nested either 2 or 3 levels deep + within `H2` or `H3` header sections. +- `H3` header sections must be nested within `H2` header sections. They cannot be + nested directly within `H1` header sections. `glfm_specification/input/gitlab_flavored_markdown/glfm_canonical_examples.txt` sample entries: @@ -670,7 +718,7 @@ The actual file should not have these prefixed `|` characters. | |## Strong but with two asterisks | -|```````````````````````````````` example +|```````````````````````````````` example gitlab |**bold** |. |<p><strong>bold</strong></p> @@ -680,7 +728,7 @@ The actual file should not have these prefixed `|` characters. | |## Strong but with HTML | -|```````````````````````````````` example +|```````````````````````````````` example gitlab |<strong> |bold |</strong> @@ -738,7 +786,7 @@ The following optional entries are supported for each example. They all default `glfm_specification/input/gitlab_flavored_markdown/glfm_example_status.yml` sample entry: ```yaml -07_99_an_example_with_incomplete_wysiwyg_implementation_1: +07_99_00_an_example_with_incomplete_wysiwyg_implementation_1: skip_update_example_snapshots: 'An explanation of the reason for skipping.' skip_update_example_snapshot_html_static: 'An explanation of the reason for skipping.' skip_update_example_snapshot_html_wysiwyg: 'An explanation of the reason for skipping.' @@ -771,40 +819,73 @@ to be specified for a Markdown example. 00_uri: &00_uri - regex: '(href|data-src)(=")(.*?)(test-file\.(png|zip)")' replacement: '\1\2URI_PREFIX\4' -01_01__section_one__example_containing_a_uri__001: +01_01_00__section_one__example_containing_a_uri__001: html: static: canonical: - 01_01_uri: *00_uri + 01_01_00_uri: *00_uri snapshot: - 01_01_uri: *00_uri + 01_01_00_uri: *00_uri wysiwyg: - 01_01_uri: *00_uri + 01_01_00_uri: *00_uri prosemirror_json: - 01_01_uri: *00_uri -07_01__gitlab_specific_markdown__footnotes__001: + 01_01_00_uri: *00_uri +07_01_00__gitlab_specific_markdown__footnotes__001: # YAML anchors which are only shared within a single example should be defined within the example shared: - 07_01_href: &07_01_href + 07_01_00_href: &07_01_00_href - regex: '(href)(=")(.+?)(")' replacement: '\1\2REF\4' - 07_01_id: &07_01_id + 07_01_00_id: &07_01_00_id - regex: '(id)(=")(.+?)(")' replacement: '\1\2ID\4' html: static: canonical: - 07_01_href: *07_01_href - 07_01_id: *07_01_id + 07_01_00_href: *07_01_00_href + 07_01_00_id: *07_01_00_id snapshot: - 07_01_href: *07_01_href - 07_01_id: *07_01_id + 07_01_00_href: *07_01_00_href + 07_01_00_id: *07_01_00_id wysiwyg: - 07_01_href: *07_01_href - 07_01_id: *07_01_id + 07_01_00_href: *07_01_00_href + 07_01_00_id: *07_01_00_id prosemirror_json: - 07_01_href: *07_01_href - 07_01_id: *07_01_id + 07_01_00_href: *07_01_00_href + 07_01_00_id: *07_01_00_id +``` + +##### `glfm_example_metadata.yml` + +[`glfm_specification/input/gitlab_flavored_markdown/glfm_example_metadata.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/input/gitlab_flavored_markdown/glfm_example_metadata.yml) +allows control over other aspects of the snapshot example generation process. + +- It is manually updated. +- The `ee` fields determine whether the example is an EE-only example. If the `ee` field is `true`, + the example will only be run by `ee/spec/requests/api/markdown_snapshot_spec.rb`, not by + `spec/requests/api/markdown_snapshot_spec.rb`. +- The `api_request_override_path` field overrides the API endpoint path which is used to + generate the `static` HTML for the specifed example. Different endpoints can generate different + HTML in some cases, so we want to be able to exercise different API endpoints for the same + Markdown. By default, the `/markdown` endpoint is used. + +`glfm_specification/input/gitlab_flavored_markdown/glfm_example_metadata.yml` sample entries: + +```yaml +--- +08_01_00__examples_using_internal_extensions__markdown_preview_api_request_overrides__001: + api_request_override_path: /groups/glfm_group/preview_markdown +08_01_00__examples_using_internal_extensions__markdown_preview_api_request_overrides__002: + api_request_override_path: /glfm_group/glfm_project/preview_markdown +08_01_00__examples_using_internal_extensions__markdown_preview_api_request_overrides__003: + api_request_override_path: /glfm_group/glfm_project/preview_markdown +08_01_00__examples_using_internal_extensions__markdown_preview_api_request_overrides__004: + api_request_override_path: /-/snippets/preview_markdown +08_01_00__examples_using_internal_extensions__markdown_preview_api_request_overrides__005: + api_request_override_path: /glfm_group/glfm_project/-/wikis/new_page/preview_markdown +08_01_00__examples_using_internal_extensions__markdown_preview_api_request_overrides__006: + ee: true + api_request_override_path: /groups/glfm_group/-/wikis/new_page/preview_markdown ``` #### Output specification files @@ -891,19 +972,19 @@ CommonMark, GFM, and GLFM example names, each with a unique canonical name. `glfm_specification/example_snapshots/examples_index.yml` sample entries: ```yaml -02_01_preliminaries_characters_and_lines_1: +02_01_00_preliminaries_characters_and_lines_1: spec_txt_example_position: 1 source_specification: commonmark -03_01_blocks_and_inlines_precedence_1: +03_01_00_blocks_and_inlines_precedence_1: spec_txt_example_position: 12 source_specification: commonmark -05_03_container_blocks_task_list_items_1: +05_03_00_container_blocks_task_list_items_1: spec_txt_example_position: 279 source_specification: github -06_04_inlines_emphasis_and_strong_emphasis_1: +06_04_00_inlines_emphasis_and_strong_emphasis_1: spec_txt_example_position: 360 source_specification: github -07_01_audio_link_1: +07_01_00_audio_link_1: spec_txt_example_position: 301 source_specification: gitlab ``` @@ -923,7 +1004,7 @@ for each entry in `glfm_specification/example_snapshots/examples_index.yml` `glfm_specification/example_snapshots/markdown.yml` sample entry: ```yaml -06_04_inlines_emphasis_and_strong_emphasis_1: | +06_04_00_inlines_emphasis_and_strong_emphasis_1: | *foo bar* ``` @@ -958,7 +1039,7 @@ Any exceptions or failures which occur when generating HTML are replaced with an `glfm_specification/example_snapshots/html.yml` sample entry: ```yaml -06_04_inlines_emphasis_and_strong_emphasis_1: +06_04_00_inlines_emphasis_and_strong_emphasis_1: canonical: | <p><em>foo bar</em></p> static: | @@ -983,7 +1064,7 @@ contains the ProseMirror JSON for each entry in `glfm_specification/example_snap `glfm_specification/example_snapshots/prosemirror_json.yml` sample entry: ```yaml -06_04_inlines_emphasis_and_strong_emphasis_1: |- +06_04_00_inlines_emphasis_and_strong_emphasis_1: |- { "type": "doc", "content": [ diff --git a/doc/development/go_guide/dependencies.md b/doc/development/go_guide/dependencies.md index 2a53fa590e3..7cad5bbf417 100644 --- a/doc/development/go_guide/dependencies.md +++ b/doc/development/go_guide/dependencies.md @@ -44,8 +44,8 @@ end with a timestamp and the first 12 characters of the commit identifier: If a VCS tag matches one of these patterns, it is ignored. -For a complete understanding of Go modules and versioning, see -[this series of blog posts](https://go.dev/blog/using-go-modules) +For a complete understanding of Go modules and versioning, see +[this series of blog posts](https://go.dev/blog/using-go-modules) on the official Go website. ## 'Module' vs 'Package' diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md index 711b0662a8c..3adafa8750f 100644 --- a/doc/development/go_guide/index.md +++ b/doc/development/go_guide/index.md @@ -145,7 +145,7 @@ Go GitLab linter plugins are maintained in the [`gitlab-org/language-tools/go/li ## Dependencies Dependencies should be kept to the minimum. The introduction of a new -dependency should be argued in the merge request, as per our [Approval Guidelines](../code_review.md#approval-guidelines). +dependency should be argued in the merge request, as per our [Approval Guidelines](../code_review.md#approval-guidelines). Both [License Scanning](../../user/compliance/license_compliance/index.md) and [Dependency Scanning](../../user/application_security/dependency_scanning/index.md) should be activated on all projects to ensure new dependencies @@ -153,7 +153,7 @@ security status and license compatibility. ### Modules -In Go 1.11 and later, a standard dependency system is available behind the name +In Go 1.11 and later, a standard dependency system is available behind the name [Go Modules](https://github.com/golang/go/wiki/Modules). It provides a way to define and lock dependencies for reproducible builds. It should be used whenever possible. @@ -166,7 +166,7 @@ projects, and makes merge requests easier to review. In some cases, such as building a Go project for it to act as a dependency of a CI run for another project, removing the `vendor/` directory means the code must be downloaded repeatedly, which can lead to intermittent problems due to rate -limiting or network failures. In these circumstances, you should +limiting or network failures. In these circumstances, you should [cache the downloaded code between](../../ci/caching/index.md#cache-go-dependencies). There was a diff --git a/doc/development/import_export.md b/doc/development/import_export.md index 6cbbb6bf716..c66ac0418ac 100644 --- a/doc/development/import_export.md +++ b/doc/development/import_export.md @@ -305,6 +305,29 @@ export_reorders: nulls_position: :nulls_last ``` +### Conditional export + +When associated resources are from outside the project, you might need to +validate that a user who is exporting the project or group can access these +associations. `include_if_exportable` accepts an array of associations for a +resource. During export, the `exportable_association?` method on the resource +is called with the association's name and user to validate if associated +resource can be included in the export. + +For example: + +```yaml +include_if_exportable: + project: + issues: + - epic_issue +``` + +This definition: + +1. Calls the issue's `exportable_association?(:epic_issue, current_user: current_user)` method. +1. If the method returns true, includes the issue's `epic_issue` association for the issue. + ### Import The import job status moves from `none` to `finished` or `failed` into different states: diff --git a/doc/development/import_project.md b/doc/development/import_project.md index 7c55d2e2668..1f3bf860257 100644 --- a/doc/development/import_project.md +++ b/doc/development/import_project.md @@ -149,26 +149,10 @@ You might see an error like `N is out of range for ActiveModel::Type::Integer wi where `N` is the integer exceeding the 4-byte integer limit. If that's the case, you are likely hitting the issue with rebalancing of `relative_position` field of the issues. -The feature flag to enable the rebalance automatically was enabled on GitLab.com. -We intend to enable it by default on self-managed instances when the issue -[Rebalance issues FF rollout](https://gitlab.com/gitlab-org/gitlab/-/issues/343368) -is implemented. - -If the feature is not enabled by default on your GitLab version, run the following -commands in the [Rails console](../administration/operations/rails_console.md) as -a workaround. Replace the ID with the ID of your project you were trying to import: - ```ruby -# Check if the feature is enabled on your instance. If it is, rebalance should work automatically on your instance -Feature.enabled?(:rebalance_issues,Project.find(ID).root_namespace) - # Check the current maximum value of relative_position Issue.where(project_id: Project.find(ID).root_namespace.all_projects).maximum(:relative_position) -# Enable `rebalance_issues` feauture and check that it was successfully enabled -Feature.enable(:rebalance_issues,Project.find(ID).root_namespace) -Feature.enabled?(:rebalance_issues,Project.find(ID).root_namespace) - # Run the rebalancing process and check if the maximum value of relative_position has changed Issues::RelativePositionRebalancingService.new(Project.find(ID).root_namespace.all_projects).execute Issue.where(project_id: Project.find(ID).root_namespace.all_projects).maximum(:relative_position) diff --git a/doc/development/integrations/jenkins.md b/doc/development/integrations/jenkins.md index f430fc380b1..bb541d26df2 100644 --- a/doc/development/integrations/jenkins.md +++ b/doc/development/integrations/jenkins.md @@ -24,7 +24,7 @@ brew services start jenkins GitLab does not allow requests to localhost or the local network by default. When running Jenkins on your local machine, you need to enable local access. 1. Log into your GitLab instance as an administrator. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Outbound requests** and check the following checkboxes: diff --git a/doc/development/integrations/secure.md b/doc/development/integrations/secure.md index 55e57a3c2ee..2c5dd1c0500 100644 --- a/doc/development/integrations/secure.md +++ b/doc/development/integrations/secure.md @@ -1,6 +1,6 @@ --- -stage: Protect -group: Container Security +stage: Secure +group: Static Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -254,7 +254,7 @@ Following the POSIX exit code standard, the scanner exits with 0 for success and Success also includes the case when vulnerabilities are found. When a CI job fails, security report results are not ingested by GitLab, even if the job -[allows failure](../../ci/yaml/#allow_failure). The report artifacts are still uploaded to GitLab and available +[allows failure](../../ci/yaml/index.md#allow_failure). The report artifacts are still uploaded to GitLab and available for [download in the pipeline security tab](../../user/application_security/vulnerability_report/pipeline.md#download-security-scan-outputs). When executing a scanning job using the [Docker-in-Docker privileged mode](../../user/application_security/sast/index.md#requirements), @@ -488,8 +488,8 @@ the risk. End-users interact with this field, whereas GitLab automatically proce ##### Identifiers The `identifiers` array describes the detected vulnerability. An identifier object's `type` and -`value` fields are used to tell if two identifiers are the same. The user interface uses the -object's `name` and `url` fields to display the identifier. +`value` fields are used to [tell if two identifiers are the same](../../user/application_security/vulnerability_report/pipeline.md#deduplication-process). +The user interface uses the object's `name` and `url` fields to display the identifier. We recommend that you use the identifiers the GitLab scanners already define: @@ -509,12 +509,10 @@ which is shared by some of the analyzers that GitLab maintains. You can [contrib new generic identifiers to if needed. Analyzers may also produce vendor-specific or product-specific identifiers, which don't belong in the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common). -The first item of the `identifiers` array is called the -[primary identifier](../../user/application_security/terminology/index.md#primary-identifier). -The primary identifier is particularly important, because it is used to +The first item of the `identifiers` array is called the +[primary identifier](../../user/application_security/terminology/index.md#primary-identifier), and +it is used to [track vulnerabilities](#tracking-and-merging-vulnerabilities) as new commits are pushed to the repository. -Identifiers are also used to [merge duplicate vulnerabilities](#tracking-and-merging-vulnerabilities) -reported for the same commit, except for `CWE` and `WASC`. Not all vulnerabilities have CVEs, and a CVE can be identified multiple times. As a result, a CVE isn't a stable identifier and you shouldn't assume it as such when tracking vulnerabilities. @@ -666,11 +664,14 @@ Users may give feedback on a vulnerability: GitLab tracks vulnerabilities so that user feedback is not lost when new Git commits are pushed to the repository. -Vulnerabilities are tracked using a combination of three attributes: +Vulnerabilities are tracked using a +[`UUIDv5`](https://gitlab.com/gitlab-org/gitlab/-/blob/1272957c4a55e616569721febccb685c056ca1e4/ee/app/models/vulnerabilities/finding.rb#L364-368) +digest, which is generated by a `SHA-1` hash of four attributes: - [Report type](#category) -- [Location fingerprint](#location) - [Primary identifier](#identifiers) +- [Location fingerprint](#location) +- Project ID Right now, GitLab cannot track a vulnerability if its location changes as new Git commits are pushed, and this results in user feedback being lost. @@ -678,12 +679,7 @@ For instance, user feedback on a SAST vulnerability is lost if the affected file is renamed or the affected line moves down. This is addressed in [issue #7586](https://gitlab.com/gitlab-org/gitlab/-/issues/7586). -In some cases, the multiple scans executed in the same CI pipeline result in duplicates -that are automatically merged using the vulnerability location and identifiers. -Two vulnerabilities are considered to be the same if they share the same [location fingerprint](#location) -and at least one [identifier](#identifiers). Two identifiers are the same if they share the same `type` and `id`. -CWE and WASC identifiers are not considered because they describe categories of vulnerability flaws, -but not specific security flaws. +See also [deduplication process](../../user/application_security/vulnerability_report/pipeline.md#deduplication-process). ##### Severity and confidence diff --git a/doc/development/internal_api/index.md b/doc/development/internal_api/index.md index 9b29af3e433..c35d40a7b7f 100644 --- a/doc/development/internal_api/index.md +++ b/doc/development/internal_api/index.md @@ -148,7 +148,7 @@ curl --request POST --header "Gitlab-Shared-Secret: <Base64 encoded token>" \ ## Authorized Keys Check This endpoint is called by the GitLab Shell authorized keys -check. Which is called by OpenSSH for +check. Which is called by OpenSSH for [fast SSH key lookup](../../administration/operations/fast_ssh_key_lookup.md). | Attribute | Type | Required | Description | diff --git a/doc/development/internal_users.md b/doc/development/internal_users.md index 95ca593e31e..24785c0cf48 100644 --- a/doc/development/internal_users.md +++ b/doc/development/internal_users.md @@ -8,6 +8,8 @@ info: "See the Technical Writers assigned to Development Guidelines: https://abo # Internal users +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97584) in GitLab 15.4, bots are indicated with a badge in user listings. + GitLab uses internal users (sometimes referred to as "bots") to perform actions or functions that cannot be attributed to a regular user. diff --git a/doc/development/lfs.md b/doc/development/lfs.md index 5900eb68294..20157e9e805 100644 --- a/doc/development/lfs.md +++ b/doc/development/lfs.md @@ -76,13 +76,13 @@ process, which writes the contents to the standard output. 1. The archive data is sent back to the client. In step 7, the `gitaly-lfs-smudge` filter must talk to Workhorse, not to -Rails, or an invalid LFS blob is saved. To support this, GitLab 13.5 +Rails, or an invalid LFS blob is saved. To support this, GitLab 13.5 [changed the default Omnibus configuration to have Gitaly talk to the Workhorse](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4592) instead of Rails. One side effect of this change: the correlation ID of the original request is not preserved for the internal API requests made by Gitaly (or `gitaly-lfs-smudge`), such as the one made in step 8. The -correlation IDs for those API requests are random values until +correlation IDs for those API requests are random values until [this Workhorse issue](https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/309) is resolved. diff --git a/doc/development/logging.md b/doc/development/logging.md index f1fa7f4c8c9..467fb68f3ae 100644 --- a/doc/development/logging.md +++ b/doc/development/logging.md @@ -385,7 +385,7 @@ end ## Additional steps with new log files 1. Consider log retention settings. By default, Omnibus rotates any - logs in `/var/log/gitlab/gitlab-rails/*.log` every hour and + logs in `/var/log/gitlab/gitlab-rails/*.log` every hour and [keep at most 30 compressed files](https://docs.gitlab.com/omnibus/settings/logs.html#logrotate). On GitLab.com, that setting is only 6 compressed files. These settings should suffice for most users, but you may need to tweak them in [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab). @@ -395,7 +395,7 @@ end a merge request to the [`gitlab_fluentd`](https://gitlab.com/gitlab-cookbooks/gitlab_fluentd) project. See [this example](https://gitlab.com/gitlab-cookbooks/gitlab_fluentd/-/merge_requests/51/diffs). -1. Be sure to update the [GitLab CE/EE documentation](../administration/logs/index.md) and the +1. Be sure to update the [GitLab CE/EE documentation](../administration/logs/index.md) and the [GitLab.com runbooks](https://gitlab.com/gitlab-com/runbooks/blob/master/docs/logging/README.md). ## Control logging visibility diff --git a/doc/development/merge_request_concepts/index.md b/doc/development/merge_request_concepts/index.md index 331f0e01579..d463f6ba290 100644 --- a/doc/development/merge_request_concepts/index.md +++ b/doc/development/merge_request_concepts/index.md @@ -10,7 +10,7 @@ info: "See the Technical Writers assigned to Development Guidelines: https://abo NOTE: The documentation below is the single source of truth for the merge request terminology and functionality. -The merge request is made up of several different key components and ideas that encompass the overall merge request experience. These concepts sometimes have competing and confusing terminology or overlap with other concepts. The concepts this will cover are: +The merge request is made up of several different key components and ideas that encompass the overall merge request experience. These concepts sometimes have competing and confusing terminology or overlap with other concepts. This page covers the following concepts: 1. Merge widget 1. Report widgets @@ -40,15 +40,17 @@ Reports are widgets within the merge request that report information about chang ## Merge checks -Merge checks are statuses that can either pass or fail and conditionally control the availability of the merge button being available within a merge request. The key distinguishing factor in a merge check is that users **do not** interact with the merge checks inside of the merge request, but are able to influence whether or not the check passes or fails. Results from the check are processed as true/false to determine whether or not a merge request can be merged. Examples include: +Merge checks are statuses that can either pass or fail and conditionally control the availability of the merge button being available within a merge request. The key distinguishing factor in a merge check is that users **do not** interact with the merge checks inside of the merge request, but are able to influence whether or not the check passes or fails. Results from the check are processed as true/false to determine whether or not a merge request can be merged. -- Merge conflicts. -- Pipeline success. -- Threads resolution. -- [External status checks](../../user/project/merge_requests/status_checks.md). -- Required approvals. +Examples of merge checks include: -When all of the required merge checks are satisfied a merge request becomes mergeable. +- Merge conflicts +- Pipeline success +- Threads resolution +- [External status checks](../../user/project/merge_requests/status_checks.md) +- Required approvals + +A merge request can be merged only when all of the required merge checks are satisfied. ## Approvals @@ -58,8 +60,8 @@ Additionally, approval settings provide configuration options to define how thos Examples of approval rules and settings include: -1. [merge request approval rules](../../user/project/merge_requests/approvals/rules.md) -1. [code owner approvals](../../user/project/code_owners.md) -1. [security approvals](../../user/application_security/index.md#security-approvals-in-merge-requests) -1. [prevent editing approval rules](../../user/project/merge_requests/approvals/settings.md#prevent-editing-approval-rules-in-merge-requests) -1. [remove all approvals when commits are added](../../user/project/merge_requests/approvals/settings.md#remove-all-approvals-when-commits-are-added-to-the-source-branch) +- [Merge request approval rules](../../user/project/merge_requests/approvals/rules.md) +- [Code owner approvals](../../user/project/code_owners.md) +- [Security approvals](../../user/application_security/index.md#security-approvals-in-merge-requests) +- [Prevent editing approval rules](../../user/project/merge_requests/approvals/settings.md#prevent-editing-approval-rules-in-merge-requests) +- [Remove all approvals when commits are added](../../user/project/merge_requests/approvals/settings.md#remove-all-approvals-when-commits-are-added-to-the-source-branch) diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md index 7ff25705ae6..895fc6f92a4 100644 --- a/doc/development/merge_request_performance_guidelines.md +++ b/doc/development/merge_request_performance_guidelines.md @@ -394,7 +394,7 @@ query for every mention of `@alice`. Caching data per transaction can be done using [RequestStore](https://github.com/steveklabnik/request_store) (use `Gitlab::SafeRequestStore` to avoid having to remember to check -`RequestStore.active?`). Caching data in Redis can be done using +`RequestStore.active?`). Caching data in Redis can be done using [Rails' caching system](https://guides.rubyonrails.org/caching_with_rails.html). ## Pagination diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 64d8b22f1b8..4e569579f37 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -1228,7 +1228,7 @@ If using a model in the migrations, you should first [clear the column cache](https://api.rubyonrails.org/classes/ActiveRecord/ModelSchema/ClassMethods.html#method-i-reset_column_information) using `reset_column_information`. -If using a model that leverages single table inheritance (STI), there are +If using a model that leverages single table inheritance (STI), there are [special considerations](database/single_table_inheritance.md#in-migrations). This avoids problems where a column that you are using was altered and cached diff --git a/doc/development/newlines_styleguide.md b/doc/development/newlines_styleguide.md index 57962129b2f..014affa3e04 100644 --- a/doc/development/newlines_styleguide.md +++ b/doc/development/newlines_styleguide.md @@ -1,108 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'backend/ruby_style_guide.md#newlines-style-guide' +remove_date: '2022-12-15' --- -# Newlines style guide +This document was moved to [another location](backend/ruby_style_guide.md#newlines-style-guide). -This style guide recommends best practices for newlines in Ruby code. - -## Rule: separate code with newlines only to group together related logic - -```ruby -# bad -def method - issue = Issue.new - - issue.save - - render json: issue -end -``` - -```ruby -# good -def method - issue = Issue.new - issue.save - - render json: issue -end -``` - -## Rule: separate code and block with newlines - -### Newline before block - -```ruby -# bad -def method - issue = Issue.new - if issue.save - render json: issue - end -end -``` - -```ruby -# good -def method - issue = Issue.new - - if issue.save - render json: issue - end -end -``` - -## Newline after block - -```ruby -# bad -def method - if issue.save - issue.send_email - end - render json: issue -end -``` - -```ruby -# good -def method - if issue.save - issue.send_email - end - - render json: issue -end -``` - -### Exception: no need for newline when code block starts or ends right inside another code block - -```ruby -# bad -def method - - if issue - - if issue.valid? - issue.save - end - - end - -end -``` - -```ruby -# good -def method - if issue - if issue.valid? - issue.save - end - end -end -``` +<!-- This redirect file can be deleted after 2022-12-15. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/packages/new_format_development.md b/doc/development/packages/new_format_development.md index f7d02f9160b..73a2b2f1f81 100644 --- a/doc/development/packages/new_format_development.md +++ b/doc/development/packages/new_format_development.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w This document guides you through adding support to GitLab for a new a [package management system](../../administration/packages/index.md). -See the already supported formats in the [Packages & Registries documentation](../../user/packages/index.md) +See the already supported formats in the [Packages and registries documentation](../../user/packages/index.md) It is possible to add a new format with only backend changes. This guide is superficial and does not cover the way the code should be written. @@ -91,7 +91,7 @@ extended when possible to keep the common package logic grouped as much as possi ### Configuration -GitLab has a `packages` section in its configuration file (`gitlab.rb`). +GitLab has a `packages` section in its configuration file (`gitlab.rb` or `gitlab.yml`). It applies to all package systems supported by GitLab. Usually you don't need to add anything there. diff --git a/doc/development/permissions.md b/doc/development/permissions.md index 8e517b8577c..d348d659cad 100644 --- a/doc/development/permissions.md +++ b/doc/development/permissions.md @@ -147,15 +147,41 @@ into different features like Merge Requests and CI flow. ## Where should permissions be checked? -By default, controllers, API endpoints, and GraphQL types/fields are responsible for authorization. See [Secure Coding Guidelines > Permissions](secure_coding_guidelines.md#permissions). +We should typically apply defense-in-depth (implementing multiple checks at +various layers) starting with low-level layers, such as finders and services, +followed by high-level layers, such as GraphQL, public REST API, and controllers. + +See [Guidelines for reusing abstractions](reusing_abstractions.md). + +Protecting the same resources at many points means that if one layer of defense is compromised +or missing, customer data is still protected by the additional layers. + +See the permissions section in the [Secure Coding Guidelines](secure_coding_guidelines.md#permissions). ### Considerations -- Many actions are completely or partially extracted to services, finders, and other classes, so it is normal to do permission checks "downstream". -- Often, authorization logic must be incorporated in DB queries to filter records. +Services or finders are appropriate locations because: + +- Multiple endpoints share services or finders so downstream logic is more likely to be re-used. +- Sometimes authorization logic must be incorporated in DB queries to filter records. +- Permission checks at the display layer should be avoided except to provide better UX + and not as a security check. For example, showing and hiding non-data elements like buttons. + +The downsides to defense-in-depth are: + - `DeclarativePolicy` rules are relatively performant, but conditions may perform database calls. -- Multiple permission checks across layers can be difficult to reason about, which is its own security risk. For example, duplicate authorization logic could diverge. -- Should we apply defense-in-depth with permission checks? [Join the discussion](https://gitlab.com/gitlab-org/gitlab/-/issues/324135) +- Higher maintenance costs. + +### Exceptions + +Developers can choose to do authorization in only a single area after weighing +the risks and drawbacks for their specific case. + +Prefer domain logic (services or finders) as the source of truth when making exceptions. + +Logic, like backend worker logic, might not need authorization based on the current user. +If the service or finder's constructor does not expect `current_user`, then it typically won't +check permissions. ### Tips diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index d57e5bbeb26..648a2aac339 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -103,7 +103,7 @@ When you need to revert a merge request, to get accelerated feedback, you can ad When this label is assigned, the following steps of the CI/CD pipeline are skipped: -- The `package-and-qa` job. +- The `e2e:package-and-test` job. - The `rspec:undercoverage` job. - The entire [Review Apps process](testing_guide/review_apps.md). @@ -221,8 +221,8 @@ that includes `rspec-profile` in their name. ### Logging -- Rails logging to `log/test.log` is disabled by default in CI - [for performance reasons](https://jtway.co/speed-up-your-rails-test-suite-by-6-in-1-line-13fedb869ec4). +- Rails logging to `log/test.log` is disabled by default in CI + [for performance reasons](https://jtway.co/speed-up-your-rails-test-suite-by-6-in-1-line-13fedb869ec4). To override this setting, provide the `RAILS_ENABLE_TEST_LOG` environment variable. @@ -341,13 +341,20 @@ We also run our test suite against PG11 upon specific database library changes i ### Current versions testing -| Where? | PostgreSQL version | Ruby version | -| ------ | ------------------ | ------------ | -| Merge requests | 12 (default version), 11 for DB library changes | 2.7 (default version) | -| `master` branch commits | 12 (default version), 11 for DB library changes | 2.7 (default version) | -| `maintenance` scheduled pipelines (every 2 hours at even hour) | 12 (default version), 11 for DB library changes | 2.7 (default version) | -| `maintenance` scheduled pipelines (every 2 hours at odd hour) | 12 (default version), 11 for DB library changes | 3.0 (set in the schedule variables) | -| `nightly` scheduled pipelines | 12 (default version), 11, 13 | 2.7 (default version) | +| Where? | PostgreSQL version | Ruby version | +|------------------------------------------------------------------------------------------------|-------------------------------------------------|--------------| +| Merge requests | 12 (default version), 11 for DB library changes | 2.7 (default version) | +| `master` branch commits | 12 (default version), 11 for DB library changes | 2.7 (default version) | +| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour) | 12 (default version), 11 for DB library changes | 2.7 (default version) | +| `maintenance` scheduled pipelines for the `ruby3` branch (every odd-numbered hour), see below. | 12 (default version), 11 for DB library changes | 3.0 (coded in the branch) | +| `nightly` scheduled pipelines for the `master` branch | 12 (default version), 11, 13 | 2.7 (default version) | + +The pipeline configuration for the scheduled pipeline testing Ruby 3 is +stored in the `ruby3-sync` branch. The pipeline updates the `ruby3` branch +with latest `master`, and then it triggers a regular branch pipeline for +`ruby3`. Any changes in `ruby3` are only for running the pipeline. It should +never be merged back to `master`. Any other Ruby 3 changes should go into +`master` directly, which should be compatible with Ruby 2.7. ### Long-term plan @@ -489,7 +496,7 @@ graph RL; class 2_3-1 criticalPath; 2_3-1 --> 1-5 - 2_4-1["package-and-qa (102 minutes)"]; + 2_4-1["e2e:package-and-test (102 minutes)"]; class 2_4-1 criticalPath; click 2_4-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914305&udv=0" 2_4-1 --> 1-2 & 2_3-1 & 1-15; diff --git a/doc/development/policies.md b/doc/development/policies.md index f0c9d0ec5f9..ddd6b8e9bce 100644 --- a/doc/development/policies.md +++ b/doc/development/policies.md @@ -10,6 +10,8 @@ The DeclarativePolicy framework is designed to assist in performance of policy c The policy used is based on the subject's class name - so `Ability.allowed?(user, :some_ability, project)` creates a `ProjectPolicy` and check permissions on that. +The Ruby gem source is available in the [declarative-policy](https://gitlab.com/gitlab-org/ruby/gems/declarative-policy) GitLab project. + ## Managing Permission Rules Permissions are broken into two parts: `conditions` and `rules`. Conditions are boolean expressions that can access the database and the environment, while rules are statically configured combinations of expressions and other rules that enable or prevent certain abilities. For an ability to be allowed, it must be enabled by at least one rule, and not prevented by any. diff --git a/doc/development/rails_update.md b/doc/development/rails_update.md index 9907a78421f..bda21860eae 100644 --- a/doc/development/rails_update.md +++ b/doc/development/rails_update.md @@ -27,7 +27,7 @@ We strive to run GitLab using the latest Rails releases to benefit from performa 1. Run `yarn patch-package @rails/ujs` after updating this to ensure our local patch file version matches. 1. Create an MR with the `pipeline:run-all-rspec` label and see if pipeline breaks. 1. To resolve and debug spec failures use `git bisect` against the rails repository. See the [debugging section](#git-bisect-against-rails) below. -1. Include links to the Gem diffs between the two versions in the merge request description. For example, this is the gem diff for +1. Include links to the Gem diffs between the two versions in the merge request description. For example, this is the gem diff for [`activesupport` 6.1.3.2 to 6.1.4.1](https://my.diffend.io/gems/activerecord/6.1.3.2/6.1.4.1). ### Prepare an MR for Gitaly diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index c13f1195df3..f300904fc19 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -188,6 +188,8 @@ Alternatively you can use the following on each spec run, bundle exec spring rspec some_spec.rb ``` +## RuboCop tasks + ## Generate initial RuboCop TODO list One way to generate the initial list is to run the Rake task `rubocop:todo:generate`: @@ -209,6 +211,18 @@ Some shells require brackets to be escaped or quoted. See [Resolving RuboCop exceptions](contributing/style_guides.md#resolving-rubocop-exceptions) on how to proceed from here. +### Run RuboCop in graceful mode + +You can run RuboCop in "graceful mode". This means all enabled cop rules are +silenced which have "grace period" activated (via `Details: grace period`). + +Run: + +```shell +bundle exec rake 'rubocop:check:graceful' +bundle exec rake 'rubocop:check:graceful[Gitlab/NamespacedClass]' +``` + ## Compile Frontend Assets You shouldn't ever need to compile frontend assets manually in development, but diff --git a/doc/development/real_time.md b/doc/development/real_time.md index 21f3ee1f3b2..f113d4dd3b7 100644 --- a/doc/development/real_time.md +++ b/doc/development/real_time.md @@ -60,7 +60,7 @@ downstream services. To mitigate this, ensure that the code establishing the new WebSocket connection is feature flagged and defaulted to `off`. A careful, percentage-based roll-out -of the feature flag ensures that effects can be observed on the +of the feature flag ensures that effects can be observed on the [WebSocket dashboard](https://dashboards.gitlab.net/d/websockets-main/websockets-overview?orgId=1) 1. Create a diff --git a/doc/development/redis/new_redis_instance.md b/doc/development/redis/new_redis_instance.md index efaf1e5a6d0..24885b40eb9 100644 --- a/doc/development/redis/new_redis_instance.md +++ b/doc/development/redis/new_redis_instance.md @@ -265,7 +265,7 @@ instances to cope without this functional partition. If we decide to keep the migration code: - We should document the migration steps. -- If we used a feature flag, we should ensure it's an +- If we used a feature flag, we should ensure it's an [ops type feature flag](../feature_flags/index.md#ops-type), as these are long-lived flags. Otherwise, we can remove the flags and conclude the project. diff --git a/doc/development/reusing_abstractions.md b/doc/development/reusing_abstractions.md index ef4e8b0310f..826782d7036 100644 --- a/doc/development/reusing_abstractions.md +++ b/doc/development/reusing_abstractions.md @@ -206,6 +206,31 @@ response = ServiceResponse.success(payload: { issue: issue }) response.payload[:issue] # => issue ``` +Error responses can also specify the failure `reason` which can be used by the caller +to understand the nature of the failure. +The caller, if an HTTP endpoint, could translate the reason symbol into an HTTP status code: + +```ruby +response = ServiceResponse.error( + message: 'Job is in a state that cannot be retried', + reason: :job_not_retrieable) + +if response.success? + head :ok +if response.reason == :job_not_retriable + head :unprocessable_entity +else + head :bad_request +end +``` + +For common failures such as resource `:not_found` or operation `:forbidden`, we could +leverage the Rails [HTTP status symbols](http://www.railsstatuscodes.com/) as long as +they are sufficiently specific for the domain logic involved. +For other failures use domain-specific reasons whenever possible. + +For example: `:job_not_retriable`, `:duplicate_package`, `:merge_request_not_mergeable`. + ### Finders Everything in `app/finders`, typically used for retrieving data from a database. diff --git a/doc/development/routing.md b/doc/development/routing.md index 3d5857b4237..54a531730f9 100644 --- a/doc/development/routing.md +++ b/doc/development/routing.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Routing -The GitLab backend is written primarily with Rails so it uses +The GitLab backend is written primarily with Rails so it uses [Rails routing](https://guides.rubyonrails.org/routing.html). Beside Rails best practices, there are few rules unique to the GitLab application. To support subgroups, GitLab project and group routes use the wildcard diff --git a/doc/development/ruby3_gotchas.md b/doc/development/ruby3_gotchas.md index dbe6fa13eee..db328b0b1a5 100644 --- a/doc/development/ruby3_gotchas.md +++ b/doc/development/ruby3_gotchas.md @@ -163,3 +163,40 @@ For Ruby 3 compliance, this should be changed to one of the following invocation - `f(**{k: v})` - `f(k: v)` + +## RSpec `with` argument matcher fails for shorthand Hash syntax + +Because keyword arguments ("kwargs") are a first-class concept in Ruby 3, keyword arguments are not +converted into internal `Hash` instances anymore. This leads to RSpec method argument matchers failing +when the receiver takes a positional options hash instead of kwargs: + +```ruby +def m(options={}); end +``` + +```ruby +expect(subject).to receive(:m).with(a: 42) +``` + +In Ruby 3 this expectations fails with the following error: + +```plaintext + Failure/Error: + + #<subject> received :m with unexpected arguments + expected: ({:a=>42}) + got: ({:a=>42}) +``` + +This happens because RSpec uses a kwargs argument matcher here, but the method takes a hash. +It works in Ruby 2, because `a: 42` is converted to a hash first and RSpec will use a hash argument matcher. + +A workaround is to not use the shorthand syntax and pass an actual `Hash` instead whenever we know a method +to take an options hash: + +```ruby +# Note the braces around the key-value pair. +expect(subject).to receive(:m).with({ a: 42 }) +``` + +For more information, see [the official issue report for RSpec](https://github.com/rspec/rspec-mocks/issues/1460). diff --git a/doc/development/scalability.md b/doc/development/scalability.md index b7ee0ca1167..66f436bd391 100644 --- a/doc/development/scalability.md +++ b/doc/development/scalability.md @@ -35,7 +35,7 @@ The application has a tight coupling to the database schema. When the application starts, Rails queries the database schema, caching the tables and column types for the data requested. Because of this schema cache, dropping a column or table while the application is running can produce 500 errors to the -user. This is why we have a +user. This is why we have a [process for dropping columns and other no-downtime changes](database/avoiding_downtime_in_migrations.md). #### Multi-tenancy @@ -61,10 +61,10 @@ There are two ways to deal with this: - Sharding. Distribute data across multiple databases. Partitioning is a built-in PostgreSQL feature and requires minimal changes -in the application. However, it +in the application. However, it [requires PostgreSQL 11](https://www.2ndquadrant.com/en/blog/partitioning-evolution-postgresql-11/). -For example, a natural way to partition is to +For example, a natural way to partition is to [partition tables by dates](https://gitlab.com/groups/gitlab-org/-/epics/2023). For example, the `events` and `audit_events` table are natural candidates for this kind of partitioning. @@ -77,9 +77,9 @@ to abstract data access into API calls that abstract the database from the application, but this is a significant amount of work. There are solutions that may help abstract the sharding to some extent -from the application. For example, we want to look at +from the application. For example, we want to look at [Citus Data](https://www.citusdata.com/product/community) closely. Citus Data -provides a Rails plugin that adds a +provides a Rails plugin that adds a [tenant ID to ActiveRecord models](https://www.citusdata.com/blog/2017/01/05/easily-scale-out-multi-tenant-apps/). Sharding can also be done based on feature verticals. This is the @@ -97,11 +97,11 @@ systems. #### Database size -A recent +A recent [database checkup shows a breakdown of the table sizes on GitLab.com](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/8022#master-1022016101-8). Since `merge_request_diff_files` contains over 1 TB of data, we want to -reduce/eliminate this table first. GitLab has support for -[storing diffs in object storage](../administration/merge_request_diffs.md), which we +reduce/eliminate this table first. GitLab has support for +[storing diffs in object storage](../administration/merge_request_diffs.md), which we [want to do on GitLab.com](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/7356). #### High availability @@ -149,7 +149,7 @@ limitation: - Use a multi-threaded connection pooler (for example, [Odyssey](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/7776). -On some Linux systems, it's possible to run +On some Linux systems, it's possible to run [multiple PgBouncer instances on the same port](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4796). On GitLab.com, we run multiple PgBouncer instances on different ports to diff --git a/doc/development/sec/index.md b/doc/development/sec/index.md new file mode 100644 index 00000000000..06c20cee0bb --- /dev/null +++ b/doc/development/sec/index.md @@ -0,0 +1,69 @@ +--- +stage: Secure +group: Static Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: index, concepts, howto +--- + +# Sec section development **(FREE)** + +The Sec section is responsible for GitLab application security features, the "Sec" part of +DevSecOps. Development guides that are specific to the Sec section are listed here. + +See [Terminology](../../user/application_security/terminology) for an overview of our shared terminology. + +## Architecture + +- [Overview](#overview) +- [Scanning](#scanning) +- [Processing, visualization, and management](#processing-visualization-and-management) +- [Severity Levels](../../user/application_security/vulnerabilities/severities.md) + +## Overview + +The architecture supporting the Secure features is split into two main parts: + +- Scanning +- Processing, visualization, and management + +```mermaid +flowchart LR + subgraph G1[Scanning] + Scanner + Analyzer + CI[CI Jobs] + end + subgraph G2[Processing, visualization, and management] + Parsers + Database + Views + Interactions + end + G1 --Report Artifact--> G2 +``` + +### Scanning + +The scanning part is responsible for finding vulnerabilities in given resources, and exporting results. +The scans are executed in CI/CD jobs via several small projects called [Analyzers](../../user/application_security/terminology/index.md#analyzer), which can be found in our [Analyzers sub-group](https://gitlab.com/gitlab-org/security-products/analyzers). +The Analyzers are wrappers around security tools called [Scanners](../../user/application_security/terminology/index.md#scanner), developed internally or externally, to integrate them into GitLab. +The Analyzers are mainly written in Go. + +Some 3rd party integrators also make additional Scanners available by following our [integration documentation](../integrations/secure.md), which leverages the same architecture. + +The results of the scans are exported as JSON reports that must comply with the [Secure report format](../../user/application_security/terminology/index.md#secure-report-format) and are uploaded as [CI/CD Job Report artifacts](../../ci/pipelines/job_artifacts.md) to make them available for processing after the pipelines completes. + +### Processing, visualization, and management + +After the data is available as a Report Artifact it can be processed by the GitLab Rails application to enable our security features, including: + +- [Security Dashboards](../../user/application_security/security_dashboard/index.md), Merge Request widget, Pipeline view, and so on. +- [Interactions with vulnerabilities](../../user/application_security/index.md#interact-with-findings-and-vulnerabilities). +- [Approval rules](../../user/application_security/index.md#security-approvals-in-merge-requests). + +Depending on the context, the security reports may be stored either in the database or stay as Report Artifacts for on-demand access. + +## CI/CD template development + +While CI/CD templates are the responsibiility of the Verify section, many are critical to the Sec Section's feature usage. +If you are working with CI/CD templates, please read the [development guide for GitLab CI/CD templates](../cicd/templates.md). diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md index 8053b4285e6..4c2f3118366 100644 --- a/doc/development/secure_coding_guidelines.md +++ b/doc/development/secure_coding_guidelines.md @@ -81,7 +81,7 @@ text = "foo\nbar" p text.match /^bar$/ ``` -The output of this example is `#<MatchData "bar">`, as Ruby treats the input `text` line by line. In order to match the whole __string__ the Regex anchors `\A` and `\z` should be used. +The output of this example is `#<MatchData "bar">`, as Ruby treats the input `text` line by line. To match the whole **string**, the Regex anchors `\A` and `\z` should be used. #### Impact diff --git a/doc/development/service_ping/implement.md b/doc/development/service_ping/implement.md index 0ebc58dd669..5448bbb4293 100644 --- a/doc/development/service_ping/implement.md +++ b/doc/development/service_ping/implement.md @@ -127,7 +127,7 @@ Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recomme #### Grouping and batch operations -The `count`, `distinct_count`, `sum`, and `average` batch counters can accept an `ActiveRecord::Relation` +The `count`, `distinct_count` and `sum` batch counters can accept an `ActiveRecord::Relation` object, which groups by a specified column. With a grouped relation, the methods do batch counting, handle errors, and returns a hash table of key-value pairs. @@ -142,9 +142,6 @@ distinct_count(Project.group(:visibility_level), :creator_id) sum(Issue.group(:state_id), :weight)) # returns => {1=>3542, 2=>6820} - -average(Issue.group(:state_id), :weight)) -# returns => {1=>3.5, 2=>2.5} ``` #### Add operation @@ -275,7 +272,7 @@ Events are handled by counter classes in the `Gitlab::UsageDataCounters` namespa 1. Listed in [`Gitlab::UsageDataCounters::COUNTERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters.rb#L5) to be then included in `Gitlab::UsageData`. -1. Specified in the metric definition using the `RedisMetric` instrumentation class as a `counter_class` option to be picked up using the [metric instrumentation](metrics_instrumentation.md) framework. Refer to the [Redis metrics](metrics_instrumentation.md#redis-metrics) documentation for an example implementation. +1. Specified in the metric definition using the `RedisMetric` instrumentation class by their `prefix` option to be picked up using the [metric instrumentation](metrics_instrumentation.md) framework. Refer to the [Redis metrics](metrics_instrumentation.md#redis-metrics) documentation for an example implementation. Inheriting classes are expected to override `KNOWN_EVENTS` and `PREFIX` constants to build event names and associated metrics. For example, for prefix `issues` and events array `%w[create, update, delete]`, three metrics will be added to the Service Ping payload: `counts.issues_create`, `counts.issues_update` and `counts.issues_delete`. @@ -385,7 +382,7 @@ Implemented using Redis methods [PFADD](https://redis.io/commands/pfadd/) and [P - `controller_actions`: the controller actions to track. - `name`: the event name. - `conditions`: optional custom conditions. Uses the same format as Rails callbacks. - - `destinations`: optional list of destinations. Currently supports `:redis_hll` and `:snowplow`. Default: [:redis_hll]. + - `destinations`: optional list of destinations. Currently supports `:redis_hll` and `:snowplow`. Default: `:redis_hll`. - `&block`: optional block that computes and returns the `custom_id` that we want to track. This overrides the `visitor_id`. Example: @@ -623,7 +620,7 @@ alt_usage_data(999) ### Add counters to build new metrics When adding the results of two counters, use the `add` Service Data method that -handles fallback values and exceptions. It also generates a valid [SQL export](index.md#export-service-ping-sql-queries-and-definitions). +handles fallback values and exceptions. It also generates a valid [SQL export](index.md#export-service-ping-data). Example: @@ -773,7 +770,7 @@ To set up Service Ping locally, you must: 1. Using the `gitlab` Rails console, manually trigger Service Ping: ```ruby - ServicePing::SubmitService.new.execute + GitlabServicePingWorker.new.perform('triggered_from_cron' => false) ``` 1. Use the `versions` Rails console to check the Service Ping was successfully received, @@ -809,7 +806,7 @@ This is the recommended approach to test Prometheus-based Service Ping. To verify your change, build a new Omnibus image from your code branch using CI/CD, download the image, and run a local container instance: -1. From your merge request, select the `qa` stage, then trigger the `package-and-qa` job. This job triggers an Omnibus +1. From your merge request, select the `qa` stage, then trigger the `e2e:package-and-test` job. This job triggers an Omnibus build in a [downstream pipeline of the `omnibus-gitlab-mirror` project](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/-/pipelines). 1. In the downstream pipeline, wait for the `gitlab-docker` job to finish. 1. Open the job logs and locate the full container name including the version. It takes the following form: `registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>`. diff --git a/doc/development/service_ping/index.md b/doc/development/service_ping/index.md index 4481fe33bda..f252eb967aa 100644 --- a/doc/development/service_ping/index.md +++ b/doc/development/service_ping/index.md @@ -374,9 +374,9 @@ Possible values are "Amazon Aurora PostgreSQL", "PostgreSQL on Amazon RDS", "Clo In GitLab 13.5, `pg_system_id` was added to send the [PostgreSQL system identifier](https://www.2ndquadrant.com/en/blog/support-for-postgresqls-system-identifier-in-barman/). -## Export Service Ping SQL queries and definitions +## Export Service Ping data -Two Rake tasks exist to export Service Ping definitions. +Rake tasks exist to export Service Ping data in different formats. - The Rake tasks export the raw SQL queries for `count`, `distinct_count`, `sum`. - The Rake tasks export the Redis counter class or the line of the Redis block for `redis_usage_data`. @@ -385,12 +385,15 @@ Two Rake tasks exist to export Service Ping definitions. In the home directory of your local GitLab installation run the following Rake tasks for the YAML and JSON versions respectively: ```shell -# for YAML export +# for YAML export of SQL queries bin/rake gitlab:usage_data:dump_sql_in_yaml -# for JSON export +# for JSON export of SQL queries bin/rake gitlab:usage_data:dump_sql_in_json +# for JSON export of Non SQL data +bin/rake gitlab:usage_data:dump_non_sql_in_json + # You may pipe the output into a file bin/rake gitlab:usage_data:dump_sql_in_yaml > ~/Desktop/usage-metrics-2020-09-02.yaml ``` @@ -405,7 +408,7 @@ To generate Service Ping, use [Teleport](https://goteleport.com/docs/) or a deta 1. Request temporary [access](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/Teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to the required environment. 1. After your approval is issued, [access the Rails console](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/Teleport/Connect_to_Rails_Console_via_Teleport.md#access-approval). -1. Run `ServicePing::SubmitService.new.execute`. +1. Run `GitlabServicePingWorker.new.perform('triggered_from_cron' => false)`. #### Trigger Service Ping with a detached screen session @@ -430,7 +433,7 @@ To generate Service Ping, use [Teleport](https://goteleport.com/docs/) or a deta 1. Run: ```shell - ServicePing::SubmitService.new.execute + GitlabServicePingWorker.new.perform('triggered_from_cron' => false) ``` 1. To detach from screen, press `ctrl + A`, `ctrl + D`. @@ -490,7 +493,7 @@ To skip database write operations, DevOps report creation, and storage of usage ```shell skip_db_write: -ServicePing::SubmitService.new(skip_db_write: true).execute +GitlabServicePingWorker.new.perform('triggered_from_cron' => false, 'skip_db_write' => true) ``` ## Monitoring diff --git a/doc/development/service_ping/metrics_instrumentation.md b/doc/development/service_ping/metrics_instrumentation.md index 9dc37386111..debb3a68c04 100644 --- a/doc/development/service_ping/metrics_instrumentation.md +++ b/doc/development/service_ping/metrics_instrumentation.md @@ -154,22 +154,24 @@ end You can use Redis metrics to track events not kept in the database, for example, a count of how many times the search bar has been used. -[Example of a merge request that adds a `Redis` metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66582). +[Example of a merge request that adds a `Redis` metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97009). + +Please note that `RedisMetric` class can only be used as the `instrumentation_class` for Redis metrics with simple counters classes (classes that only inherit `BaseCounter` and set `PREFIX` and `KNOWN_EVENTS` constants). In case the counter class has additional logic included in it, a new `instrumentation_class`, inheriting from `RedisMetric`, needs to be created. This new class needs to include the additional logic from the counter class. Count unique values for `source_code_pushes` event. Required options: - `event`: the event name. -- `counter_class`: one of the counter classes from the `Gitlab::UsageDataCounters` namespace; it should implement `read` method or inherit it from `BaseCounter`. +- `prefix`: the value of the `PREFIX` constant used in the counter classes from the `Gitlab::UsageDataCounters` namespace. ```yaml time_frame: all data_source: redis -instrumentation_class: 'RedisMetric' +instrumentation_class: RedisMetric options: event: pushes - counter_class: SourceCodeCounter + prefix: source_code ``` ### Availability-restrained Redis metrics @@ -197,14 +199,14 @@ You must also use the class's name in the YAML setup. ```yaml time_frame: all data_source: redis -instrumentation_class: 'MergeUsageCountRedisMetric' +instrumentation_class: MergeUsageCountRedisMetric options: event: pushes - counter_class: SourceCodeCounter + prefix: source_code ``` ## Redis HyperLogLog metrics - + You can use Redis HyperLogLog metrics to track events not kept in the database and incremented for unique values such as unique users, for example, a count of how many different users used the search bar. @@ -215,7 +217,7 @@ Count unique values for `i_quickactions_approve` event. ```yaml time_frame: 28d data_source: redis_hll -instrumentation_class: 'RedisHLLMetric' +instrumentation_class: RedisHLLMetric options: events: - i_quickactions_approve @@ -246,7 +248,7 @@ You must also use the class's name in the YAML setup. ```yaml time_frame: 28d data_source: redis_hll -instrumentation_class: 'MergeUsageCountRedisHLLMetric' +instrumentation_class: MergeUsageCountRedisHLLMetric options: events: - i_quickactions_approve @@ -286,7 +288,7 @@ You must also include the instrumentation class name in the YAML setup. ```yaml time_frame: 28d -instrumentation_class: 'IssuesBoardsCountMetric' +instrumentation_class: IssuesBoardsCountMetric ``` ## Generic metrics @@ -337,13 +339,13 @@ To create a stub instrumentation for a Service Ping metric, you can use a dedica The generator takes the class name as an argument and the following options: - `--type=TYPE` Required. Indicates the metric type. It must be one of: `database`, `generic`, `redis`, `numbers`. -- `--operation` Required for `database` & `numebers` type. +- `--operation` Required for `database` & `numbers` type. - For `database` it must be one of: `count`, `distinct_count`, `estimate_batch_distinct_count`, `sum`, `average`. - For `numbers` it must be: `add`. - `--ee` Indicates if the metric is for EE. ```shell -rails generate gitlab:usage_metric CountIssues --type database +rails generate gitlab:usage_metric CountIssues --type database --operation distinct_count create lib/gitlab/usage/metrics/instrumentations/count_issues_metric.rb create spec/lib/gitlab/usage/metrics/instrumentations/count_issues_metric_spec.rb ``` diff --git a/doc/development/service_ping/troubleshooting.md b/doc/development/service_ping/troubleshooting.md index 29ab334f867..4431d5df3ff 100644 --- a/doc/development/service_ping/troubleshooting.md +++ b/doc/development/service_ping/troubleshooting.md @@ -4,7 +4,7 @@ group: Product Intelligence info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Troubleshooting +# Troubleshooting Service Ping ## Service Ping Payload drop @@ -58,7 +58,7 @@ checking the configuration file of your GitLab instance: - Using the Admin Area: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Expand **Usage Statistics**. 1. Are you able to check or uncheck the checkbox to disable Service Ping? @@ -115,7 +115,7 @@ To work around this bug, you have two options: sudo gitlab-ctl reconfigure ``` - 1. In GitLab, on the top bar, select **Menu > Admin**. + 1. In GitLab, on the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Expand **Usage Statistics**. 1. Clear the **Enable Service Ping** checkbox. diff --git a/doc/development/shell_scripting_guide/index.md b/doc/development/shell_scripting_guide/index.md index 3d58fabad72..0591d2c64d0 100644 --- a/doc/development/shell_scripting_guide/index.md +++ b/doc/development/shell_scripting_guide/index.md @@ -38,7 +38,7 @@ in a particular case. According to the [GitLab installation requirements](../../install/requirements.md), this guide covers only those shells that are used by -[supported Linux distributions](../../install/requirements.md#supported-linux-distributions), +[supported Linux distributions](../../administration/package_information/supported_os.md#supported-operating-systems), that is: - [POSIX Shell](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html) diff --git a/doc/development/sidekiq/compatibility_across_updates.md b/doc/development/sidekiq/compatibility_across_updates.md index 1d369b5a970..ac34d099202 100644 --- a/doc/development/sidekiq/compatibility_across_updates.md +++ b/doc/development/sidekiq/compatibility_across_updates.md @@ -18,13 +18,13 @@ several possible situations: ## Adding new workers -On GitLab.com, we -[do not currently have a Sidekiq deployment in the canary stage](https://gitlab.com/gitlab-org/gitlab/-/issues/19239). +On GitLab.com, we +[do not currently have a Sidekiq deployment in the canary stage](https://gitlab.com/gitlab-org/gitlab/-/issues/19239). This means that a new worker than can be scheduled from an HTTP endpoint may be scheduled from canary but not run on Sidekiq until the full production deployment is complete. This can be several hours later than scheduling the job. For some workers, this will not be a problem. For -others - particularly [latency-sensitive jobs](worker_attributes.md#latency-sensitive-jobs) - +others - particularly [latency-sensitive jobs](worker_attributes.md#latency-sensitive-jobs) - this will result in a poor user experience. This only applies to new worker classes when they are first introduced. diff --git a/doc/development/sidekiq/idempotent_jobs.md b/doc/development/sidekiq/idempotent_jobs.md index 5d1ebce763e..da36cdc72aa 100644 --- a/doc/development/sidekiq/idempotent_jobs.md +++ b/doc/development/sidekiq/idempotent_jobs.md @@ -78,7 +78,7 @@ GitLab supports two deduplication strategies: - `until_executing`, which is the default strategy - `until_executed` -More [deduplication strategies have been suggested](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/195). +More [deduplication strategies have been suggested](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/195). If you are implementing a worker that could benefit from a different strategy, please comment in the issue. diff --git a/doc/development/snowplow/index.md b/doc/development/snowplow/index.md index 24cd9093267..42a968b9df0 100644 --- a/doc/development/snowplow/index.md +++ b/doc/development/snowplow/index.md @@ -31,7 +31,7 @@ Snowplow tracking is enabled on GitLab.com, and we use it for most of our tracki To enable Snowplow tracking on a self-managed instance: -1. On the top bar, select **Menu > Admin**, then select **Settings > General**. +1. On the top bar, select **Main menu > Admin**, then select **Settings > General**. Alternatively, go to `admin/application_settings/general` in your browser. 1. Expand **Snowplow**. diff --git a/doc/development/snowplow/troubleshooting.md b/doc/development/snowplow/troubleshooting.md index 42a433e6a94..3ad4c6c9549 100644 --- a/doc/development/snowplow/troubleshooting.md +++ b/doc/development/snowplow/troubleshooting.md @@ -4,7 +4,13 @@ group: Product Intelligence info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Troubleshooting +# Troubleshooting Snowplow + +## Monitoring + +This page covers dashboards and alerts coming from a number of internal tools. + +For a brief video overview of the tools used to monitor Snowplow usage, please check out [this internal video](https://www.youtube.com/watch?v=NxPS0aKa_oU) (you must be logged into GitLab Unfiltered to view). ## Good events drop diff --git a/doc/development/sql.md b/doc/development/sql.md index 7101bf7fb4b..029874011c4 100644 --- a/doc/development/sql.md +++ b/doc/development/sql.md @@ -80,7 +80,7 @@ USING GIN(column_name gin_trgm_ops); ``` The key here is the `GIN(column_name gin_trgm_ops)` part. This creates a -[GIN index](https://www.postgresql.org/docs/current/gin.html) +[GIN index](https://www.postgresql.org/docs/current/gin.html) with the operator class set to `gin_trgm_ops`. These indexes _can_ be used by `ILIKE` / `LIKE` and can lead to greatly improved performance. One downside of these indexes is that they can easily get quite large (depending @@ -397,7 +397,7 @@ default. While `WHERE IN` and `WHERE EXISTS` can be used to produce the same data it is recommended to use `WHERE EXISTS` whenever possible. While in many cases -PostgreSQL can optimise `WHERE IN` quite well there are also many cases where +PostgreSQL can optimize `WHERE IN` quite well there are also many cases where `WHERE EXISTS` performs (much) better. In Rails you have to use this by creating SQL fragments: diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 79a72981e3f..221d6b89b20 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -83,6 +83,31 @@ You can silence deprecation warnings by setting the environment variable SILENCE_DEPRECATIONS=1 bin/rspec spec/models/project_spec.rb ``` +### Test order + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93137) in GitLab 15.4. + +All new spec files are run in [random order](https://gitlab.com/gitlab-org/gitlab/-/issues/337399) +to surface flaky tests that are dependent on test order. + +When randomized: + +- The string `# order random` is added below the example group description. +- The used seed is shown in the spec output below the test suite summary. For example, `Randomized with seed 27443`. + +For a list of spec files which are still run in defined order, see [`rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml). + +To make spec files run in random order, check their order dependency with: + +```shell +scripts/rspec_check_order_dependence spec/models/project_spec.rb +``` + +If the specs pass the check the script removes them from +[`rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml) automatically. + +If the specs fail the check they must be fixed before than can run in random order. + ### Test speed GitLab has a massive test suite that, without [parallelization](../pipelines.md#test-suite-parallelization), can take hours @@ -232,7 +257,7 @@ Here is an example of when `let_it_be` cannot be used, but `let_it_be_with_reloa ```ruby let_it_be(:user) { create(:user) } -let_it_be_with_reload(:project) { create(:project) } # The test will fail if `let_it_be` is used +let_it_be_with_reload(:project) { create(:project) } # The test will fail if `let_it_be` is used context 'with a developer' do before do @@ -422,7 +447,7 @@ Use the coverage reports to ensure your tests cover 100% of your code. ### System / Feature tests NOTE: -Before writing a new system test, +Before writing a new system test, [please consider **not** writing one](testing_levels.md#consider-not-writing-a-system-test)! - Feature specs should be named `ROLE_ACTION_spec.rb`, such as @@ -711,6 +736,7 @@ should either: - Add `require_dependency 're2'` to files in your library that need `re2` gem, to make this requirement explicit. This approach is preferred. - Add it to the spec itself. +- Use `rubocop_spec_helper` for RuboCop related specs. It takes around one second to load tests that are using `fast_spec_helper` instead of 30+ seconds in case of a regular `spec_helper`. @@ -909,7 +935,7 @@ By default, Sidekiq jobs are enqueued into a jobs array and aren't processed. If a test queues Sidekiq jobs and need them to be processed, the `:sidekiq_inline` trait can be used. -The `:sidekiq_might_not_need_inline` trait was added when +The `:sidekiq_might_not_need_inline` trait was added when [Sidekiq inline mode was changed to fake mode](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/15479) to all the tests that needed Sidekiq to actually process jobs. Tests with this trait should be either fixed to not rely on Sidekiq processing jobs, or their @@ -1048,7 +1074,7 @@ Most tests for Elasticsearch logic relate to: There are some exceptions, such as checking for structural changes rather than individual records in an index. -The `:elastic_with_delete_by_query` trait was added to reduce run time for pipelines by creating and deleting indices +The `:elastic_delete_by_query` trait was added to reduce run time for pipelines by creating and deleting indices at the start and end of each context only. The [Elasticsearch DeleteByQuery API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html) is used to delete data in all indices in between examples to ensure a clean index. @@ -1081,12 +1107,11 @@ Snowplow performs **runtime type checks** by using the [contracts gem](https://r Because Snowplow is **by default disabled in tests and development**, it can be hard to **catch exceptions** when mocking `Gitlab::Tracking`. -To catch runtime errors due to type checks, you can enable Snowplow in tests. Mark the spec with -`:snowplow` and use the `expect_snowplow_event` helper, which checks for +To catch runtime errors due to type checks you can use `expect_snowplow_event`, which checks for calls to `Gitlab::Tracking#event`. ```ruby -describe '#show', :snowplow do +describe '#show' do it 'tracks snowplow events' do get :show @@ -1111,7 +1136,7 @@ end When you want to ensure that no event got called, you can use `expect_no_snowplow_event`. ```ruby - describe '#show', :snowplow do + describe '#show' do it 'does not track any snowplow events' do get :show @@ -1357,6 +1382,47 @@ RSpec.configure do |config| end ``` +### Testing Ruby constants + +When testing code that uses Ruby constants, focus the test on the behavior that depends on the constant, +rather than testing the values of the constant. + +For example, the following is preferred because it tests the behavior of the class method `.categories`. + +```ruby + describe '.categories' do + it 'gets CE unique category names' do + expect(described_class.categories).to include( + 'deploy_token_packages', + 'user_packages', + # ... + 'kubernetes_agent' + ) + end + end +``` + +On the other hand, testing the value of the constant itself, often only repeats the values +in the code and the test, which provides little value. + +```ruby + describe CATEGORIES do + it 'has values' do + expect(CATEGORIES).to eq([ + 'deploy_token_packages', + 'user_packages', + # ... + 'kubernetes_agent' + ]) + end +end +``` + +In critical cases where an error on a constant could have a catastrophic impact, +testing the constant values might be useful as an added safeguard. For example, +if it could bring down the entire GitLab service, cause a customer to be billed more than they should be, +or [cause the universe to implode](../contributing/verify/index.md#do-not-cause-our-universe-to-implode). + ### Factories GitLab uses [factory_bot](https://github.com/thoughtbot/factory_bot) as a test fixture replacement. diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md index bfda94b1f1d..b17ca9e6f8f 100644 --- a/doc/development/testing_guide/end_to_end/best_practices.md +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -62,13 +62,13 @@ In those and similar cases we need to include the test case link by other means. To illustrate, there are two tests in the shared examples in [`qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb): ```ruby -shared_examples 'unselected maintainer' do |testcase| +RSpec.shared_examples 'unselected maintainer' do |testcase| it 'user fails to push', testcase: testcase do ... end end -shared_examples 'selected developer' do |testcase| +RSpec.shared_examples 'selected developer' do |testcase| it 'user pushes and merges', testcase: testcase do ... end @@ -415,7 +415,7 @@ except(page).to have_no_text('hidden') Unfortunately, that's not automatically the case for the predicate methods that we add to our [page objects](page_objects.md). We need to [create our own negatable matchers](https://relishapp.com/rspec/rspec-expectations/v/3-9/docs/custom-matchers/define-a-custom-matcher#matcher-with-separate-logic-for-expect().to-and-expect().not-to). -The initial example uses the `have_job` matcher which is derived from the +The initial example uses the `have_job` matcher which is derived from the [`has_job?` predicate method of the `Page::Project::Pipeline::Show` page object](https://gitlab.com/gitlab-org/gitlab/-/blob/87864b3047c23b4308f59c27a3757045944af447/qa/qa/page/project/pipeline/show.rb#L53). To create a negatable matcher, we use `has_no_job?` for the negative case: diff --git a/doc/development/testing_guide/end_to_end/feature_flags.md b/doc/development/testing_guide/end_to_end/feature_flags.md index 33f73304a26..9a39161f1ad 100644 --- a/doc/development/testing_guide/end_to_end/feature_flags.md +++ b/doc/development/testing_guide/end_to_end/feature_flags.md @@ -195,12 +195,10 @@ End-to-end tests should pass with a feature flag enabled before it is enabled on There are two ways to confirm that end-to-end tests pass: - If a merge request adds or edits a [feature flag definition file](../../feature_flags/index.md#feature-flag-definition-and-validation), - two `package-and-qa` jobs (`package-and-qa-ff-enabled` and `package-and-qa-ff-disabled`) are included automatically in the merge request - pipeline. One job enables the defined feature flag and the other job disables it. The jobs execute the same suite of tests to confirm - that they pass with the feature flag either enabled or disabled. -- In some cases, if `package-and-qa` hasn't been triggered automatically, or if it has run the tests with the default feature flag values - (which might not be desired), you can create a Draft MR that enables the feature flag to ensure that all E2E tests pass with the feature - flag enabled. + two `e2e:package-and-test` jobs (`ee:instance-parallel` and `ee:instance-parallel-ff-inverse`) are included automatically in the merge request pipeline. + One job runs the application with default feature flag state and another sets it to inverse value. The jobs execute the same suite of tests to confirm that they pass with the feature flag either enabled or disabled. +- In some cases, if end-to-end test jobs didn't trigger automatically, or if it has run the tests with the default feature flag values (which might not be desired), + you can create a Draft MR that enables the feature flag to ensure that all E2E tests pass with the feature flag enabled and disabled. ### Troubleshooting end-to-end test failures with feature flag enabled @@ -216,8 +214,8 @@ If enabling the feature flag results in E2E test failures, you can browse the ar ### Test execution during feature development If an end-to-end test enables a feature flag, the end-to-end test suite can be used to test changes in a merge request -by running the `package-and-qa` job in the merge request pipeline. If the feature flag and relevant changes have already been merged, you can confirm that the tests -pass on the default branch. The end-to-end tests run on the default branch every two hours, and the results are posted to a +by running the `e2e:package-and-test` job in the merge request pipeline. If the feature flag and relevant changes have already been merged, you can confirm that the tests +pass on the default branch. The end-to-end tests run on the default branch every two hours, and the results are posted to a [Test Session Report, which is available in the testcase-sessions project](https://gitlab.com/gitlab-org/quality/testcase-sessions/-/issues?label_name%5B%5D=found%3Amain). If the relevant tests do not enable the feature flag themselves, you can check if the tests will need to be updated by opening diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index 989d090d581..cc9c02a65ce 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -8,16 +8,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## What is end-to-end testing? -End-to-end testing is a strategy used to check whether your application works +End-to-end (e2e) testing is a strategy used to check whether your application works as expected across the entire software stack and architecture, including integration of all micro-services and components that are supposed to work together. ## How do we test GitLab? -We use [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab) to build GitLab packages and then we -test these packages using the [GitLab QA orchestrator](https://gitlab.com/gitlab-org/gitlab-qa) tool, which is -a black-box testing framework for the API and the UI. +We use [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab) to build GitLab packages and then we test these packages +using the [GitLab QA orchestrator](https://gitlab.com/gitlab-org/gitlab-qa) tool to run the end-to-end tests located in the `qa` directory. ### Testing nightly builds @@ -33,11 +32,9 @@ You can find these pipelines at <https://gitlab.com/gitlab-org/quality/staging/p ### Testing code in merge requests -#### Using the `package-and-qa` job +#### Using the package-and-test job -It is possible to run end-to-end tests for a merge request, eventually being run in -a pipeline in the [`gitlab-org/gitlab-qa-mirror`](https://gitlab.com/gitlab-org/gitlab-qa-mirror) project, -by triggering the `package-and-qa` manual action in the `qa` stage (not +It is possible to run end-to-end tests for a merge request by triggering the `e2e:package-and-test` manual action in the `qa` stage (not available for forks). **This runs end-to-end tests against a custom EE (with an Ultimate license) @@ -59,7 +56,7 @@ graph TB subgraph "`gitlab-org/gitlab` pipeline" A1[`build-images` stage<br>`build-qa-image` and `build-assets-image` jobs] - A2[`qa` stage<br>`package-and-qa` job] + A2[`qa` stage<br>`e2e:package-and-test` job] end subgraph "`gitlab-org/build/omnibus-gitlab-mirror` pipeline" @@ -72,38 +69,30 @@ subgraph "`gitlab-org/gitlab-qa-mirror` pipeline" ``` 1. In the [`gitlab-org/gitlab` pipeline](https://gitlab.com/gitlab-org/gitlab): - 1. Developer triggers the `package-and-qa` manual action (available once the `build-qa-image` and + 1. Developer triggers the `e2e:package-and-test` manual action (available once the `build-qa-image` and `build-assets-image` jobs are done), that can be found in GitLab merge - requests. This starts a chain of pipelines in multiple projects. - 1. The script being executed triggers a pipeline in + requests. This starts a e2e test child pipeline. + 1. E2E child pipeline triggers a downstream pipeline in [`gitlab-org/build/omnibus-gitlab-mirror`](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror) and polls for the resulting status. We call this a _status attribution_. 1. In the [`gitlab-org/build/omnibus-gitlab-mirror` pipeline](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror): 1. Docker image is being built and pushed to its Container Registry. - 1. Finally, the `Trigger:qa-test` job triggers a new end-to-end pipeline in - [`gitlab-org/gitlab-qa-mirror`](https://gitlab.com/gitlab-org/gitlab-qa-mirror/pipelines) and polls for the resulting status. + 1. Once Docker images are built and pushed jobs in `test` stage are started -1. In the [`gitlab-org/gitlab-qa-mirror` pipeline](https://gitlab.com/gitlab-org/gitlab-qa-mirror): +1. In the `Test` stage: 1. Container for the Docker image stored in the [`gitlab-org/build/omnibus-gitlab-mirror`](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror) registry is spun-up. 1. End-to-end tests are run with the `gitlab-qa` executable, which spin up a container for the end-to-end image from the [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) registry. -1. The result of the [`gitlab-org/gitlab-qa-mirror` pipeline](https://gitlab.com/gitlab-org/gitlab-qa-mirror) is being - propagated upstream (through polling from upstream pipelines), through [`gitlab-org/build/omnibus-gitlab-mirror`](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror), back to the [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) merge request. - -We plan to [add more specific information](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/156) -about the tests included in each job/scenario that runs in `gitlab-org/gitlab-qa-mirror`. - NOTE: You may have noticed that we use `gitlab-org/build/omnibus-gitlab-mirror` instead of -`gitlab-org/omnibus-gitlab`, and `gitlab-org/gitlab-qa-mirror` instead of `gitlab-org/gitlab-qa`. +`gitlab-org/omnibus-gitlab`. This is due to technical limitations in the GitLab permission model: the ability to run a pipeline against a protected branch is controlled by the ability to push/merge to this branch. This means that for developers to be able to trigger a pipeline for the default branch in -`gitlab-org/omnibus-gitlab`/`gitlab-org/gitlab-qa`, they would need to have the -Maintainer role for those projects. +`gitlab-org/omnibus-gitlab`, they would need to have the Maintainer role for this project. For security reasons and implications, we couldn't open up the default branch to all the Developers. -Hence we created these mirrors where Developers and Maintainers are allowed to push/merge to the default branch. +Hence we created this mirror where Developers and Maintainers are allowed to push/merge to the default branch. This problem was discovered in <https://gitlab.com/gitlab-org/gitlab-qa/-/issues/63#note_107175160> and the "mirror" work-around was suggested in <https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4717>. A feature proposal to segregate access control regarding running pipelines from ability to push/merge was also created at <https://gitlab.com/gitlab-org/gitlab/-/issues/24585>. @@ -111,22 +100,19 @@ A feature proposal to segregate access control regarding running pipelines from #### With merged results pipelines In a merged results pipeline, the pipeline runs on a new ref that contains the merge result of the source and target branch. -However, this ref is not available to the `gitlab-qa-mirror` pipeline. -For this reason, the end-to-end tests on a merged results pipeline would use the head of the merge request source branch. +The end-to-end tests on a merged results pipeline would use the new ref instead of the head of the merge request source branch. ```mermaid graph LR -A["a1b1c1 - branch HEAD (CI_MERGE_REQUEST_SOURCE_BRANCH_SHA)"] -B["x1y1z1 - master HEAD"] -C["d1e1f1 - merged results (CI_COMMIT_SHA)"] +A["x1y1z1 - master HEAD"] +B["d1e1f1 - merged results (CI_COMMIT_SHA)"] -A --> C -B --> C +A --> B -A --> E["E2E tests"] -C --> D["Merged results pipeline"] +B --> C["Merged results pipeline"] +C --> D["E2E tests"] ``` ##### Running custom tests @@ -140,7 +126,7 @@ a flaky test we first want to make sure that it's no longer flaky. We can do that using the `ce:custom-parallel` and `ee:custom-parallel` jobs. Both are manual jobs that you can configure using custom variables. When clicking the name (not the play icon) of one of the parallel jobs, -you are prompted to enter variables. You can use any of +you are prompted to enter variables. You can use any of [the variables that can be used with `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-gitlab-environment-variables) as well as these: @@ -150,8 +136,8 @@ as well as these: | `QA_TESTS` | The tests to run (no default, which means run all the tests in the scenario). Use file paths as you would when running tests via RSpec, for example, `qa/specs/features/ee/browser_ui` would include all the `EE` UI tests. | | `QA_RSPEC_TAGS` | The RSpec tags to add (no default) | -For now, -[manual jobs with custom variables don't use the same variable when retried](https://gitlab.com/gitlab-org/gitlab/-/issues/31367), +For now, +[manual jobs with custom variables don't use the same variable when retried](https://gitlab.com/gitlab-org/gitlab/-/issues/31367), so if you want to run the same tests multiple times, specify the same variables in each `custom-parallel` job (up to as many of the 10 available jobs that you want to run). @@ -165,7 +151,7 @@ automatically started: it runs the QA smoke suite against the You can also manually start the `review-qa-all`: it runs the full QA suite against the [Review App](../review_apps.md). -**This runs end-to-end tests against a Review App based on +**This runs end-to-end tests against a Review App based on [the official GitLab Helm chart](https://gitlab.com/gitlab-org/charts/gitlab/), itself deployed with custom [Cloud Native components](https://gitlab.com/gitlab-org/build/CNG) built from your merge request's changes.** @@ -197,7 +183,7 @@ Use these environment variables to configure metrics export: | -------- | -------- | ----------- | | `QA_INFLUXDB_URL` | `true` | Should be set to `https://influxdb.quality.gitlab.net`. No default value. | | `QA_INFLUXDB_TOKEN` | `true` | InfluxDB write token that can be found under `Influxdb auth tokens` document in `Gitlab-QA` `1Password` vault. No default value. | -| `QA_RUN_TYPE` | `false` | Arbitrary name for test execution, like `package-and-qa`. Automatically inferred from the project name for live environment test executions. No default value. | +| `QA_RUN_TYPE` | `false` | Arbitrary name for test execution, like `package-and-test`. Automatically inferred from the project name for live environment test executions. No default value. | | `QA_EXPORT_TEST_METRICS` | `false` | Flag to enable or disable metrics export. Defaults to `true`. | ## Test reports @@ -231,7 +217,7 @@ a link to the current test report. Each type of scheduled pipeline generates a static link for the latest test report according to its stage: -- [`master`](https://storage.googleapis.com/gitlab-qa-allure-reports/package-and-qa/master/index.html) +- [`master`](https://storage.googleapis.com/gitlab-qa-allure-reports/e2e-package-and-test/master/index.html) - [`staging-full`](https://storage.googleapis.com/gitlab-qa-allure-reports/staging-full/master/index.html) - [`staging-sanity`](https://storage.googleapis.com/gitlab-qa-allure-reports/staging-sanity/master/index.html) - [`staging-sanity-no-admin`](https://storage.googleapis.com/gitlab-qa-allure-reports/staging-sanity-no-admin/master/index.html) @@ -244,7 +230,7 @@ Each type of scheduled pipeline generates a static link for the latest test repo If you are not [testing code in a merge request](#testing-code-in-merge-requests), there are two main options for running the tests. If you want to run the existing tests against a live GitLab instance or against a pre-built Docker image, -use the [GitLab QA orchestrator](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md). See also +use the [GitLab QA orchestrator](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md). See also [examples of the test scenarios you can run via the orchestrator](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#examples). On the other hand, if you would like to run against a local development GitLab @@ -263,7 +249,7 @@ architecture. See the [documentation about it](https://gitlab.com/gitlab-org/git Once you decided where to put [test environment orchestration scenarios](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/lib/gitlab/qa/scenario) and [instance-level scenarios](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features), take a look at the [GitLab QA README](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa/README.md), -the [GitLab QA orchestrator README](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md), +the [GitLab QA orchestrator README](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md), and [the already existing instance-level scenarios](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features). ### Consider **not** writing an end-to-end test diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md index 322f2412e5b..1abaf3ef323 100644 --- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md +++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md @@ -15,27 +15,28 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec |-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `:elasticsearch` | The test requires an Elasticsearch service. It is used by the [instance-level scenario](https://gitlab.com/gitlab-org/gitlab-qa#definitions) [`Test::Integration::Elasticsearch`](https://gitlab.com/gitlab-org/gitlab/-/blob/72b62b51bdf513e2936301cb6c7c91ec27c35b4d/qa/qa/ee/scenario/test/integration/elasticsearch.rb) to include only tests that require Elasticsearch. | | `:except` | The test is to be run in their typical execution contexts _except_ as specified. See [test execution context selection](execution_context_selection.md) for more information. | -| `:feature_flag` | The test uses a feature flag and therefore requires an administrator account to run. When `scope` is set to `:global`, the test will be skipped on all live .com environments. Otherwise, it will be skipped only on Canary, Production, and Preprod. See [testing with feature flags](../../../development/testing_guide/end_to_end/feature_flags.md) for more details. | +| `:feature_flag` | The test uses a feature flag and therefore requires an administrator account to run. When `scope` is set to `:global`, the test will be skipped on all live .com environments. Otherwise, it will be skipped only on Canary, Production, and Preprod. See [testing with feature flags](../../../development/testing_guide/end_to_end/feature_flags.md) for more details. | | `:geo` | The test requires two GitLab Geo instances - a primary and a secondary - to be spun up. | | `:gitaly_cluster` | The test runs against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. | | `:github` | The test requires a GitHub personal access token. | | `:group_saml` | The test requires a GitLab instance that has SAML SSO enabled at the group level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. | | `:instance_saml` | The test requires a GitLab instance that has SAML SSO enabled at the instance level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. | -| `:integrations` | This aims to test the available [integrations](../../../user/project/integrations/index.md#available-integrations). The test requires Docker to be installed in the run context. It will provision the containers and can be run against a local instance or using the `gitlab-qa` scenario `Test::Integration::Integrations` | +| `:integrations` | This aims to test the available [integrations](../../../user/project/integrations/index.md#available-integrations). The test requires Docker to be installed in the run context. It will provision the containers and can be run against a local instance or using the `gitlab-qa` scenario `Test::Integration::Integrations` | | `:issue`, `:issue_${num}` | Optional links to issues which might be related to the spec. Helps keep track of related issues and can also be used by tools that create test reports. Currently added automatically to `Allure` test report. Multiple tags can be used by adding an optional numeric suffix like `issue_1`, `issue_2` etc. | -| `:service_ping_disabled` | The test interacts with the GitLab configuration service ping at the instance level to turn Admin Area setting service ping checkbox on or off. This tag will have the test run only in the `service_ping_disabled` job and must be paired with the `:orchestrated` and `:requires_admin` tags. | +| `:service_ping_disabled` | The test interacts with the GitLab configuration service ping at the instance level to turn Admin Area setting service ping checkbox on or off. This tag will have the test run only in the `service_ping_disabled` job and must be paired with the `:orchestrated` and `:requires_admin` tags. | | `:jira` | The test requires a Jira Server. [GitLab-QA](https://gitlab.com/gitlab-org/gitlab-qa) provisions the Jira Server in a Docker container when the `Test::Integration::Jira` test scenario is run. | | `:kubernetes` | The test includes a GitLab instance that is configured to be run behind an SSH tunnel, allowing a TLS-accessible GitLab. This test also includes provisioning of at least one Kubernetes cluster to test against. _This tag is often be paired with `:orchestrated`._ | | `:ldap_no_server` | The test requires a GitLab instance to be configured to use LDAP. To be used with the `:orchestrated` tag. It does not spin up an LDAP server at orchestration time. Instead, it creates the LDAP server at runtime. | | `:ldap_no_tls` | The test requires a GitLab instance to be configured to use an external LDAP server with TLS not enabled. | | `:ldap_tls` | The test requires a GitLab instance to be configured to use an external LDAP server with TLS enabled. | | `:mattermost` | The test requires a GitLab Mattermost service on the GitLab instance. | -| `:metrics` | The test requires a GitLab instance where [dedicated metrics exporters](../../../administration/monitoring/prometheus/web_exporter.md) are running alongside Puma and Sidekiq. | +| `:metrics` | The test requires a GitLab instance where [dedicated metrics exporters](../../../administration/monitoring/prometheus/web_exporter.md) are running alongside Puma and Sidekiq. | | `:mixed_env` | The test should only be executed in environments that have a paired canary version available through traffic routing based on the existence of the `gitlab_canary=true` cookie. Tests in this category are switching the cookie mid-test to validate mixed deployment environments. | | `:object_storage` | The test requires a GitLab instance to be configured to use multiple [object storage types](../../../administration/object_storage.md). Uses MinIO as the object storage server. | | `:only` | The test is only to be run in specific execution contexts. See [test execution context selection](execution_context_selection.md) for more information. | | `:orchestrated` | The GitLab instance under test may be [configured by `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#orchestrated-tests) to be different to the default GitLab configuration, or `gitlab-qa` may launch additional services in separate Docker containers, or both. Tests tagged with `:orchestrated` are excluded when testing environments where we can't dynamically modify the GitLab configuration (for example, Staging). | -| `:packages` | The test requires a GitLab instance that has the [Package Registry](../../../administration/packages/#gitlab-package-registry-administration) enabled. | +| `:packages` | The test requires a GitLab instance that has the [Package Registry](../../../administration/packages/index.md#gitlab-package-registry-administration) enabled. | +| `:product_group` | Specifies what product group the test belongs to. See [Product sections, stages, groups, and categories](https://about.gitlab.com/handbook/product/categories) for the comprehensive groups list. | | `:quarantine` | The test has been [quarantined](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/debugging-qa-test-failures/#quarantining-tests), runs in a separate job that only includes quarantined tests, and is allowed to fail. The test is skipped in its regular job so that if it fails it doesn't hold up the pipeline. Note that you can also [quarantine a test only when it runs in a specific context](execution_context_selection.md#quarantine-a-test-for-a-specific-environment). | | `:relative_url` | The test requires a GitLab instance to be installed under a [relative URL](../../../install/relative_url.md). | | `:reliable` | The test has been [promoted to a reliable test](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/reliable-tests/#promoting-an-existing-test-to-reliable) meaning it passes consistently in all pipelines, including merge requests. | @@ -44,7 +45,7 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec | `:requires_git_protocol_v2` | The test requires that Git protocol version 2 is enabled on the server. It's assumed to be enabled by default but if not the test can be skipped by setting `QA_CAN_TEST_GIT_PROTOCOL_V2` to `false`. | | `:requires_praefect` | The test requires that the GitLab instance uses [Gitaly Cluster](../../../administration/gitaly/praefect.md) (a.k.a. Praefect) as the repository storage . It's assumed to be used by default but if not the test can be skipped by setting `QA_CAN_TEST_PRAEFECT` to `false`. | | `:runner` | The test depends on and sets up a GitLab Runner instance, typically to run a pipeline. | -| `:sanity_feature_flags` | The test verifies the functioning of the feature flag handling part of the test framework | +| `:sanity_feature_flags` | The test verifies the functioning of the feature flag handling part of the test framework | | `:skip_live_env` | The test is excluded when run against live deployed environments such as Staging, Canary, and Production. | | `:skip_fips_env` | The test is excluded when run against an environment in FIPS mode. | | `:skip_signup_disabled` | The test uses UI to sign up a new user and is skipped in any environment that does not allow new user registration via the UI. | diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md index 3c8df7d3416..7409f15969d 100644 --- a/doc/development/testing_guide/flaky_tests.md +++ b/doc/development/testing_guide/flaky_tests.md @@ -123,7 +123,7 @@ reproduction. ### Hanging specs -If a spec hangs, it might be caused by a [bug in Rails](https://github.com/rails/rails/issues/34310): +If a spec hangs, it might be caused by a [bug in Rails](https://github.com/rails/rails/issues/45994): - <https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81112> - <https://gitlab.com/gitlab-org/gitlab/-/issues/337039> diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 2845dde9a24..22a8792bac6 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -420,7 +420,11 @@ it('passes', () => { ### Waiting in tests Sometimes a test needs to wait for something to happen in the application before it continues. -Avoid using [`setTimeout`](https://developer.mozilla.org/en-US/docs/Web/API/setTimeout) because it makes the reason for waiting unclear. Instead use one of the following approaches. + +You should try to avoid: + +- [`setTimeout`](https://developer.mozilla.org/en-US/docs/Web/API/setTimeout) because it makes the reason for waiting unclear. Additionally, it is faked in our tests so its usage is tricky. +- [`setImmediate`](https://developer.mozilla.org/en-US/docs/Web/API/Window/setImmediate) because it is no longer supported in Jest 27 and later. See [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7002) for details. #### Promises and Ajax calls @@ -448,14 +452,15 @@ it('waits for an Ajax call', async () => { }); ``` -If you are not able to register handlers to the `Promise`, for example because it is executed in a synchronous Vue life cycle hook, take a look at the [waitFor](#wait-until-axios-requests-finish) helpers or you can flush all pending `Promise`s: +If you cannot register handlers to the `Promise`, for example because it is executed in a synchronous Vue lifecycle hook, take a look at the [`waitFor`](#wait-until-axios-requests-finish) helpers or flush all pending `Promise`s with: **in Jest:** ```javascript -it('waits for an Ajax call', () => { +it('waits for an Ajax call', async () => { synchronousFunction(); - jest.runAllTicks(); + + await waitForPromises(); expect(something).toBe('done'); }); @@ -872,7 +877,7 @@ This will create a new fixture located at `tmp/tests/frontend/fixtures-ee/graphql/releases/graphql/queries/all_releases.query.graphql.json`. You can import the JSON fixture in a Jest test using the `test_fixtures` alias -[as described below](#use-fixtures). +[as described previously](#use-fixtures). ## Data-driven tests @@ -1073,7 +1078,7 @@ testAction( <!-- vale gitlab.Spelling = NO --> -The Axios Utils mock module located in `spec/frontend/mocks/ce/lib/utils/axios_utils.js` contains two helper methods for Jest tests that spawn HTTP requests. +The Axios Utils mock module located in `spec/frontend/__helpers__/mocks/axios_utils.js` contains two helper methods for Jest tests that spawn HTTP requests. These are very useful if you don't have a handle to the request's Promise, for example when a Vue component does a request as part of its life cycle. <!-- vale gitlab.Spelling = YES --> diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index cd7c70e2eaa..e50902c4995 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w This document describes various guidelines and best practices for automated testing of the GitLab project. -It is meant to be an _extension_ of the +It is meant to be an _extension_ of the [Thoughtbot testing style guide](https://github.com/thoughtbot/guides/tree/master/testing-rspec). If this guide defines a rule that contradicts the Thoughtbot guide, this guide takes precedence. Some guidelines may be repeated verbatim to stress their diff --git a/doc/development/testing_guide/testing_migrations_guide.md b/doc/development/testing_guide/testing_migrations_guide.md index 261a4f4a27e..3006a2230ac 100644 --- a/doc/development/testing_guide/testing_migrations_guide.md +++ b/doc/development/testing_guide/testing_migrations_guide.md @@ -317,7 +317,7 @@ To test these you usually have to: - Verify that the expected jobs were scheduled, with the correct set of records, the correct batch size, interval, etc. -The behavior of the background migration itself needs to be verified in a +The behavior of the background migration itself needs to be verified in a [separate test for the background migration class](#example-background-migration-test). This spec tests the diff --git a/doc/development/value_stream_analytics.md b/doc/development/value_stream_analytics.md index 0d545fa8e3f..bbea89d5645 100644 --- a/doc/development/value_stream_analytics.md +++ b/doc/development/value_stream_analytics.md @@ -6,33 +6,86 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Value stream analytics development guide -Value stream analytics calculates the time between two arbitrary events recorded on domain objects and provides aggregated statistics about the duration. +For information on how to configure value stream analytics (VSA) in GitLab, see our [analytics documentation](../user/analytics/value_stream_analytics.md). -For information on how to configure value stream analytics in GitLab, see our [analytics documentation](../user/analytics/value_stream_analytics.md). +## How does Value Stream Analytics work? -## Stage +Value Stream Analytics calculates the duration between two timestamp columns or timestamp +expressions and runs various aggregations on the data. -During development, events occur that move issues and merge requests through different stages of progress until they are considered finished. These stages can be expressed with the `Stage` model. +For example: -Example stage: +- Duration between the Merge Request creation time and Merge Request merge time. +- Duration between the Issue creation time and Issue close time. -- Name: Development -- Start event: Issue created -- End event: Issue first mentioned in commit -- Parent: `Group: gitlab-org` +This duration is exposed in various ways: + +- Aggregation: median, average +- Listing: list the duration for individual Merge Request and Issue records + +Apart from the durations, we expose the record count within a stage. + +## Feature availability + +- Group level (licensed): Requires Ultimate or Premium subscription. This version is the most +feature-full. +- Project level (licensed): We are continually adding features to project level VSA to bring it in line with group level VSA. +- Project level (FOSS): Keep it as is. + +|Feature|Group level (licensed)|Project level (licensed)|Project level (FOSS)| +|-|-|-|-| +|Create custom value streams|Yes|No, only one value stream (default) is present with the default stages|no, only one value stream (default) is present with the default stages| +|Create custom stages|Yes|No|No| +|Filtering (author, label, milestone, etc.)|Yes|Yes|Yes| +|Stage time chart|Yes|No|No| +|Total time chart|Yes|No|No| +|Task by type chart|Yes|No|No| +|DORA Metrics|Yes|Yes|No| +|Cycle time and lead time summary (Key metrics)|Yes|Yes|No| +|New issues, commits and deploys (Key metrics)|Yes, excluding commits|Yes|Yes| +|Uses aggregated backend|Yes|No|No| +|Date filter behavior|Filters items [finished within the date range](https://gitlab.com/groups/gitlab-org/-/epics/6046)|Filters items by creation date.|Filters items by creation date.| +|Authorization|At least reporter|At least reporter|Can be public.| + +## VSA core domain objects + +### Stages + +A stage represents an event pair (start and end events) with additional metadata, such as the name +of the stage. Stages are configurable by the user within the pairing rules defined in the backend. + +**Example stage: Code Review** + +- Start event identifier: Merge request creation time. +- Start event column: uses the `merge_requests.created_at` timestamp column. +- End event identifier: Merge request merge time. +- End event column: uses the `merge_request_metrics.merged_at` timestamp column. +- Stage event hash ID: a calculated hash for the pair of start and end event identifiers. + - If two stages have the same configuration of start and end events, then their stage event hash. + IDs are identical. + - The stage event hash ID is later used to store the aggregated data in partitioned database tables. + +Historically, value stream analytics defined [7 stages](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/analytics/cycle_analytics/default_stages.rb) +which are always available to the end-users regardless of the subscription. + +### Value streams + +Value streams are container objects for the stages. There can be multiple value streams per group +focusing on different aspects of the Dev Ops lifecycle. ### Events Events are the smallest building blocks of the value stream analytics feature. A stage consists of two events: -- Start -- End +- Start event +- End event These events play a key role in the duration calculation. Formula: `duration = end_event_time - start_event_time` -To make the duration calculation flexible, each `Event` is implemented as a separate class. They're responsible for defining a timestamp expression that is used in the calculation query. +To make the duration calculation flexible, each `Event` is implemented as a separate class. +They're responsible for defining a timestamp expression that is used in the calculation query. #### Implementing an `Event` class @@ -81,7 +134,7 @@ def apply_query_customization(query) end ``` -### Validating start and end events +#### Validating start and end events Some start/end event pairs are not "compatible" with each other. For example: @@ -171,23 +224,7 @@ graph LR; MergeRequestLabelRemoved --> MergeRequestLabelRemoved; ``` -### Parent - -Teams and organizations might define their own way of building software, thus stages can be completely different. For each stage, a parent object needs to be defined. - -Currently supported parents: - -- `Project` -- `Group` - -#### How parent relationship it work - -1. User navigates to the value stream analytics page. -1. User selects a group. -1. Backend loads the defined stages for the selected group. -1. Additions and modifications to the stages are persisted within the selected group only. - -### Default stages +## Default stages The [original implementation](https://gitlab.com/gitlab-org/gitlab/-/issues/847) of value stream analytics defined 7 stages. These stages are always available for each parent, however altering these stages is not possible. @@ -209,31 +246,15 @@ The reason for this was that we'd like to add the abilities to hide and order st For a new calculation or a query, implement it as a new method call in the `DataCollector` class. -## Database query - -Structure of the database query: - -```sql -SELECT (customized by: Median or RecordsFetcher or DataForDurationChart) -FROM OBJECT_TYPE (Issue or MergeRequest) -INNER JOIN (several JOIN statements, depending on the events) -WHERE - (Filter by the PARENT model, example: filter Issues from Project A) - (Date range filter based on the OBJECT_TYPE.created_at) - (Check if the START_EVENT is earlier than END_EVENT, preventing negative duration) -``` - -Structure of the `SELECT` statement for `Median`: +To support the aggregated value stream analytics backend, these classes were reimplemented within [`Aggregated`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/analytics/cycle_analytics/aggregated) namespace. -```sql -SELECT (calculate median from START_EVENT_TIME-END_EVENT_TIME) -``` +### Database query backend -Structure of the `SELECT` statement for `DataForDurationChart`: +VSA supports two backends: [aggregated](value_stream_analytics/value_stream_analytics_aggregated_backend.md) and "live". The live query backend can be +considered legacy, which will be phased out at some point. -```sql -SELECT (START_EVENT_TIME-END_EVENT_TIME) as duration, END_EVENT.timestamp -``` +- "live": uses the standard `IssuableFinders`. +- aggregated: queries data from pre-aggregated database tables. ## High-level overview @@ -244,6 +265,31 @@ SELECT (START_EVENT_TIME-END_EVENT_TIME) as duration, END_EVENT.timestamp - Responsible for composing queries and define feature specific business logic. - `DataCollector`, `Event`, `StageEvents`, etc. +## Frontend + +[Project VSA](../user/analytics/value_stream_analytics.md) is available for all users and: + +- Includes a mixture of key and DORA metrics based on the tier. +- Uses the set of [default stages](#default-stages). + +[Group VSA](../user/group/value_stream_analytics/index.md) is only available for licensed users and extends project VSA to include: + +- An [overview stage](https://gitlab.com/gitlab-org/gitlab/-/issues/321438). +- The ability to create custom value streams. + +The group and project level VSA frontends are both built with Vue and Vuex and follow a similar pattern: + +- The `index.js` file extracts any URL query parameters, creates the Vue app and Vuex store, and dispatches an `initialize` Vuex action. +- The `base.vue` file is used to render the main components for each page, metrics, filters, charts, and the stage table. + +The group VSA Vuex store makes use of [Vuex modules](https://vuex.vuejs.org/guide/modules.html) to separate some of the state and logic used for rendering the charts. + +### Shared components + +Parts of the UI are shared between project VSA and group VSA such as the stage table and path. These shared components live in the project VSA directory `app/assets/javascripts/cycle_analytics/components` and are included at the group level VSA where needed. + +All the frontend code for group-level features are located in `ee/app/assets/javascripts/analytics/cycle_analytics/components`. + ## Testing Since we have a lots of events and possible pairings, testing each pairing is not possible. The rule is to have at least one test case using an `Event` class. @@ -252,3 +298,31 @@ Writing a test case for a stage using a new `Event` can be challenging since dat - Different parents: `Group` or `Project` - Different calculations: `Median`, `RecordsFetcher` or `DataForDurationChart` + +The VSA frontend is tested extensively on two different levels (integration, unit): + +- End-to-end integration tests using a real backend via Capybara and RSpec. +- Jest frontend tests with pre-generated data fixtures. + +## Development setup and testing + +Running Value Stream Analytics can be done via the [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit). By default, you'll be able to view the project-level (FOSS) version of the feature. + +If your GDK is up and running, you can run the seed script to generate some data: + +```shell +SEED_CYCLE_ANALYTICS=true SEED_VSA=true FILTER=cycle_analytics rake db:seed_fu +``` + +The data generator script creates a new group and a new project with issue and merge request +data (see the output of the script). To view the group-level version of the feature, you +need to request a license for your GDK instance. + +After this step, you can access the group level value stream analytics page where you can create +value streams and stages. The data aggregation might be delayed so you might not see the +data right after the stage creation. To speed up this process, you can run the following command +in your rails console (`rails c`): + +```ruby +Analytics::CycleAnalytics::ReaggregationWorker.new.perform +``` diff --git a/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md b/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md index 79262e2d0dc..6087d4bd8f7 100644 --- a/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md +++ b/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md @@ -21,8 +21,7 @@ Value Stream Analytics (VSA). ## Current Status -As of 14.8 the aggregated VSA backend is used only in the `gitlab-org` group, for testing purposes -. We plan to gradually roll it out in the next major release (15.0) for the rest of the groups. +The aggregated backend is used by default since GitLab 15.0 on the group-level. ## Motivation @@ -53,44 +52,6 @@ database with a minimal development effort. used by the system. - Example: `MIN(issues.created_at, issues.updated_at)` -## How does Value Stream Analytics work? - -Value Stream Analytics calculates the duration between two timestamp columns or timestamp -expressions and runs various aggregations on the data. - -Examples: - -- Duration between the Merge Request creation time and Merge Request merge time. -- Duration between the Issue creation time and Issue close time. - -This duration is exposed in various ways: - -- Aggregation: median, average -- Listing: list the duration for individual Merge Request and Issue records - -Apart from the durations, we expose the record count within a stage. - -### Stages - -A stage represents an event pair (start and end events) with additional metadata, such as the name -of the stage. Stages are configurable by the user within the pairing rules defined in the backend. - -**Example stage: Code Review** - -- Start event identifier: Merge Request creation time -- Start event column: uses the `merge_requests.created_at` timestamp column. -- End event identifier: Merge Request merge time -- End event column: uses the `merge_request_metrics.merged_at` timestamp column. -- Stage event hash ID: a calculated hash for the pair of start and end event identifiers. - - If two stages have the same configuration of start and end events, then their stage event hash - IDs are identical. - - The stage event hash ID is later used to store the aggregated data in partitioned database tables. - -### Value streams - -Value streams are container objects for the stages. There can be multiple value streams per group -or project focusing on different aspects of the Dev Ops lifecycle. - ### Example configuration  diff --git a/doc/development/work_items.md b/doc/development/work_items.md index 3625f85eb82..f15c66ae847 100644 --- a/doc/development/work_items.md +++ b/doc/development/work_items.md @@ -36,7 +36,7 @@ Here are some problems with current issues usage and why we are looking into wor differences in common interactions that the user needs to hold a complicated mental model of how they each behave. - Issues are not extensible enough to support all of the emerging jobs they need to facilitate. -- Codebase maintainability and feature development becomes a bigger challenge as we grow the Issue type. +- Codebase maintainability and feature development becomes a bigger challenge as we grow the Issue type beyond its core role of issue tracking into supporting the different work item types and handling logic and structure differences. - New functionality is typically implemented with first class objects that import behavior from issues via @@ -204,3 +204,33 @@ provide a smooth migration path of epics to WIT with minimal disruption to user We will move towards work items, work item types, and custom widgets (CW) in an iterative process. For a rough outline of the work ahead of us, see [epic 6033](https://gitlab.com/groups/gitlab-org/-/epics/6033). + +## Redis HLL Counter Schema + +We need a more scalable Redis counter schema for work items that is inclusive of Plan xMAU, Project Management xMAU, Certify xMAU, and +Product Planning xMAU. We cannot aggregate and dedupe events across features within a group or at the stage level with +our current Redis slot schema. + +All three Plan product groups will be using the same base object (`work item`). Each product group still needs to +track MAU. + +### Proposed aggregate counter schema + +```mermaid +graph TD + Event[Specific Interaction Counter] --> AC[Aggregate Counters] + AC --> Plan[Plan xMAU] + AC --> PM[Project Management xMAU] + AC --> PP[Product Planning xMAU] + AC --> Cer[Certify xMAU] + AC --> WI[Work Items Users] +``` + +### Implementation + +The new aggregate schema is already implemented and we are already tracking work item unique actions +in [GitLab.com](https://gitlab.com). + +For implementation details, this [MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93231) can be used +as a reference. The MR covers the definition of new unique actions, event tracking in the code and also +adding the new unique actions to the required aggregate counters. diff --git a/doc/development/workhorse/index.md b/doc/development/workhorse/index.md index 962124248ef..f210f511954 100644 --- a/doc/development/workhorse/index.md +++ b/doc/development/workhorse/index.md @@ -10,7 +10,7 @@ GitLab Workhorse is a smart reverse proxy for GitLab. It handles "large" HTTP requests such as file downloads, file uploads, Git push/pull and Git archive downloads. -Workhorse itself is not a feature, but there are +Workhorse itself is not a feature, but there are [several features in GitLab](gitlab_features.md) that would not work efficiently without Workhorse. The canonical source for Workhorse is diff --git a/doc/gitlab-basics/start-using-git.md b/doc/gitlab-basics/start-using-git.md index c6de723246c..875bd00ee55 100644 --- a/doc/gitlab-basics/start-using-git.md +++ b/doc/gitlab-basics/start-using-git.md @@ -148,7 +148,7 @@ between your computer and GitLab. ``` 1. GitLab requests your username and password: - - If you have 2FA enabled for your account, you must use a [Personal Access Token](../user/profile/personal_access_tokens.md) + - If you have 2FA enabled for your account, you must [clone using a token](#clone-using-a-token) with `read_repository` or `write_repository` permissions instead of your account's password. - If you don't have 2FA enabled, use your account's password. @@ -163,6 +163,24 @@ On Windows, if you enter your password incorrectly multiple times and an `Access add your namespace (username or group) to the path: `git clone https://namespace@gitlab.com/gitlab-org/gitlab.git`. +#### Clone using a token + +Clone with HTTPS using a token if: + +- You want to use 2FA. +- You want to have a revokable set of credentials scoped to one or more repositories. + +You can use any of these tokens to authenticate when cloning over HTTPS: + +- [Personal access tokens](../user/profile/personal_access_tokens.md). +- [Deploy tokens](../user/project/deploy_tokens/index.md). +- [Project access tokens](../user/project/settings/project_access_tokens.md). +- [Group access tokens](../user/group/settings/group_access_tokens.md). + +```shell +git clone https://<username>:<token>@gitlab.example.com/tanuki/awesome_project.git +``` + ### Convert a local directory into a repository You can initialize a local folder so Git tracks it as a repository. diff --git a/doc/install/aws/manual_install_aws.md b/doc/install/aws/manual_install_aws.md index 7494dc1e95e..2aa2aa0c3f7 100644 --- a/doc/install/aws/manual_install_aws.md +++ b/doc/install/aws/manual_install_aws.md @@ -483,7 +483,7 @@ Connect to your GitLab instance via **Bastion Host A** using [SSH Agent Forwardi #### Disable Let's Encrypt -Because we're adding our SSL certificate at the load balancer, we do not need the GitLab built-in support for Let's Encrypt. Let's Encrypt [is enabled by default](https://docs.gitlab.com/omnibus/settings/ssl.html#lets-encrypt-integration) when using an `https` domain in GitLab 10.7 and later, so we must explicitly disable it: +Because we're adding our SSL certificate at the load balancer, we do not need the GitLab built-in support for Let's Encrypt. Let's Encrypt [is enabled by default](https://docs.gitlab.com/omnibus/settings/ssl.html#enable-the-lets-encrypt-integration) when using an `https` domain in GitLab 10.7 and later, so we must explicitly disable it: 1. Open `/etc/gitlab/gitlab.rb` and disable it: @@ -605,7 +605,7 @@ Now that we have our EC2 instance ready, follow the [documentation to install Gi #### Add Support for Proxied SSL -As we are terminating SSL at our [load balancer](#load-balancer), follow the steps at [Supporting proxied SSL](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) to configure this in `/etc/gitlab/gitlab.rb`. +As we are terminating SSL at our [load balancer](#load-balancer), follow the steps at [Supporting proxied SSL](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination) to configure this in `/etc/gitlab/gitlab.rb`. Remember to run `sudo gitlab-ctl reconfigure` after saving the changes to the `gitlab.rb` file. diff --git a/doc/install/azure/index.md b/doc/install/azure/index.md index 7f155a78140..dd2e7d678e8 100644 --- a/doc/install/azure/index.md +++ b/doc/install/azure/index.md @@ -233,7 +233,7 @@ The first thing that appears is the sign-in page. GitLab creates an administrato The credentials are: - Username: `root` -- Password: the password is automatically created, and there are +- Password: the password is automatically created, and there are [two ways to find it](https://docs.bitnami.com/azure/faq/get-started/find-credentials/). After signing in, be sure to immediately [change the password](../../user/profile/index.md#change-your-password). @@ -248,7 +248,7 @@ in this section whenever you need to update GitLab. To determine the version of GitLab you're currently running: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Dashboard**. 1. Find the version under the **Components** table. @@ -294,7 +294,7 @@ up-to-date GitLab instance. ## Next steps and further configuration Now that you have a functional GitLab instance, follow the -[next steps](../index.md#next-steps) to learn what more you can do with your +[next steps](../next_steps.md) to learn what more you can do with your new installation. ## Troubleshooting diff --git a/doc/install/cloud_native/index.md b/doc/install/cloud_native/index.md index f3265500877..d971cd419e9 100644 --- a/doc/install/cloud_native/index.md +++ b/doc/install/cloud_native/index.md @@ -1,51 +1,11 @@ --- -stage: Systems -group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -comments: false -description: Install GitLab in a cloud native environment -type: index +redirect_to: 'https://docs.gitlab.com/charts/' +remove_date: '2023-09-09' --- -# Cloud Native GitLab **(FREE SELF)** +This document was moved to [another location](https://docs.gitlab.com/charts/). -[Cloud Native GitLab](https://gitlab.com/gitlab-org/build/CNG) provides cloud -native containers to deploy GitLab. These containers may be deployed and managed -via Helm using GitLab Charts or GitLab Operator on Kubernetes, OpenShift, -and Kubernetes compatible container platforms: - -- [Helm charts](https://docs.gitlab.com/charts/): The cloud native Helm chart - installs GitLab and all of its components on Kubernetes. Use this method if - your infrastructure is built on Kubernetes and you're familiar with how it - works. The methods for management, observability, and some concepts are - different than traditional deployments. -- [GitLab Operator](https://docs.gitlab.com/operator/): The GitLab Operator - provides an installation and management method for GitLab following the - [Kubernetes Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/). - You can also use the GitLab Operator to run GitLab in an - [OpenShift](../openshift_and_gitlab/index.md) environment. - -Here's an overview of how the containers are built: - -```mermaid -graph TD - subgraph Code - CNG --> HC - CNG --> GOP - HC --> GOP - end - - subgraph Deploy - GOP --> K8s - GOP --> OS - CNG --> DC - HC --> K8s - end - - CNG[Cloud Native GitLab containers] - HC[Helm Chart] - K8s(Kubernetes) - GOP[GitLab Operator] - OS(OpenShift) - DC(Docker Compose) -``` +<!-- This redirect file can be deleted after <2023-09-09>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
\ No newline at end of file diff --git a/doc/install/docker.md b/doc/install/docker.md index 7ec1b7c741a..bf08fecc9ca 100644 --- a/doc/install/docker.md +++ b/doc/install/docker.md @@ -301,7 +301,7 @@ point to a valid URL. To receive e-mails from GitLab you have to configure the [SMTP settings](https://docs.gitlab.com/omnibus/settings/smtp.html) because the GitLab Docker image doesn't have an SMTP server installed. You may also be interested in -[enabling HTTPS](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +[enabling HTTPS](https://docs.gitlab.com/omnibus/settings/ssl.html). After you make all the changes you want, you will need to restart the container in order to reconfigure GitLab: @@ -462,7 +462,9 @@ In most cases, upgrading GitLab is as easy as downloading the newest Docker To upgrade GitLab that was [installed using Docker Engine](#install-gitlab-using-docker-engine): -1. Take a [backup](#back-up-gitlab). +1. Take a [backup](#back-up-gitlab). As a minimum, back up [the database](#create-a-database-backup) and + the GitLab secrets file. + 1. Stop the running container: ```shell @@ -506,7 +508,9 @@ when upgrading between major versions. To upgrade GitLab that was [installed using Docker Compose](#install-gitlab-using-docker-compose): -1. Take a [backup](#back-up-gitlab). +1. Take a [backup](#back-up-gitlab). As a minimum, back up [the database](#create-a-database-backup) and + the GitLab secrets file. + 1. Download the newest release and upgrade your GitLab instance: ```shell @@ -527,8 +531,11 @@ We recommend you convert from the same version of CE to EE (for example, CE 14.1 This is not explicitly necessary, and any standard upgrade (for example, CE 14.0 to EE 14.1) should work. The following steps assume that you are upgrading the same version. -1. Take a [backup](#back-up-gitlab). +1. Take a [backup](#back-up-gitlab). As a minimum, back up [the database](#create-a-database-backup) and + the GitLab secrets file. + 1. Stop the current CE container, and remove or rename it. + 1. To create a new container with GitLab EE, replace `ce` with `ee` in your `docker run` command or `docker-compose.yml` file. However, reuse the CE container name, port and file mappings, and version. @@ -538,6 +545,22 @@ The following steps assume that you are upgrading the same version. This is an optional step. If you [installed the documentation site](#install-the-product-documentation), see how to [upgrade to another version](../administration/docs_self_host.md#upgrade-using-docker). +### Downgrade GitLab + +To downgrade GitLab after an upgrade: + +1. Follow the upgrade procedure, but [specify the tag for the original version of GitLab](#use-tagged-versions-of-gitlab) + instead of `latest`. + +1. Restore the [database backup you made](#create-a-database-backup) as part of the upgrade. + + - Restoring is required to back out database data and schema changes (migrations) made as part of the upgrade. + - GitLab backups must be restored to the exact same version and edition. + - [Follow the restore steps for Docker images](../raketasks/restore_gitlab.md#restore-for-docker-image-and-gitlab-helm-chart-installations), including + stopping Puma and Sidekiq. Only the database must be restored, so add + `SKIP=artifacts,repositories,registry,uploads,builds,pages,lfs,packages,terraform_state` + to the `gitlab-backup restore` command line arguments. + ## Back up GitLab You can create a GitLab backup with: @@ -554,6 +577,23 @@ If configuration is provided entirely via the `GITLAB_OMNIBUS_CONFIG` environmen meaning no configuration is set directly in the `gitlab.rb` file, then there is no need to back up the `gitlab.rb` file. +WARNING: +[Backing up the GitLab secrets file](../raketasks/backup_gitlab.md#storing-configuration-files) is required +to avoid [complicated steps](../raketasks/backup_restore.md#when-the-secrets-file-is-lost) when recovering +GitLab from backup. The secrets file is stored at `/etc/gitlab/gitlab-secrets.json` inside the container, or +`$GITLAB_HOME/config/gitlab-secrets.json` [on the container host](#set-up-the-volumes-location). + +### Create a database backup + +A database backup is required to roll back GitLab upgrade if you encounter issues. + +```shell +docker exec -t <container name> gitlab-backup create SKIP=artifacts,repositories,registry,uploads,builds,pages,lfs,packages,terraform_state +``` + +The backup is written to `/var/opt/gitlab/backups` which should be on a +[volume mounted by Docker](#set-up-the-volumes-location). + ## Installing GitLab Community Edition [GitLab CE Docker image](https://hub.docker.com/r/gitlab/gitlab-ce/) diff --git a/doc/install/google_cloud_platform/index.md b/doc/install/google_cloud_platform/index.md index b63ab8d4e2b..e924b6f7c41 100644 --- a/doc/install/google_cloud_platform/index.md +++ b/doc/install/google_cloud_platform/index.md @@ -42,7 +42,7 @@ To deploy GitLab on GCP you must create a virtual machine:  -1. To select the size, type, and desired [operating system](../requirements.md#supported-linux-distributions), +1. To select the size, type, and desired [operating system](../../administration/package_information/supported_os.md#supported-operating-systems), select **Change** under `Boot disk`. select **Select** when finished. 1. As a last step allow HTTP and HTTPS traffic, then select **Create**. The process finishes in a few seconds. @@ -117,8 +117,8 @@ here's how you configure GitLab to be aware of the change: ### Configuring HTTPS with the domain name -Although not needed, it's strongly recommended to secure GitLab with a TLS -certificate. Follow the steps in the [Omnibus documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). +Although not needed, it's strongly recommended to secure GitLab with a +[TLS certificate](https://docs.gitlab.com/omnibus/settings/ssl.html). ### Configuring the email SMTP settings diff --git a/doc/install/index.md b/doc/install/index.md index 413c5116c14..89203c652d1 100644 --- a/doc/install/index.md +++ b/doc/install/index.md @@ -7,46 +7,14 @@ description: Read through the GitLab installation methods. type: index --- -# Installation **(FREE SELF)** +# Install GitLab **(FREE SELF)** -GitLab can be installed in most GNU/Linux distributions, and with several -cloud providers. To get the best experience from GitLab, you must balance -performance, reliability, ease of administration (backups, upgrades, and -troubleshooting), and the cost of hosting. +You can install GitLab on most GNU/Linux distributions, on several +cloud providers, and in Kubernetes clusters. -## Requirements +To get the best experience, you should balance performance, reliability, +ease of administration (backups, upgrades, and troubleshooting) with the cost of hosting. -Before you install GitLab, be sure to review the [system requirements](requirements.md). -The system requirements include details about the minimum hardware, software, -database, and additional requirements to support GitLab. +To get started, [choose your installation method](install_methods.md). -## Choose the installation method - -Depending on your platform, select from the following available methods to -install GitLab: - -| Installation method | Description | When to choose | -|----------------------------------------------------------------|-------------|----------------| -| [Linux package](https://docs.gitlab.com/omnibus/installation/) | The official deb/rpm packages (also known as Omnibus GitLab) that contains a bundle of GitLab and the components it depends on, including PostgreSQL, Redis, and Sidekiq. | This method is recommended for getting started. The Linux packages are mature, scalable, and are used today on GitLab.com. If you need additional flexibility and resilience, we recommend deploying GitLab as described in the [reference architecture documentation](../administration/reference_architectures/index.md). | -| [Helm charts](https://docs.gitlab.com/charts/) | The cloud native Helm chart for installing GitLab and all of its components on Kubernetes. | When installing GitLab on Kubernetes, it has some trade-offs that you must be aware of: <br/>- Administration and troubleshooting requires Kubernetes knowledge.<br/>- It can be more expensive for smaller installations. The default installation requires more resources than a single node Linux package deployment, as most services are deployed in a redundant fashion.<br/><br/> Use this method if your infrastructure is built on Kubernetes and you're familiar with how it works. The methods for management, observability, and some concepts are different than traditional deployments. | -| [Docker](docker.md) | The GitLab packages, Dockerized. | Use this method if you're familiar with Docker. | -| [Source](installation.md) | Install GitLab and all of its components from scratch. | Use this method if none of the previous methods are available for your platform. Can be used for unsupported systems like \*BSD.| -| [GitLab Environment Toolkit (GET)](https://gitlab.com/gitlab-org/gitlab-environment-toolkit#documentation) | The GitLab Environment toolkit provides a set of automation tools to deploy a [reference architecture](../administration/reference_architectures/index.md) on most major cloud providers. | Customers are very welcome to trial and evaluate GET today, however be aware of [key limitations](https://gitlab.com/gitlab-org/gitlab-environment-toolkit#missing-features-to-be-aware-of) of the current iteration. For production environments further manual setup is required based on your specific requirements. | -| [GitLab Operator](https://docs.gitlab.com/operator/) | The GitLab Operator provides an installation and management method for GitLab following the [Kubernetes Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/). | Use the GitLab Operator to run GitLab in an [OpenShift](openshift_and_gitlab/index.md) environment. | - -## Install GitLab on cloud providers - -Regardless of the installation method, you can install GitLab on several cloud -providers, assuming the cloud provider supports it. Here are several possible installation -methods, the majority which use the Linux packages: - -| Cloud provider | Description | -|---------------------------------------------------------------|-------------| -| [AWS (HA)](aws/index.md) | Install GitLab on AWS using the community AMIs provided by GitLab. | -| [Google Cloud Platform (GCP)](google_cloud_platform/index.md) | Install GitLab on a VM in GCP. | -| [Azure](azure/index.md) | Install GitLab from Azure Marketplace. | - -## Next steps - -After you complete the steps for installing GitLab, you can -[configure your instance](next_steps.md). +If you already have a running instance, learn how to [configure it](next_steps.md). diff --git a/doc/install/install_methods.md b/doc/install/install_methods.md new file mode 100644 index 00000000000..6c62deba6fa --- /dev/null +++ b/doc/install/install_methods.md @@ -0,0 +1,51 @@ +--- +stage: Systems +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +comments: false +description: Read through the GitLab installation methods. +type: index +--- + +# Installation methods + +You can install GitLab by using any of the following methods. + +| Installation method | Description | When to choose | +|----------------------------------------------------------------|-------------|----------------| +| [Linux package](https://docs.gitlab.com/omnibus/installation/) | The official deb/rpm packages (also known as Omnibus GitLab). The package has GitLab and dependent components, including PostgreSQL, Redis, and Sidekiq. | Use if you want the most mature, scalable method. This version is also used on GitLab.com. <br>- For additional flexibility and resilience, see the [reference architecture documentation](../administration/reference_architectures/index.md).<br>- Review the [system requirements](requirements.md).<br>- View the [list of supported Linux operating systems](../administration/package_information/supported_os.md#supported-operating-systems). | +| [Helm chart](https://docs.gitlab.com/charts/) | A chart for installing a cloud-native version of GitLab and its components on Kubernetes. | Use if your infrastructure is on Kubernetes and you're familiar with how it works. Management, observability, and some concepts are different than traditional deployments.<br/>- Administration and troubleshooting requires Kubernetes knowledge.<br/>- It can be more expensive for smaller installations. The default installation requires more resources than a single node Linux package deployment, because most services are deployed in a redundant fashion.<br/><br/> | +| [Docker](docker.md) | The GitLab packages in a Docker container. | Use if you're familiar with Docker. | +| [Source](installation.md) | GitLab and its components from scratch. | Use if none of the previous methods are available for your platform. Can use for unsupported systems like \*BSD.| +| [GitLab Environment Toolkit (GET)](https://gitlab.com/gitlab-org/gitlab-environment-toolkit#documentation) | A set of automation tools. | Use to deploy a [reference architecture](../administration/reference_architectures/index.md) on most major cloud providers. Has some [limitations](https://gitlab.com/gitlab-org/gitlab-environment-toolkit#missing-features-to-be-aware-of) and manual setup for production environments. | +| [GitLab Operator](https://docs.gitlab.com/operator/) | An installation and management method that follows the [Kubernetes Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/). | Use to run GitLab in an [OpenShift](openshift_and_gitlab/index.md) environment. | + +## Cloud providers + +You can install GitLab on several cloud providers. + +| Cloud provider | Description | +|---------------------------------------------------------------|-------------| +| [AWS (HA)](aws/index.md) | Install GitLab on AWS using the community AMIs provided by GitLab. | +| [Google Cloud Platform (GCP)](google_cloud_platform/index.md) | Install GitLab on a VM in GCP. | +| [Azure](azure/index.md) | Install GitLab from Azure Marketplace. | + +## Unsupported Linux distributions and Unix-like operating systems + +- Arch Linux +- Fedora +- FreeBSD +- Gentoo +- macOS + +Installation of GitLab on these operating systems is possible, but not supported. +See the [installation from source guide](installation.md) and the [installation guides](https://about.gitlab.com/install/) for more information. + +See [OS versions that are no longer supported](../administration/package_information/supported_os.md#os-versions-that-are-no-longer-supported) for Omnibus installs page +for a list of supported and unsupported OS versions as well as the last support GitLab version for that OS. + +## Microsoft Windows + +GitLab is developed for Linux-based operating systems. +It does **not** run on Microsoft Windows, and we have no plans to support it in the near future. For the latest development status, view this [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/22337). +Consider using a virtual machine to run GitLab. diff --git a/doc/install/installation.md b/doc/install/installation.md index 2f6adb06322..5982e354ae5 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -129,7 +129,7 @@ sudo apt-get install libkrb5-dev ### Git -From GitLab 13.6, we recommend you use the +From GitLab 13.6, we recommend you use the [Git version provided by Gitaly](https://gitlab.com/gitlab-org/gitaly/-/issues/2729) that: @@ -239,7 +239,7 @@ sudo make install GitLab has several daemons written in Go. To install GitLab we need a Go compiler. The instructions below assume you use 64-bit -Linux. You can find downloads for other platforms at the +Linux. You can find downloads for other platforms at the [Go download page](https://go.dev/dl). ```shell diff --git a/doc/install/migrate/compare_sm_to_saas.md b/doc/install/migrate/compare_sm_to_saas.md new file mode 100644 index 00000000000..df79987a2fa --- /dev/null +++ b/doc/install/migrate/compare_sm_to_saas.md @@ -0,0 +1,124 @@ +--- +stage: Systems +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Comparison of GitLab self-managed with GitLab SaaS + +GitLab SaaS is the largest hosted instance of GitLab in the world, managed by an +[all-remote team](https://about.gitlab.com/company/culture/all-remote/) that knows GitLab best. With GitLab SaaS, updates, maintenance, and patches are all performed by this team. + +Self-managed GitLab gives you a deeper breadth of control over many of the functions and systems of the application. + +## Administration + +In GitLab SaaS, administration tasks are limited compared to a self-managed application. + +In a self-managed instance: + +- You have complete access and administrative control over the application, including the [Admin Area](../../user/admin_area/settings/index.md). +- You can impersonate, create, add, and remove users. +- You can assign the [`Auditor`](../../administration/auditor_users.md) user type and `External` role. + +On GitLab SaaS: + +- You have limited administrative control. For example, you cannot impersonate, create, add, or remove users. +- You cannot access the [Admin Area](../../user/admin_area/settings/index.md). +- You cannot assign the `Auditor` user type and `External` role. + +## Logs + +Logs give insight into your processes and can help GitLab Support maintain your application and resolve problems. + +In a self-managed instance: + +- You have full access to system logs. + +On GitLab SaaS: + +- You do not have access to system logs because they are at the instance level, and managed by the GitLab [infrastructure team](https://about.gitlab.com/handbook/engineering/infrastructure/). +- You can view [Audit Events](../../administration/audit_events.md) and the [GitLab API](../../api/audit_events.md). +- You must [request audit information](https://about.gitlab.com/handbook/support/workflows/log_requests.html) from the Support team. + +## Runners + +Runners are available for both SaaS and self-managed applications. + +In a self-managed instance, your runner availability and options are broader, but there are more [security concerns](https://docs.gitlab.com/runner/security/#security-for-self-managed-runners) to consider. + +On GitLab SaaS: + +- Private [runners](../../ci/runners/index.md) are available for GitLab SaaS [groups](../../user/group/index.md) and [projects](../../user/project/index.md). +- Shared runners provided by GitLab SaaS are not configurable. Each runner instance is used once for only one job, ensuring any sensitive data left on the system is destroyed after the job is complete. +- Shared runners are subject to usage limits and are [plan specific](https://about.gitlab.com/pricing/). + +## Custom Git hooks + +In a self-managed instance you can use any custom Git hooks. + +On GitLab SaaS: + +- SaaS users do not have access to the file system, and cannot use custom Git hooks. +- You can use [webhooks](../../user/project/integrations/webhooks.md) as an alternative. + +## API and GraphQL + +In a self-managed instance, users can access all API endpoints, including those that require instance `admin` permissions. + +On GitLab SaaS: + +- SaaS users have access to all of the [API endpoints](../../api/index.md) except those that require instance `admin` permissions. +- Only authorized GitLab engineers have administrative access. + +## Authentication + +In a self-managed instance: + +- You can use an internal encryption key for your data store. +- You can view console logs. +- You can enforce jobs on every pipeline across the group or organization. +- You have control over your data backup. +- You can use the [Interactive Web Terminal](../../ci/interactive_web_terminal/index.md#interactive-web-terminals) for shared runners. + +On GitLab SaaS: + +- You cannot use internal encryption key for the data store ([bring-your-own-key](https://about.gitlab.com/handbook/engineering/security/vulnerability_management/encryption-policy.html#rolling-your-own-crypto)). +- You cannot view console logs. +- You cannot enforce jobs on every pipeline across the group or organization. +- You cannot configure or control data backups. You must use [group](../../api/group_import_export.md) and [project](../../api/project_import_export.md) export. +- The [Interactive Web Terminal](../../ci/interactive_web_terminal/index.md#interactive-web-terminals) is not available for shared runners. + +## Public or private projects + +Project privacy is different when using a self-managed application or GitLab SaaS. + +In a self-managed instance, you control who can view your projects. + +On GitLab SaaS: + +- The GitLab SaaS instance is open to the public. +- When your projects are set as `Public`, they are open to everyone on the public internet. + +## Encryption + +In a self-managed instance, you control the encryption type and configuration. + +On GitLab SaaS: + +- An [Access Management Process](https://about.gitlab.com/handbook/engineering/security/#access-management-process) is in place. +- All data on GitLab.com is encrypted at rest by default. Access to encryption keys is strictly managed by GitLab. +- GitLab does not access your tenant data except as part of a verified service request from you. + +## Support + +In a self-managed instance: + +- You can access any of your back-end systems. +- Our Support team can request logs to assist you. + +On GitLab SaaS: + +- For your privacy and security, there is no public access to GitLab back-end systems. +- Support staff work with [Site Reliability Engineers](https://about.gitlab.com/job-families/engineering/infrastructure/site-reliability-engineer/) to support the [infrastructure](https://about.gitlab.com/handbook/engineering/infrastructure/). +- GitLab Support can access instance logs and view projects, as well as impersonate users. The Support Team can access your logs. diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 93d66dc92a9..03870897417 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -4,38 +4,9 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# GitLab installation minimum requirements **(FREE SELF)** +# Installation system requirements **(FREE SELF)** -This page includes information about both the supported operating systems and -the minimum requirements needed to install and use GitLab. - -## Operating Systems - -### Supported Linux distributions - -See the [list of supported operating systems](../administration/package_information/supported_os.md#supported-operating-systems). - -For the installation options, see [the main installation page](index.md). - -### Unsupported Linux distributions and Unix-like operating systems - -- Arch Linux -- Fedora -- FreeBSD -- Gentoo -- macOS - -Installation of GitLab on these operating systems is possible, but not supported. -See the [installation from source guide](installation.md) and the [installation guides](https://about.gitlab.com/install/) for more information. - -See [OS versions that are no longer supported](../administration/package_information/supported_os.md#os-versions-that-are-no-longer-supported) for Omnibus installs page -for a list of supported and unsupported OS versions as well as the last support GitLab version for that OS. - -### Microsoft Windows - -GitLab is developed for Linux-based operating systems. -It does **not** run on Microsoft Windows, and we have no plans to support it in the near future. For the latest development status view this [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/22337). -Consider using a virtual machine to run GitLab. +This page includes information about the minimum requirements you need to install and use GitLab. ## Software requirements diff --git a/doc/integration/advanced_search/elasticsearch.md b/doc/integration/advanced_search/elasticsearch.md index dc3dc4d2012..755dc5230e9 100644 --- a/doc/integration/advanced_search/elasticsearch.md +++ b/doc/integration/advanced_search/elasticsearch.md @@ -51,7 +51,7 @@ Memory, CPU, and storage resource amounts vary depending on the amount of data y each node should have: - [Memory](https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html#_memory): 8 GiB (minimum). -- [CPU](https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html#_cpus): Modern processor with multiple cores. GitLab.com has minimal CPU requirements for Elasticsearch. Multiple cores provide extra concurrency, which is more beneficial than faster CPUs. +- [CPU](https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html#_cpus): Modern processor with multiple cores. GitLab has minimal CPU requirements for Elasticsearch. Multiple cores provide extra concurrency, which is more beneficial than faster CPUs. - [Storage](https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html#_disks): Use SSD storage. The total storage size of all Elasticsearch nodes is about 50% of the total size of your Git repositories. It includes one primary and one replica. The [`estimate_cluster_size`](#gitlab-advanced-search-rake-tasks) Rake task ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221177) in GitLab 13.10) uses total repository size to estimate the Advanced Search storage requirements. ## Install Elasticsearch @@ -166,7 +166,7 @@ may need to set the `production -> elasticsearch -> indexer_path` setting in you ### View indexing errors Errors from the [GitLab Elasticsearch Indexer](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer) are reported in -the [`sidekiq.log`](../../administration/logs/index.md#sidekiqlog) file with a `json.exception.class` of `Gitlab::Elastic::Indexer::Error`. +the [`elasticsearch.log`](../../administration/logs/index.md#elasticsearchlog) file and the [`sidekiq.log`](../../administration/logs/index.md#sidekiqlog) file with a `json.exception.class` of `Gitlab::Elastic::Indexer::Error`. These errors may occur when indexing Git repository data. ## Enable Advanced Search @@ -175,7 +175,7 @@ For GitLab instances with more than 50GB repository data you can follow the inst To enable Advanced Search, you must have administrator access to GitLab: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Advanced Search**. NOTE: @@ -223,7 +223,7 @@ The following Elasticsearch settings are available: | `Number of Elasticsearch shards` | Elasticsearch indices are split into multiple shards for performance reasons. In general, you should use at least 5 shards, and indices with tens of millions of documents need to have more shards ([see below](#guidance-on-choosing-optimal-cluster-configuration)). Changes to this value do not take effect until the index is recreated. You can read more about tradeoffs in the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/scalability.html). | | `Number of Elasticsearch replicas` | Each Elasticsearch shard can have a number of replicas. These are a complete copy of the shard, and can provide increased query performance or resilience against hardware failure. Increasing this value increases total disk space required by the index. | | `Limit the number of namespaces and projects that can be indexed` | Enabling this allows you to select namespaces and projects to index. All other namespaces and projects use database search instead. If you enable this option but do not select any namespaces or projects, none are indexed. [Read more below](#limit-the-number-of-namespaces-and-projects-that-can-be-indexed).| -| `Using AWS hosted Elasticsearch with IAM credentials` | Sign your Elasticsearch requests using [AWS IAM authorization](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html), [AWS EC2 Instance Profile Credentials](https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-iam-instance-profile.html#getting-started-create-iam-instance-profile-cli), or [AWS ECS Tasks Credentials](https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-iam-roles.html). Please refer to [Identity and Access Management in Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html) for details of AWS hosted OpenSearch domain access policy configuration. | +| `Using AWS OpenSearch Service with IAM credentials` | Sign your OpenSearch requests using [AWS IAM authorization](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html), [AWS EC2 Instance Profile Credentials](https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-iam-instance-profile.html#getting-started-create-iam-instance-profile-cli), or [AWS ECS Tasks Credentials](https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-iam-roles.html). Please refer to [Identity and Access Management in Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html) for details of AWS hosted OpenSearch domain access policy configuration. | | `AWS Region` | The AWS region in which your OpenSearch Service is located. | | `AWS Access Key` | The AWS access key. | | `AWS Secret Access Key` | The AWS secret access key. | @@ -271,7 +271,7 @@ You can improve the language support for Chinese and Japanese languages by utili To enable languages support: 1. Install the desired plugins, please refer to [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/plugins/7.9/installation.html) for plugins installation instructions. The plugins must be installed on every node in the cluster, and each node must be restarted after installation. For a list of plugins, see the table later in this section. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Advanced Search**. 1. Locate **Custom analyzers: language support**. 1. Enable plugins support for **Indexing**. @@ -292,7 +292,7 @@ For guidance on what to install, see the following Elasticsearch language plugin To disable the Elasticsearch integration: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Advanced Search**. 1. Uncheck **Elasticsearch indexing** and **Search with Elasticsearch enabled**. 1. Select **Save changes**. @@ -308,7 +308,7 @@ To disable the Elasticsearch integration: ## Unpause Indexing -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Advanced Search**. 1. Expand **Advanced Search**. 1. Clear the **Pause Elasticsearch indexing** checkbox. @@ -331,7 +331,7 @@ index alias to it which becomes the new `primary` index. At the end, we resume t To trigger the reindexing process: 1. Sign in to your GitLab instance as an administrator. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Advanced Search**. 1. Expand **Elasticsearch zero-downtime reindexing**. 1. Select **Trigger cluster reindexing**. @@ -348,7 +348,7 @@ While the reindexing is running, you can follow its progress under that same sec > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55681) in GitLab 13.12. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Advanced Search**. 1. Expand **Elasticsearch zero-downtime reindexing**, and you'll find the following options: @@ -396,7 +396,7 @@ Sometimes, you might want to abandon the unfinished reindex job and resume the i bundle exec rake gitlab:elastic:mark_reindex_failed RAILS_ENV=production ``` -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Advanced Search**. 1. Expand **Advanced Search**. 1. Clear the **Pause Elasticsearch indexing** checkbox. @@ -562,7 +562,7 @@ For basic guidance on choosing a cluster configuration you may refer to [Elastic - A good guideline is to ensure you keep the number of shards per node below 20 per GB heap it has configured. A node with a 30GB heap should therefore have a maximum of 600 shards, but the further below this limit you can keep it the better. This generally helps the cluster stay in good health. - Number of Elasticsearch shards: - Small shards result in small segments, which increases overhead. Aim to keep the average shard size between at least a few GB and a few tens of GB. - - Another consideration is the number of documents. To determine the number of shards to use, sum the numbers in the **Menu > Admin > Dashboard > Statistics** pane (the number of documents to be indexed), divide by 5 million, and add 5. For example: + - Another consideration is the number of documents. To determine the number of shards to use, sum the numbers in the **Main menu > Admin > Dashboard > Statistics** pane (the number of documents to be indexed), divide by 5 million, and add 5. For example: - If you have fewer than about 2,000,000 documents, use the default of 5 shards - 10,000,000 documents: `10000000/5000000 + 5` = 7 shards - 100,000,000 documents: `100000000/5000000 + 5` = 25 shards @@ -639,7 +639,7 @@ Make sure to prepare for this task by having a ``` This enqueues a Sidekiq job for each project that needs to be indexed. - You can view the jobs in **Menu > Admin > Monitoring > Background Jobs > Queues Tab** + You can view the jobs in **Main menu > Admin > Monitoring > Background Jobs > Queues Tab** and select `elastic_commit_indexer`, or you can query indexing status using a Rake task: ```shell diff --git a/doc/integration/advanced_search/elasticsearch_troubleshooting.md b/doc/integration/advanced_search/elasticsearch_troubleshooting.md index fb558441d6a..e1a566541c2 100644 --- a/doc/integration/advanced_search/elasticsearch_troubleshooting.md +++ b/doc/integration/advanced_search/elasticsearch_troubleshooting.md @@ -5,10 +5,68 @@ group: Global Search info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Elasticsearch troubleshooting **(PREMIUM SELF)** +# Troubleshooting Elasticsearch **(PREMIUM SELF)** Use the following information to troubleshoot Elasticsearch issues. +## Set configurations in the Rails console + +See [Starting a Rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session). + +### List attributes + +To list all available attributes: + +1. Open the Rails console (`gitlab rails c`). +1. Run the following command: + +```ruby +ApplicationSetting.last.attributes +``` + +The output contains all the settings available in [Elasticsearch integration](../../integration/advanced_search/elasticsearch.md), such as `elasticsearch_indexing`, `elasticsearch_url`, `elasticsearch_replicas`, and `elasticsearch_pause_indexing`. + +### Set attributes + +To set an Elasticsearch integration setting, run a command like: + +```ruby +ApplicationSetting.last.update(elasticsearch_url: '<your ES URL and port>') + +#or + +ApplicationSetting.last.update(elasticsearch_indexing: false) +``` + +### Get attributes + +To check if the settings have been set in [Elasticsearch integration](../../integration/advanced_search/elasticsearch.md) or in the Rails console, run a command like: + +```ruby +Gitlab::CurrentSettings.elasticsearch_url + +#or + +Gitlab::CurrentSettings.elasticsearch_indexing +``` + +### Change the password + +To change the Elasticsearch password, run the following commands: + +```ruby +es_url = Gitlab::CurrentSettings.current_application_settings + +# Confirm the current Elasticsearch URL +es_url.elasticsearch_url + +# Set the Elasticsearch URL +es_url.elasticsearch_url = "http://<username>:<password>@your.es.host:<port>" + +# Save the change +es_url.save! +``` + ## View logs One of the most valuable tools for identifying issues with the Elasticsearch @@ -58,8 +116,9 @@ There are a couple of ways to achieve that: ::Gitlab::CurrentSettings.elasticsearch_limit_indexing? # Whether or not Elasticsearch is limited only to certain projects/namespaces ``` -- Confirm searches use Elasticsearch by accessing the [rails console] - (../../administration/operations/rails_console.md) and running the following commands: +- Confirm searches use Elasticsearch by accessing the + [rails console](../../administration/operations/rails_console.md) and running the following + commands: ```rails u = User.find_by_email('email_of_user_doing_search') diff --git a/doc/integration/akismet.md b/doc/integration/akismet.md index 9a85511836f..a2b70d42bb6 100644 --- a/doc/integration/akismet.md +++ b/doc/integration/akismet.md @@ -1,6 +1,6 @@ --- -stage: Ecosystem -group: Integrations +stage: Anti-Abuse +group: Anti-Abuse info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -30,7 +30,7 @@ To use Akismet: 1. Sign in or create a new account. 1. Select **Show** to reveal the API key, and copy the API key's value. 1. Sign in to GitLab as an administrator. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Reporting** (`/admin/application_settings/reporting`). 1. Select the **Enable Akismet** checkbox. 1. Fill in the API key from step 3. diff --git a/doc/integration/cas.md b/doc/integration/cas.md index 38305967246..45c79cd9726 100644 --- a/doc/integration/cas.md +++ b/doc/integration/cas.md @@ -71,8 +71,8 @@ configure CAS for back-channel logout. 1. Save the configuration file. -1. [Reconfigure](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) or - [restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to - take effect if you installed GitLab via Omnibus or from source respectively. +1. For the changes to take effect: + - If you installed via Omnibus, [reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + - If you installed from source, [restart GitLab](../administration/restart_gitlab.md#installations-from-source). On the sign in page there should now be a CAS tab in the sign in form. diff --git a/doc/integration/datadog.md b/doc/integration/datadog.md index b8624545c41..42337006189 100644 --- a/doc/integration/datadog.md +++ b/doc/integration/datadog.md @@ -27,7 +27,7 @@ project, group, or instance level: 1. *For project-level or group-level integrations:* In GitLab, go to your project or group. 1. *For instance-level integrations:* 1. Sign in to GitLab as a user with administrator access. - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Integrations**. 1. Scroll to **Add an integration**, and select **Datadog**. 1. Select **Active** to enable the integration. diff --git a/doc/integration/ding_talk.md b/doc/integration/ding_talk.md index 437648b1adf..71dadd766b2 100644 --- a/doc/integration/ding_talk.md +++ b/doc/integration/ding_talk.md @@ -83,4 +83,6 @@ Sign in to DingTalk Open Platform and create an application on it. DingTalk gene 1. Save the configuration file. -1. [Reconfigure](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect if you installed GitLab via Omnibus or from source respectively. +1. For the changes to take effect: + - If you installed via Omnibus, [reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + - If you installed from source, [restart GitLab](../administration/restart_gitlab.md#installations-from-source). diff --git a/doc/integration/facebook.md b/doc/integration/facebook.md index 99ebce32936..ea5a3cc6d38 100644 --- a/doc/integration/facebook.md +++ b/doc/integration/facebook.md @@ -104,8 +104,9 @@ Facebook. Facebook generates an app ID and secret key for you to use. 1. Save the configuration file. -1. [Reconfigure](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect if you - installed GitLab via Omnibus or from source respectively. +1. For the changes to take effect: + - If you installed via Omnibus, [reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + - If you installed from source, [restart GitLab](../administration/restart_gitlab.md#installations-from-source). On the sign in page there should now be a Facebook icon below the regular sign in form. Select the icon to begin the authentication process. Facebook asks the diff --git a/doc/integration/gitpod.md b/doc/integration/gitpod.md index 271505d3d6f..c2b27e79d6e 100644 --- a/doc/integration/gitpod.md +++ b/doc/integration/gitpod.md @@ -47,7 +47,7 @@ For GitLab self-managed instances, a GitLab administrator needs to: 1. Set up a Gitpod instance to integrate with GitLab. Refer to the [Gitpod documentation](https://www.gitpod.io/docs/self-hosted/latest) to get your instance up and running. 1. Enable it in GitLab: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Gitpod** configuration section. 1. Select the **Enable Gitpod integration** checkbox. diff --git a/doc/integration/google.md b/doc/integration/google.md index 4862b898ed5..80176fac41b 100644 --- a/doc/integration/google.md +++ b/doc/integration/google.md @@ -117,10 +117,9 @@ On your GitLab server: ``` 1. Save the configuration file. -1. [Reconfigure](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) - or [restart GitLab](../administration/restart_gitlab.md#installations-from-source) for - the changes to take effect if you installed GitLab via Omnibus or from source - respectively. +1. For the changes to take effect: + - If you installed via Omnibus, [reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + - If you installed from source, [restart GitLab](../administration/restart_gitlab.md#installations-from-source). On the sign in page there should now be a Google icon below the regular sign in form. Select the icon to begin the authentication process. Google asks the diff --git a/doc/integration/jenkins.md b/doc/integration/jenkins.md index 5b779f22bd3..0a4c2c27c31 100644 --- a/doc/integration/jenkins.md +++ b/doc/integration/jenkins.md @@ -128,7 +128,7 @@ Configure the GitLab integration with Jenkins in one of the following ways. GitLab recommends this approach for Jenkins integrations because it is easier to configure than the [webhook integration](#configure-a-webhook). -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Jenkins**. 1. Select the **Active** checkbox. @@ -177,7 +177,7 @@ If you get this error message while configuring GitLab, the following are possib - GitLab is unable to reach your Jenkins instance at the address. If your GitLab instance is self-managed, try pinging the Jenkins instance at the domain provided on the GitLab instance. - The Jenkins instance is at a local address and is not included in the - [GitLab installation's allowlist](../security/webhooks.md#allowlist-for-local-requests). + [GitLab installation's allowlist](../security/webhooks.md#create-an-allowlist-for-local-requests). - The credentials for the Jenkins instance do not have sufficient access or are invalid. - The **Enable authentication for `/project` end-point** checkbox is not selected in your [Jenkin's plugin configuration](#configure-the-jenkins-server). diff --git a/doc/integration/jira/configure.md b/doc/integration/jira/configure.md index bfeac230f89..58789afff46 100644 --- a/doc/integration/jira/configure.md +++ b/doc/integration/jira/configure.md @@ -22,7 +22,7 @@ Prerequisites: To configure your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Jira**. 1. Select **Enable integration**. diff --git a/doc/integration/jira/development_panel.md b/doc/integration/jira/development_panel.md index 4d5a9f92257..d52d86c5658 100644 --- a/doc/integration/jira/development_panel.md +++ b/doc/integration/jira/development_panel.md @@ -38,7 +38,7 @@ After the integration is [set up on GitLab and Jira](#configure-the-integration) - Refer to any Jira issue by its ID (in uppercase) in GitLab branch names, commit messages, and merge request titles. - See the linked branches, commits, and merge requests in Jira issues. -- Create GitLab branches from Jira Cloud issues ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66032) in GitLab 14.2). +- Create GitLab branches from Jira Cloud issues ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66032) in GitLab 14.2 for the GitLab for Jira app). At this time, merge requests are called "pull requests" in Jira issues. This name may change in a future Jira release. @@ -85,9 +85,10 @@ documentation for [central administration of project integrations](../../user/ad ## Limitations -This integration is not supported on GitLab instances under a -[relative URL](https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab). -For example, `http://example.com/gitlab`. +- This integration is not supported on GitLab instances under a +[relative URL](https://docs.gitlab.com/omnibus/settings/configuration.html#configure-a-relative-url-for-gitlab) +(for example, `http://example.com/gitlab`). +- [Creating a branch](https://gitlab.com/gitlab-org/gitlab/-/issues/2647) is only supported by the GitLab for Jira app and is not available within the DVCS integration. See [officially supported DVCS features](https://confluence.atlassian.com/adminjiraserver/integrating-with-development-tools-938846890.html) for more information. ## Troubleshoot the development panel diff --git a/doc/integration/jira/dvcs.md b/doc/integration/jira/dvcs.md index 43a5349e0e5..ce097a4db23 100644 --- a/doc/integration/jira/dvcs.md +++ b/doc/integration/jira/dvcs.md @@ -174,7 +174,7 @@ Error obtaining access token. Cannot access https://gitlab.example.com from Jira - The [GitLab Jira integration](index.md) requires GitLab to connect to Jira. Any TLS issues that arise from a private certificate authority or self-signed certificate are resolved - [on the GitLab server](https://docs.gitlab.com/omnibus/settings/ssl.html#other-certificate-authorities), + [on the GitLab server](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates), as GitLab is the TLS client. - The Jira Development panel integration requires Jira to connect to GitLab, which causes Jira to be the TLS client. If your GitLab server's certificate is not @@ -277,7 +277,7 @@ In the example above, the merge requests feature is disabled. To resolve the issue, enable the relevant feature: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. Use the toggles to enable the features as needed. diff --git a/doc/integration/jira/issues.md b/doc/integration/jira/issues.md index 2d9e928e654..98dd4526fd9 100644 --- a/doc/integration/jira/issues.md +++ b/doc/integration/jira/issues.md @@ -54,10 +54,9 @@ You can [disable comments](#disable-comments-on-jira-issues) on issues. You can prevent merge requests from being merged if they do not refer to a Jira issue. To enforce this: -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. -1. Under **Merge checks**, select the **Require an associated issue from Jira** checkbox. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge checks** section, select **Require an associated issue from Jira**. 1. Select **Save**. After you enable this feature, a merge request that doesn't reference an associated diff --git a/doc/integration/kerberos.md b/doc/integration/kerberos.md index 257ba4e6708..5c9af96ebe8 100644 --- a/doc/integration/kerberos.md +++ b/doc/integration/kerberos.md @@ -110,13 +110,15 @@ set up GitLab to create a new account when a Kerberos user tries to sign in. ### Link a Kerberos account to an existing GitLab account +> Kerberos SPNEGO [renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96335) to Kerberos in GitLab 15.4. + If you're an administrator, you can link a Kerberos account to an existing GitLab account. To do so: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select a user, then select the **Identities** tab. -1. Select 'Kerberos SPNEGO' in the 'Provider' dropdown box. +1. From the **Provider** dropdown list, select **Kerberos**. 1. Make sure the **Identifier** corresponds to the Kerberos username. 1. Select **Save changes**. @@ -125,7 +127,7 @@ If you're not an administrator: 1. In the top-right corner, select your avatar. 1. Select **Edit profile**. 1. On the left sidebar, select **Account**. -1. In the **Service sign-in** section, select **Connect Kerberos SPNEGO**. +1. In the **Service sign-in** section, select **Connect Kerberos**. If you don't see a **Service sign-in** Kerberos option, follow the requirements in [Enable single sign-on](#enable-single-sign-on). @@ -153,7 +155,7 @@ With that information at hand: ``` 1. As an administrator, you can confirm the new, blocked account: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users** and review the **Blocked** tab. 1. You can enable the user. 1. If `block_auto_created_users` is false, the Kerberos user is @@ -305,15 +307,12 @@ We [deprecated](../update/deprecations.md#omniauth-kerberos-gem) password-based Kerberos sign-ins in GitLab 14.3 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/2908) it in GitLab 15.0. You must switch to ticket-based sign in. -Depending on your existing GitLab configuration, the 'Sign in with: -Kerberos SPNEGO' button may already be visible on your GitLab sign-in -page. If not, then add the settings [described above](#configuration). +Depending on your existing GitLab configuration, **Sign in with: +Kerberos** may already be visible on your GitLab sign-in page. +If not, then add the settings [described above](#configuration). -Once you have verified that the 'Kerberos SPNEGO' button works -without entering any passwords, you can proceed to disable -password-based Kerberos sign-ins. To do this you need only need to -remove the OmniAuth provider named `kerberos` from your `gitlab.yml` / -`gitlab.rb` file. +To disable password-based Kerberos sign-ins, remove the OmniAuth provider +`kerberos` from your `gitlab.yml`/`gitlab.rb` file. **For installations from source** @@ -365,9 +364,18 @@ mechanisms it supports to GitLab. If it doesn't support any of the mechanisms GitLab supports, authentication fails with a message like this in the log: ```plaintext -OmniauthKerberosSpnegoController: failed to process Negotiate/Kerberos authentication: gss_accept_sec_context did not return GSS_S_COMPLETE: An unsupported mechanism was requested Unknown error +OmniauthKerberosController: failed to process Negotiate/Kerberos authentication: gss_accept_sec_context did not return GSS_S_COMPLETE: An unsupported mechanism was requested Unknown error ``` +There are a number of potential causes and solutions for this error message. + +#### Kerberos integration not using a dedicated port + +GitLab CI/CD doesn’t work with a Kerberos-enabled GitLab instance unless the Kerberos integration +is configured to [use a dedicated port](kerberos.md#http-git-access-with-kerberos-token-passwordless-authentication). + +#### Lack of connectivity between client machine and Kerberos server + This is usually seen when the browser is unable to contact the Kerberos server directly. It falls back to an unsupported mechanism known as [`IAKERB`](https://k5wiki.kerberos.org/wiki/Projects/IAKERB), which tries to use @@ -377,6 +385,8 @@ If you're experiencing this error, ensure there is connectivity between the client machine and the Kerberos server - this is a prerequisite! Traffic may be blocked by a firewall, or the DNS records may be incorrect. +#### Mismatched forward and reverse DNS records for GitLab instance hostname + Another failure mode occurs when the forward and reverse DNS records for the GitLab server do not match. Often, Windows clients work in this case while Linux clients fail. They use reverse DNS while detecting the Kerberos @@ -389,6 +399,8 @@ match. So for instance, if you access GitLab as `gitlab.example.com`, resolving to IP address `1.2.3.4`, then `4.3.2.1.in-addr.arpa` must be a `PTR` record for `gitlab.example.com`. +#### Missing Kerberos libraries on browser or client machine + Finally, it's possible that the browser or client machine lack Kerberos support completely. Ensure that the Kerberos libraries are installed and that you can authenticate to other Kerberos services. diff --git a/doc/integration/mattermost/index.md b/doc/integration/mattermost/index.md index 3293732b59b..1e57e45aef3 100644 --- a/doc/integration/mattermost/index.md +++ b/doc/integration/mattermost/index.md @@ -79,7 +79,7 @@ mattermost_nginx['ssl_certificate'] = "/etc/gitlab/ssl/mattermost-nginx.crt" mattermost_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/mattermost-nginx.key" ``` -where `mattermost-nginx.crt` and `mattermost-nginx.key` are SSL cert and key, respectively. +where `mattermost-nginx.crt` is the SSL certificate and `mattermost-nginx.key` is the SSL key. Once the configuration is set, run `sudo gitlab-ctl reconfigure` to apply the changes. @@ -140,7 +140,7 @@ mattermost['gitlab_user_api_endpoint'] = "http://gitlab.example.com/api/v4/user" Save the changes and then run `sudo gitlab-ctl reconfigure`. If there are no errors your GitLab and GitLab Mattermost should be configured correctly. -### Specify numeric user and group identifiers +## Specify numeric user and group identifiers Omnibus GitLab creates a user and group `mattermost`. You can specify the numeric identifiers for these users in `/etc/gitlab/gitlab.rb` as follows: @@ -152,7 +152,7 @@ mattermost['gid'] = 1234 Run `sudo gitlab-ctl reconfigure` to apply the changes. -### Setting custom environment variables +## Setting custom environment variables If necessary you can set custom environment variables to be used by Mattermost via `/etc/gitlab/gitlab.rb`. This can be useful if the Mattermost server @@ -165,7 +165,7 @@ mattermost['env'] = {"HTTP_PROXY" => "my_proxy", "HTTPS_PROXY" => "my_proxy", "N Run `sudo gitlab-ctl reconfigure` to apply the changes. -### Connecting to the bundled PostgreSQL database +## Connecting to the bundled PostgreSQL database If you need to connect to the bundled PostgreSQL database and are using the default Omnibus GitLab database configuration, you can connect as the PostgreSQL superuser: @@ -174,14 +174,14 @@ the PostgreSQL superuser: sudo gitlab-psql -d mattermost_production ``` -### Back up GitLab Mattermost +## Back up GitLab Mattermost GitLab Mattermost is not included in the regular [Omnibus GitLab backup](../../raketasks/backup_restore.md) Rake task. The general Mattermost [backup and disaster recovery](https://docs.mattermost.com/deploy/backup-disaster-recovery.html) documentation can be used as a guide on what needs to be backed up. -#### Back up the bundled PostgreSQL database +### Back up the bundled PostgreSQL database If you need to back up the bundled PostgreSQL database and are using the default Omnibus GitLab database configuration, you can back up using this command: @@ -189,7 +189,7 @@ If you need to back up the bundled PostgreSQL database and are using the default sudo -i -u gitlab-psql -- /opt/gitlab/embedded/bin/pg_dump -h /var/opt/gitlab/postgresql mattermost_production | gzip > mattermost_dbdump_$(date --rfc-3339=date).sql.gz ``` -#### Back up the `data` directory and `config.json` +### Back up the `data` directory and `config.json` Mattermost has a `data` directory and `config.json` file that need to be backed up as well: @@ -197,7 +197,7 @@ Mattermost has a `data` directory and `config.json` file that need to be backed sudo tar -zcvf mattermost_data_$(date --rfc-3339=date).gz -C /var/opt/gitlab/mattermost data config.json ``` -### Restore GitLab Mattermost +## Restore GitLab Mattermost If you have previously [created a backup of GitLab Mattermost](#back-up-gitlab-mattermost), you can run the following commands to restore it: @@ -227,11 +227,11 @@ sudo chown mattermost:mattermost /var/opt/gitlab/mattermost/config.json sudo gitlab-ctl start mattermost ``` -### Mattermost Command Line Tools (CLI) +## Mattermost Command Line Tools (CLI) [`mmctl`](https://docs.mattermost.com/manage/mmctl-command-line-tool.html) is a CLI tool for the Mattermost server which is installed locally and uses the Mattermost API, but may also be used remotely. You must configure Mattermost either for local connections or authenticate as an administrator with local login credentials (not through GitLab SSO). The executable is located at `/opt/gitlab/embedded/bin/mmctl`. -#### Use `mmctl` through a local connection +### Use `mmctl` through a local connection For local connections, the `mmctl` binary and Mattermost must be run from the same server. To enable the local socket: @@ -269,7 +269,7 @@ wd3g5zpepjgbfjgpdjaas7yj6a: feedbackbot (feedbackbot@localhost) There are 4 users on local instance ``` -#### Use `mmctl` through a remote connection +### Use `mmctl` through a remote connection For remote connections or local connections where the socket cannot be used, create a non SSO user and give that user admin privileges. Those credentials @@ -522,7 +522,6 @@ sequenceDiagram For help and support around your GitLab Mattermost deployment please see: -- [Troubleshooting Forum](https://forum.mattermost.com/t/how-to-use-the-troubleshooting-forum/150) for configuration questions and issues. -- [Troubleshooting FAQ](https://docs.mattermost.com/install/troubleshooting.html). +- [Troubleshooting Mattermost issues](https://docs.mattermost.com/install/troubleshooting.html). - [Mattermost GitLab Issues Support Handbook](https://docs.mattermost.com/process/support.html?highlight=omnibus#gitlab-issues). - [GitLab Mattermost issue tracker](https://gitlab.com/gitlab-org/gitlab-mattermost/-/issues) for verified bugs with repro steps. diff --git a/doc/integration/oauth_provider.md b/doc/integration/oauth_provider.md index 962f5c4e5fb..21184f7b678 100644 --- a/doc/integration/oauth_provider.md +++ b/doc/integration/oauth_provider.md @@ -79,7 +79,7 @@ To add a new application for a group: To create an application for your GitLab instance: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Applications**. 1. Select **New application**. @@ -103,8 +103,11 @@ When applications are deleted, all grants and tokens associated with the applica ## Authorized applications -Every application you authorize with your GitLab credentials is shown -in the **Authorized applications** section under **Settings > Applications**. +To see all the application you've authorized with your GitLab credentials: + +1. On the top bar, in the top right corner, select your avatar. +1. Select **Edit profile** and then select **Applications**. +1. Scroll down to the **Authorized applications** section. The GitLab OAuth 2 applications support scopes, which allow various actions that any given application can perform. Available scopes are depicted in the following table. @@ -124,3 +127,13 @@ application can perform. Available scopes are depicted in the following table. | `email` | Grants read-only access to the user's primary email address using [OpenID Connect](openid_connect_provider.md). | At any time you can revoke any access by clicking **Revoke**. + +## Hashed OAuth application secrets + +> Introduced in GitLab 15.4 [with a flag](../administration/feature_flags.md) named `hash_oauth_secrets`. Disabled by default. + +FLAG: +On self-managed GitLab, by default, this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `hash_oauth_secrets`. +On GitLab.com, this feature is not available. + +By default, OAuth application secrets are stored as plain text in the database. When enabled, OAuth application secrets are stored in the database in hashed format and are only available to users immediately after creating OAuth applications. diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index e297c13a2da..0dfc78b508b 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -46,7 +46,7 @@ configure the settings that are common for all providers. Setting | Description | Default value ---------------------------|-------------|-------------- `allow_single_sign_on` | Enables you to list the providers that automatically create a GitLab account. The provider names are available in the **OmniAuth provider name** column in the [supported providers table](#supported-providers). | The default is `false`. If `false`, users must be created manually, or they can't sign in using OmniAuth. -`auto_link_ldap_user` | If enabled, creates an LDAP identity in GitLab for users that are created through an OmniAuth provider. You can enable this setting if you have the [LDAP (ActiveDirectory)](../administration/auth/ldap/index.md) integration enabled. Requires the `uid` of the user to be the same in both LDAP and the OmniAuth provider. | The default is `false`. +`auto_link_ldap_user` | If enabled, creates an LDAP identity in GitLab for users that are created through an OmniAuth provider. You can enable this setting if you have [LDAP integration](../administration/auth/ldap/index.md) enabled. Requires the `uid` of the user to be the same in both LDAP and the OmniAuth provider. | The default is `false`. `block_auto_created_users` | If enabled, blocks users that are automatically created from signing in until they are approved by an administrator. | The default is `true`. If you set the value to `false`, make sure you only define providers for `allow_single_sign_on` that you can control, like SAML or Google. Otherwise, any user on the internet can sign in to GitLab without an administrator's approval. To change these settings: @@ -195,7 +195,7 @@ By default, sign-in is enabled for all the OAuth providers configured in `config To enable or disable an OmniAuth provider: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings**. 1. Expand **Sign-in restrictions**. 1. In the **Enabled OAuth authentication sources** section, select or clear the checkbox for each provider you want to enable or disable. @@ -437,6 +437,34 @@ then override the icon in one of two ways: } ``` +## Change apps or configuration + +Because GitLab doesn't support having multiple providers in OAuth, GitLab configuration and user identification must be +updated at the same time if the provider or app is changed. + +These instructions apply to all methods of authentication where GitLab stores an `extern_uid` and it is the only data used +for user authentication. + +When changing apps within a provider, if the user `extern_uid` does not change, only the GitLab configuration must be +updated. + +To swap configurations: + +1. Change provider configuration in your `gitlab.rb` file. +1. Update `extern_uid` for all users that have an identity in GitLab for the previous provider. + +To find the `extern_uid`, look at an existing user's current `extern_uid` for an ID that matches the appropriate field in +your current provider for the same user. + +There are two methods to update the `extern_uid`: + +- Using the [Users API](../api/users.md#user-modification). Pass the provider name and the new `extern_uid`. +- Using the [Rails console](../administration/operations/rails_console.md): + + ```ruby + Identity.where(extern_uid: 'old-id').update!(extern_uid: 'new-id')` + ``` + ## Limitations Most supported OmniAuth providers don't support Git over HTTP password authentication. diff --git a/doc/integration/recaptcha.md b/doc/integration/recaptcha.md index 4963dea19a4..a5fd8db63bd 100644 --- a/doc/integration/recaptcha.md +++ b/doc/integration/recaptcha.md @@ -1,6 +1,6 @@ --- -stage: Ecosystem -group: Integrations +stage: Manage +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -17,7 +17,7 @@ To use reCAPTCHA, first create a site and private key. 1. Go to the [Google reCAPTCHA page](https://www.google.com/recaptcha/admin). 1. To get reCAPTCHA v2 keys, fill in the form and select **Submit**. 1. Sign in to your GitLab server as an administrator. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Reporting** (`admin/application_settings/reporting`). 1. Expand **Spam and Anti-bot Protection**. 1. In the reCAPTCHA fields, enter the keys you obtained in the previous steps. diff --git a/doc/integration/salesforce.md b/doc/integration/salesforce.md index 5300018e888..70d6e0aa0d8 100644 --- a/doc/integration/salesforce.md +++ b/doc/integration/salesforce.md @@ -81,9 +81,10 @@ To get the credentials (a pair of Client ID and Client Secret), you must [create  1. Save the configuration file. -1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) or - [restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes - to take effect if you installed GitLab via Omnibus or from source respectively. + +1. For the changes to take effect: + - If you installed via Omnibus, [reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + - If you installed from source, [restart GitLab](../administration/restart_gitlab.md#installations-from-source). On the sign in page, there should now be a Salesforce icon below the regular sign in form. Select the icon to begin the authentication process. Salesforce asks the user to sign in and authorize the GitLab application. diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 0c517d07f41..ef31f276025 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -143,7 +143,9 @@ as described in the section on [Security](#security). Otherwise, your users are 1. Change the value of `issuer` to a unique name, which identifies the application to the IdP. -1. For the changes to take effect, you must [reconfigure](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) GitLab if you installed via Omnibus or [restart GitLab](../administration/restart_gitlab.md#installations-from-source) if you installed from source. +1. For the changes to take effect: + - If you installed via Omnibus, [reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + - If you installed from source, [restart GitLab](../administration/restart_gitlab.md#installations-from-source). 1. Register the GitLab SP in your SAML 2.0 IdP, using the application name specified in `issuer`. @@ -169,7 +171,8 @@ is returned to GitLab and signed in. You can configure GitLab to use multiple SAML identity providers if: -- Each provider has a unique name set that matches a name set in `args`. +- Each provider has a unique name set that matches a name set in `args`. At least one provider **must** have the name `saml` to mitigate a + [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/366450) in GitLab 14.6 and newer. - The providers' names are: - Used in OmniAuth configuration for properties based on the provider name. For example, `allowBypassTwoFactor`, `allowSingleSignOn`, and `syncProfileFromProvider`. @@ -182,9 +185,9 @@ Example multiple providers configuration for Omnibus GitLab: ```ruby gitlab_rails['omniauth_providers'] = [ { - name: 'saml_1', + name: 'saml', args: { - name: 'saml_1', # This is mandatory and must match the provider name + name: 'saml', # This is mandatory and must match the provider name strategy_class: 'OmniAuth::Strategies::SAML', assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml_1/callback', # URL must match the name of the provider ... # Put here all the required arguments similar to a single provider @@ -192,9 +195,9 @@ gitlab_rails['omniauth_providers'] = [ label: 'Provider 1' # Differentiate the two buttons and providers in the UI }, { - name: 'saml_2', + name: 'saml1', args: { - name: 'saml_2', # This is mandatory and must match the provider name + name: 'saml1', # This is mandatory and must match the provider name strategy_class: 'OmniAuth::Strategies::SAML', assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml_2/callback', # URL must match the name of the provider ... # Put here all the required arguments similar to a single provider @@ -210,9 +213,9 @@ Example providers configuration for installations from source: omniauth: providers: - { - name: 'saml_1', + name: 'saml', args: { - name: 'saml_1', # This is mandatory and must match the provider name + name: 'saml', # This is mandatory and must match the provider name strategy_class: 'OmniAuth::Strategies::SAML', assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml_1/callback', # URL must match the name of the provider ... # Put here all the required arguments similar to a single provider @@ -220,9 +223,9 @@ omniauth: label: 'Provider 1' # Differentiate the two buttons and providers in the UI } - { - name: 'saml_2', + name: 'saml1', args: { - name: 'saml_2', # This is mandatory and must match the provider name + name: 'saml1', # This is mandatory and must match the provider name strategy_class: 'OmniAuth::Strategies::SAML', assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml_2/callback', # URL must match the name of the provider ... # Put here all the required arguments similar to a single provider @@ -800,8 +803,8 @@ If you have any questions on configuring the SAML app, please contact your provi ### Okta setup notes -1. In the Okta administrator section, make sure to select Classic UI view in the top left corner. From there, choose to **Add an App**. -1. When the app screen comes up you see another button to **Create an App** and +1. In the Okta administrator section choose **Applications**. +1. When the app screen comes up you see another button to **Create App Integration** and choose SAML 2.0 on the next screen. 1. Optionally, you can add a logo (you can choose it from <https://about.gitlab.com/press/>). You must @@ -859,117 +862,4 @@ connect to the Google Workspace SAML app. ## Troubleshooting -### SAML Response - -You can find the base64-encoded SAML Response in the [`production_json.log`](../administration/logs/index.md#production_jsonlog). This response is sent from the IdP, and contains user information that is consumed by GitLab. Many errors in the SAML integration can be solved by decoding this response and comparing it to the SAML settings in the GitLab configuration file. - -### GitLab+SAML Testing Environments - -To troubleshoot, [a complete GitLab+SAML testing environment using Docker compose](https://gitlab.com/gitlab-com/support/toolbox/replication/tree/master/compose_files) -is available. - -If you only require a SAML provider for testing, a [quick start guide to start a Docker container](../administration/troubleshooting/test_environments.md#saml) with a plug and play SAML 2.0 Identity Provider (IdP) is available. - -### 500 error after login - -If you see a "500 error" in GitLab when you are redirected back from the SAML -sign-in page, this could indicate that: - -- GitLab couldn't get the email address for the SAML user. Ensure the IdP provides a claim containing the user's - email address using the claim name `email` or `mail`. -- The certificate set your `gitlab.rb` file for `idp_cert_fingerprint` or `idp_cert` file is incorrect. -- Your `gitlab.rb` file is set to enable `idp_cert_fingerprint`, and `idp_cert` is being provided, or the reverse. - -### 422 error after login - -If you see a "422 error" in GitLab when you are redirected from the SAML -sign-in page, you might have an incorrectly configured Assertion Consumer -Service (ACS) URL on the identity provider. - -Make sure the ACS URL points to `https://gitlab.example.com/users/auth/saml/callback`, where -`gitlab.example.com` is the URL of your GitLab instance. - -If the ACS URL is correct, and you still have errors, review the other -[Troubleshooting](#troubleshooting) sections. - -If you are sure that the ACS URL is correct, proceed to the [Redirect back to the login screen with no evident error](#redirect-back-to-the-login-screen-with-no-evident-error) -section for further troubleshooting steps. - -### Redirect back to the login screen with no evident error - -If after signing in into your SAML server you are redirected back to the sign in page and -no error is displayed, check your `production.log` file. It most likely contains the -message `Can't verify CSRF token authenticity`. This means that there is an error during -the SAML request, but in GitLab 11.7 and earlier this error never reaches GitLab due to -the CSRF check. - -To bypass this you can add `skip_before_action :verify_authenticity_token` to the -`omniauth_callbacks_controller.rb` file immediately before the `after_action :verify_known_sign_in` line and -comment out the `protect_from_forgery` line using a `#`. Restart Puma for this -change to take effect. This allows the error to hit GitLab, where it can then -be seen in the usual logs, or as a flash message on the login screen. - -That file is located in `/opt/gitlab/embedded/service/gitlab-rails/app/controllers` -for Omnibus installations and by default in `/home/git/gitlab/app/controllers` for -installations from source. Restart Puma using the `sudo gitlab-ctl restart puma` -command on Omnibus installations and `sudo service gitlab restart` on installations -from source. - -You may also find the [SAML Tracer](https://addons.mozilla.org/en-US/firefox/addon/saml-tracer/) -(Firefox) and [SAML Chrome Panel](https://chrome.google.com/webstore/detail/saml-chrome-panel/paijfdbeoenhembfhkhllainmocckace) -(Chrome) browser extensions useful in your debugging. - -### Invalid audience - -This error means that the IdP doesn't recognize GitLab as a valid sender and -receiver of SAML requests. Make sure to: - -- Add the GitLab callback URL to the approved audiences of the IdP server. -- Avoid trailing whitespace in the `issuer` string. - -### Missing claims, or `Email can't be blank` errors - -The IdP server needs to pass certain information in order for GitLab to either -create an account, or match the login information to an existing account. `email` -is the minimum amount of information that needs to be passed. If the IdP server -is not providing this information, all SAML requests fail. - -Make sure this information is provided. - -Another issue that can result in this error is when the correct information is being sent by -the IdP, but the attributes don't match the names in the OmniAuth `info` hash. In this case, -you must set `attribute_statements` in the SAML configuration to -[map the attribute names in your SAML Response to the corresponding OmniAuth `info` hash names](#attribute_statements). - -### Key validation error, Digest mismatch or Fingerprint mismatch - -These errors all come from a similar place, the SAML certificate. SAML requests -must be validated using either a fingerprint, a certificate, or a validator. - -For this requirement, be sure to take the following into account: - -- If a fingerprint is used, it must be the SHA1 fingerprint -- If no certificate is provided in the settings, a fingerprint or fingerprint - validator needs to be provided and the response from the server must contain - a certificate (`<ds:KeyInfo><ds:X509Data><ds:X509Certificate>`) -- If a certificate is provided in the settings, it is no longer necessary for - the request to contain one. In this case the fingerprint or fingerprint - validators are optional - -If none of the above described scenarios is valid, the request -fails with one of the mentioned errors. - -### User is blocked when signing in through SAML - -The following are the most likely reasons that a user is blocked when signing in through SAML: - -- In the configuration, `gitlab_rails['omniauth_block_auto_created_users'] = true` is set and this is the user's first time signing in. -- There are [`required_groups`](#required-groups) configured, but the user is not a member of one. - -### Google workspace troubleshooting tips - -The Google Workspace documentation on [SAML app error messages](https://support.google.com/a/answer/6301076?hl=en) is helpful for debugging if you are seeing an error from Google while signing in. -Pay particular attention to the following 403 errors: - -- `app_not_configured` -- `app_not_configured_for_user` +See our [troubleshooting SAML guide](../user/group/saml_sso/troubleshooting.md). diff --git a/doc/integration/sourcegraph.md b/doc/integration/sourcegraph.md index 72ad0bcc32d..731c21c17fa 100644 --- a/doc/integration/sourcegraph.md +++ b/doc/integration/sourcegraph.md @@ -49,7 +49,7 @@ You can skip this step if you already have your GitLab repositories searchable i ### Configure your GitLab instance with Sourcegraph -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Sourcegraph** configuration section. 1. Check **Enable Sourcegraph**. diff --git a/doc/integration/twitter.md b/doc/integration/twitter.md index 2218529a729..aa9014adc49 100644 --- a/doc/integration/twitter.md +++ b/doc/integration/twitter.md @@ -88,7 +88,8 @@ Twitter. Twitter generates a client ID and secret key for you to use. 1. Save the configuration file. -1. [Reconfigure](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect if you - installed GitLab via Omnibus or from source respectively. +1. For the changes to take effect: + - If you installed via Omnibus, [reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + - If you installed from source, [restart GitLab](../administration/restart_gitlab.md#installations-from-source). On the sign in page there should now be a Twitter icon below the regular sign in form. Select the icon to begin the authentication process. Twitter asks the user to sign in and authorize the GitLab application. If everything goes well the user is returned to GitLab and signed in. diff --git a/doc/operations/error_tracking.md b/doc/operations/error_tracking.md index 08acf77b6c7..22e21c01fbd 100644 --- a/doc/operations/error_tracking.md +++ b/doc/operations/error_tracking.md @@ -46,7 +46,7 @@ least Maintainer [permissions](../user/permissions.md) to enable the Sentry inte Make sure to give the token at least the following scopes: `project:read`, `event:read`, and `event:write` (for resolving events). 1. In GitLab, enable error tracking: - 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Error Tracking**. 1. Select **Enable error tracking**. 1. In GitLab, ensure error tracking is active. @@ -136,10 +136,7 @@ FLAG: By default this feature is not available. To make it available on self-managed GitLab, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `integrated_error_tracking`. The feature is not ready for production use. -On GitLab.com, this feature is not available. - -WARNING: -Turning on integrated error tracking may impact performance, depending on your error rates. +On GitLab.com, please follow [our user guide](https://gitlab.com/gitlab-org/opstrace/opstrace/-/blob/main/docs/guides/user/error_tracking.md) to get started. Integrated error tracking is a lightweight alternative to Sentry backend. You still use Sentry SDK with your application. But you don't need to deploy Sentry diff --git a/doc/operations/feature_flags.md b/doc/operations/feature_flags.md index 9e7d452c259..0dccaa6bfe9 100644 --- a/doc/operations/feature_flags.md +++ b/doc/operations/feature_flags.md @@ -37,7 +37,7 @@ with GitLab, so it's up to developers to use a compatible client library and To create and enable a feature flag: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Feature Flags**. 1. Select **New feature flag**. 1. Enter a name that starts with a letter and contains only lowercase letters, digits, underscores (`_`), @@ -180,7 +180,7 @@ For example: To create a user list: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Feature Flags**. 1. Select **View user lists** 1. Select **New user list**. @@ -196,7 +196,7 @@ When viewing a list, you can rename it by selecting **Edit** (**{pencil}**). To add users to a user list: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Feature Flags**. 1. Select **Edit** (**{pencil}**) next to the list you want to add users to. 1. Select **Add Users**. @@ -210,7 +210,7 @@ To add users to a user list: To remove users from a user list: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Feature Flags**. 1. Select **Edit** (**{pencil}**) next to the list you want to change. 1. Select **Remove** (**{remove}**) next to the ID you want to remove. @@ -224,7 +224,7 @@ code so that you can clean it up when it's time to remove the feature flag. To search for code references of a feature flag: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Feature Flags**. 1. Edit the feature flag you want to remove. 1. Select **More actions** (**{ellipsis_v}**). @@ -235,7 +235,7 @@ To search for code references of a feature flag: In [GitLab 13.0 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/8621), to disable a feature flag for a specific environment: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Feature Flags**. 1. For the feature flag you want to disable, select **Edit** (**{pencil}**). 1. To disable the flag: @@ -250,7 +250,7 @@ to disable a feature flag for a specific environment: To disable a feature flag for all environments: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Feature Flags**. 1. For the feature flag you want to disable, slide the Status toggle to **Disabled**. @@ -265,7 +265,7 @@ Then prepare your application with a client library. To get the access credentials that your application needs to communicate with GitLab: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Feature Flags**. 1. Select **Configure** to view the following: - **API URL**: URL where the client (application) connects to get a list of feature flags. diff --git a/doc/operations/incident_management/alerts.md b/doc/operations/incident_management/alerts.md index a4b34807094..7e4223c0820 100644 --- a/doc/operations/incident_management/alerts.md +++ b/doc/operations/incident_management/alerts.md @@ -120,7 +120,7 @@ Prerequisite: To view the logs for an alert: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Alerts**. 1. Select the alert you want to view. 1. Below the title of the alert, select the **Metrics** tab. @@ -198,7 +198,7 @@ To assign an alert: 1. Display the list of current alerts: - 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Alerts**. 1. Select your desired alert to display its details. @@ -226,7 +226,7 @@ add a to-do item: 1. Display the list of current alerts: - 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Alerts**. 1. Select your desired alert to display its **Alert Management Details View**. diff --git a/doc/operations/incident_management/escalation_policies.md b/doc/operations/incident_management/escalation_policies.md index c24824e55f8..56ff733e395 100644 --- a/doc/operations/incident_management/escalation_policies.md +++ b/doc/operations/incident_management/escalation_policies.md @@ -22,7 +22,7 @@ Prerequisite: To create an escalation policy: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Escalation Policies**. 1. Select **Add an escalation policy**. 1. Enter the policy's name and description, and @@ -46,7 +46,7 @@ the paged users is created on the alert. To update an escalation policy: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Escalation Policies**. 1. Select **Edit escalation policy** (**{pencil}**). 1. Edit the information. @@ -56,7 +56,7 @@ To update an escalation policy: To delete an escalation policy: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Escalation Policies**. 1. Select **Delete escalation policy** (**{remove}**). 1. On the confirmation dialog, select **Delete escalation policy**. diff --git a/doc/operations/incident_management/incident_timeline_events.md b/doc/operations/incident_management/incident_timeline_events.md new file mode 100644 index 00000000000..743f9b429d6 --- /dev/null +++ b/doc/operations/incident_management/incident_timeline_events.md @@ -0,0 +1,94 @@ +--- +stage: Monitor +group: Respond +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Timeline events + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344059) in GitLab 15.2 [with a flag](../../administration/feature_flags.md) named `incident_timeline`. Enabled by default. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `incident_timeline`. +On GitLab.com, this feature is available. + +Incident timelines are an important part of record keeping for incidents. +Timelines can show executives and external viewers what happened during an incident, +and which steps were taken for it to be resolved. + +## View the timeline + +Incident timeline events are listed in ascending order of the date and time. +They are grouped with dates and are listed in ascending order of the time when they occurred: + + + +To view the event timeline of an incident: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Monitor > Incidents**. +1. Select an incident. +1. Select the **Timeline** tab. + +## Create an event + +You can create a timeline event in many ways in GitLab. + +### Using the form + +Create a timeline event manually using the form. + +Prerequisites: + +- You must have at least the Developer role for the project. + +To create a timeline event: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Monitor > Incidents**. +1. Select an incident. +1. Select the **Timeline** tab. +1. Select **Add new timeline event**. +1. Complete the required fields. +1. Select **Save** or **Save and add another event**. + +### Using a quick action + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368721) in GitLab 15.4. + +You can create a timeline event using the `/timeline` [quick action](../../user/project/quick_actions.md). + +### From a comment on the incident + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344058) in GitLab 15.4. + +Prerequisites: + +- You must have at least the Developer role for the project. + +To create a timeline event from a comment on the incident: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Monitor > Incidents**. +1. Select an incident. +1. Create a comment or choose an existing comment. +1. On the comment you want to add, select **Add comment to incident timeline** (**{clock}**). + +The comment is shown on the incident timeline as a timeline event. + +## Delete an event + +You can also delete timeline events. + +Prerequisites: + +- You must have at least the Developer role for the project. + +To delete a timeline event: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Monitor > Incidents**. +1. Select an incident. +1. Select the **Timeline** tab. +1. On the right of a timeline event, select **More actions** (**{ellipsis_v}**) and then select **Delete**. +1. To confirm, select **Delete Event**. diff --git a/doc/operations/incident_management/incidents.md b/doc/operations/incident_management/incidents.md index c1a4c1eb93e..2cb2e5f8045 100644 --- a/doc/operations/incident_management/incidents.md +++ b/doc/operations/incident_management/incidents.md @@ -205,64 +205,10 @@ field populated. ### Timeline events -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344059) in GitLab 15.2 [with a flag](../../administration/feature_flags.md) named `incident_timeline`. Enabled by default. +Incident timelines give a high-level overview of what happened +during an incident, and the steps that were taken for it to be resolved. -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `incident_timeline`. -On GitLab.com, this feature is available. - -Incident timelines are an important part of record keeping for incidents. -They give a high-level overview, to executives and external viewers, of what happened during the incident, -and the steps that were taken for it to be resolved. - -#### View the event timeline - -Incident timeline events are listed in ascending order of the date and time. -They are grouped with dates and are listed in ascending order of the time when they occured: - - - -To view the event timeline of an incident: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Monitor > Incidents**. -1. Select an incident. -1. Select the **Timeline** tab. - -#### Create a timeline event - -Create a timeline event manually using the form. - -Prerequisites: - -- You must have at least the Developer role for the project. - -To create a timeline event: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Monitor > Incidents**. -1. Select an incident. -1. Select the **Timeline** tab. -1. Select **Add new timeline event**. -1. Complete the required fields. -1. Select **Save** or **Save and add another event**. - -#### Delete a timeline event - -You can also delete timeline events. - -Prerequisites: - -- You must have at least the Developer role for the project. - -To delete a timeline event: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Monitor > Incidents**. -1. Select an incident. -1. Select the **Timeline** tab. -1. On the right of a timeline event, select **More actions** (**{ellipsis_v}**) and then select **Delete**. -1. To confirm, select **Delete Event**. +Read more about [timeline events](incident_timeline_events.md) and how to enable this feature. ### Recent updates view **(PREMIUM)** @@ -297,32 +243,28 @@ as a column in the Incidents List, and as a field on newly created Incidents. If the incident isn't closed before the SLA period ends, GitLab adds a `missed::SLA` label to the incident. -## Incident actions - -There are different actions available to help triage and respond to incidents. - -### Assign incidents +## Assign incidents Assign incidents to users that are actively responding. Select **Edit** in the right-hand side bar to select or clear assignees. -### Associate a milestone +## Associate a milestone Associate an incident to a milestone by selecting **Edit** next to the milestone feature in the right-hand side bar. -### Change severity +## Change severity See [Incident List](#incident-list) for a full description of the severity levels available. Select **Edit** in the right-hand side bar to change the severity of an incident. You can also change the severity using the [`/severity` quick action](../../user/project/quick_actions.md). -### Add a to-do item +## Add a to-do item Add a to-do for incidents that you want to track in your to-do list. Select **Add a to do** at the top of the right-hand side bar to add a to-do item. -### Change incident status +## Change incident status > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5716) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `incident_escalations`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/345769) in GitLab 14.10. @@ -343,7 +285,7 @@ In GitLab 15.1 and earlier, updating the status of an [incident created from an also updates the alert status. In [GitLab 15.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/356057), the alert status is independent and does not update when the incident status changes. -### Change escalation policy **(PREMIUM)** +## Change escalation policy **(PREMIUM)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5716) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `incident_escalations`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/345769) in GitLab 14.10. @@ -362,30 +304,30 @@ In GitLab 15.1 and earlier, the escalation policy for [incidents created from al reflects the alert's escalation policy and cannot be changed. In [GitLab 15.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/356057), the incident escalation policy is independent and can be changed. -### Manage incidents from Slack +## Manage incidents from Slack Slack slash commands allow you to control GitLab and view GitLab content without leaving Slack. Learn how to [set up Slack slash commands](../../user/project/integrations/slack_slash_commands.md) and how to [use the available slash commands](../../integration/slash_commands.md). -### Associate Zoom calls +## Associate Zoom calls GitLab enables you to [associate a Zoom meeting with an issue](../../user/project/issues/associate_zoom_meeting.md) for synchronous communication during incident management. After starting a Zoom call for an incident, you can associate the conference call with an issue. Your team members can join the Zoom call without requesting a link. -### Linked resources +## Linked resources -In an incident, you can [links to various resources](linked_resources.md), +In an incident, you can add [links to various resources](linked_resources.md), for example: - The incident Slack channel - Zoom meeting - Resources for resolving the incidents -### Embed metrics in incidents +## Embed metrics in incidents You can embed metrics anywhere [GitLab Markdown](../../user/markdown.md) is used, such as descriptions, comments on issues, and merge requests. Embedding @@ -398,7 +340,7 @@ You can embed both [GitLab-hosted metrics](../metrics/embed.md) and [Grafana metrics](../metrics/embed_grafana.md) in incidents and issue templates. -### Automatically close incidents via recovery alerts +## Automatically close incidents via recovery alerts > - [Introduced for Prometheus Integrations](https://gitlab.com/gitlab-org/gitlab/-/issues/13401) in GitLab 12.5. > - [Introduced for HTTP Integrations](https://gitlab.com/gitlab-org/gitlab/-/issues/13402) in GitLab 13.4. diff --git a/doc/operations/incident_management/linked_resources.md b/doc/operations/incident_management/linked_resources.md index d2254a30f91..3fe4a325cdb 100644 --- a/doc/operations/incident_management/linked_resources.md +++ b/doc/operations/incident_management/linked_resources.md @@ -29,7 +29,7 @@ Linked resources for an incident are listed under the **Summary** tab. To view the linked resources of an incident: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Incidents**. 1. Select an incident. @@ -43,24 +43,40 @@ Prerequisites: To add a linked resource: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Incidents**. 1. Select an incident. 1. In the **Linked resources** section, select the plus icon (**{plus-square}**). 1. Complete the required fields. 1. Select **Add**. +### Link Zoom meetings from an incident **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230853) in GitLab 15.4. + +Use the `/zoom` [quick action](../../user/project/quick_actions.md) to add multiple Zoom links to an incident: + +```plaintext +/zoom https://example.zoom.us/j/123456789 +``` + +You can also submit a short optional description with the link. The description shows instead of the URL in the **Linked resources** section of the incident issue: + +```plaintext +/zoom https://example.zoom.us/j/123456789, Low on memory incident +``` + ## Remove a linked resource You can also remove a linked resource. -Prerequisities: +Prerequisites: - You must have at least the Reporter role for the project. To remove a linked resource: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Incidents**. 1. Select an incident. 1. In the **Linked resources** section, select **Remove** (**{close}**). diff --git a/doc/operations/incident_management/oncall_schedules.md b/doc/operations/incident_management/oncall_schedules.md index 9b2e9159429..f1fb3503195 100644 --- a/doc/operations/incident_management/oncall_schedules.md +++ b/doc/operations/incident_management/oncall_schedules.md @@ -28,7 +28,7 @@ Prerequisite: To create an on-call schedule: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > On-call Schedules**. 1. Select **Add a schedule**. 1. Enter the schedule's name and description and select a time zone. @@ -43,7 +43,7 @@ create [rotations](#rotations) for your schedule. To update a schedule: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > On-call Schedules**. 1. Select **Edit schedule** (**{pencil}**). 1. Edit the information. @@ -56,7 +56,7 @@ interval (if one is set) to the corresponding times in the new time zone. To delete a schedule: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > On-call Schedules**. 1. Select **Delete escalation policy** (**{remove}**). 1. On the confirmation dialog, select **Delete schedule**. @@ -67,7 +67,7 @@ Add rotations to an existing schedule to put your team members on-call. To create a rotation: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > On-call Schedules**. 1. Select the **Add a rotation** link. 1. Enter the following information: @@ -85,7 +85,7 @@ To create a rotation: To edit a rotation: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > On-call Schedules**. 1. In the **Rotations** section, select **Edit rotation** (**{pencil}**). 1. Edit the information. @@ -95,7 +95,7 @@ To edit a rotation: To delete a rotation: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > On-call Schedules**. 1. In the **Rotations** section, select **Delete rotation** (**{remove}**). 1. On the confirmation dialog, select **Delete rotation**. diff --git a/doc/operations/incident_management/paging.md b/doc/operations/incident_management/paging.md index 3eeeb67bf51..837fc9c72f5 100644 --- a/doc/operations/incident_management/paging.md +++ b/doc/operations/incident_management/paging.md @@ -27,7 +27,7 @@ Email notifications are available in projects for triggered alerts. Project members with the **Owner** or **Maintainer** roles have the option to receive a single email notification for new alerts. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Monitor**. 1. Expand **Alerts**. 1. On the **Alert settings** tab, select the diff --git a/doc/operations/incident_management/status_page.md b/doc/operations/incident_management/status_page.md index fe75c1812c8..ae4d75396ae 100644 --- a/doc/operations/incident_management/status_page.md +++ b/doc/operations/incident_management/status_page.md @@ -45,7 +45,7 @@ Prerequisite: To provide GitLab with the AWS account information needed to push content to your Status Page: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Monitor**. 1. Expand **Status Page**. 1. Select the **Active** checkbox. @@ -96,7 +96,7 @@ the issue can potentially [publish comments to your GitLab Status Page](#publish After creating the CI/CD variables, configure the Project you want to use for Incident issues: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Monitor**. 1. Expand **Status page**. 1. Fill in your cloud provider's credentials and make sure to select the **Active** checkbox. diff --git a/doc/operations/metrics/alerts.md b/doc/operations/metrics/alerts.md index 12bd975db5d..6017e2ee16c 100644 --- a/doc/operations/metrics/alerts.md +++ b/doc/operations/metrics/alerts.md @@ -17,16 +17,6 @@ your team when environment performance falls outside of the boundaries you set. Alerts are not currently supported for [Prometheus cluster integrations](../../user/clusters/integrations.md). -<!--- start_remove The following content will be removed on remove_date: '2022-09-22' --> - -## External Prometheus instances (removed) - -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/219142) in GitLab 13.2 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/338834) in 15.0. -To manually configure a Prometheus server, we recommend -you use the [generic alerts integration](../incident_management/integrations.md). - -<!--- end_remove --> - ## Trigger actions from alerts **(ULTIMATE)** Alerts can be used to trigger actions, like opening an issue automatically diff --git a/doc/operations/metrics/embed_grafana.md b/doc/operations/metrics/embed_grafana.md index 17ff0ff01a3..1a1ac77ce23 100644 --- a/doc/operations/metrics/embed_grafana.md +++ b/doc/operations/metrics/embed_grafana.md @@ -14,7 +14,7 @@ embed Grafana panels using either: ## Use Grafana-rendered images -You can embed live [Grafana](https://docs.gitlab.com/omnibus/settings/grafana.html) panels as +You can embed live [Grafana](https://docs.gitlab.com/omnibus/settings/grafana.html) panels as [a direct link](https://grafana.com/docs/grafana/v7.5/sharing/share-panel/#use-direct-link). Your Grafana instance must: diff --git a/doc/operations/product_analytics.md b/doc/operations/product_analytics.md index 98ba6a9203c..e21770bc579 100644 --- a/doc/operations/product_analytics.md +++ b/doc/operations/product_analytics.md @@ -1,81 +1,45 @@ --- stage: Analytics -group: Product Intelligence +group: Product Analytics info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Product Analytics **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225167) in GitLab 13.3. -> - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. -> - It's able to be enabled or disabled per-project. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to enable it. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225167) in GitLab 13.3 [with a flag](../administration/feature_flags.md) named `product_analytics`. Disabled by default. -GitLab allows you to go from planning an application to getting feedback. Feedback -is not just observability, but also knowing how people use your product. -Product Analytics uses events sent from your application to know how they are using it. -It's based on [Snowplow](https://github.com/snowplow/snowplow), the best open-source -event tracker. With Product Analytics, you can receive and analyze the Snowplow data -inside GitLab. +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `product_analytics`. On GitLab.com, this feature is not available. The feature is not ready for production use. -## Enable or disable Product Analytics +GitLab enables you to go from planning an application to getting feedback. You can use +Product Analytics to receive and analyze events sent from your application. This analysis +provides observability information and feedback on how people use your product. -Product Analytics is under development and not ready for production use. It's -deployed behind a feature flag that's **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) -can enable it for your instance. Product Analytics can be enabled or disabled per-project. +Events are collected by a [Rails collector](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36443) and +then processed with [Snowplow](https://github.com/snowplow/snowplow). Events are stored in a GitLab database. -To enable it: +## View Product Analytics -```ruby -# Instance-wide -Feature.enable(:product_analytics) -# or by project -Feature.enable(:product_analytics, Project.find(<project ID>)) -``` +You can view the event data collected about your applications. -To disable it: +Prerequisite: -```ruby -# Instance-wide -Feature.disable(:product_analytics) -# or by project -Feature.disable(:product_analytics, Project.find(<project ID>)) -``` +- You must have at least the Reporter role. -## Access Product Analytics +To access Product Analytics: -After enabling the feature flag for Product Analytics, you can access the -user interface: +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Monitor > Product Analytics**. -1. Sign in to GitLab as a user with at least the Reporter role. -1. Navigate to **Monitor > Product Analytics**. +The Product Analytics interface contains: -The user interface contains: +- An Events tab that shows the recent events and a total count. +- A Graph tab that shows graphs based on events of the last 30 days. +- A Test tab that sends a sample event payload. +- A Setup page containing the code to implement in your application. -- An Events page that shows the recent events and a total count. -- A test page that sends a sample event. -- A setup page containing the code to implement in your application. - -## Rate limits for Product Analytics +## Rate limits While Product Analytics is under development, it's rate-limited to **100 events per minute** per project. This limit prevents the events table in the database from growing too quickly. - -## Data storage for Product Analytics - -Product Analytics stores events are stored in GitLab database. - -WARNING: -This data storage is experimental, and GitLab is likely to remove this data during -future development. - -## Event collection - -Events are collected by [Rails collector](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36443), -allowing GitLab to ship the feature fast. Due to scalability issue, GitLab plans -to switch to a separate application, such as -[snowplow-go-collector](https://gitlab.com/gitlab-org/snowplow-go-collector), for event collection. diff --git a/doc/policy/maintenance.md b/doc/policy/maintenance.md index e77a0459150..02a1c5a53dc 100644 --- a/doc/policy/maintenance.md +++ b/doc/policy/maintenance.md @@ -12,11 +12,11 @@ patch, and security releases. New releases are announced on the [GitLab blog](ht Our current policy is: -- Backporting bug fixes for **only the current stable release** at any given time. (See [patch releases](#patch-releases).) -- Backporting security fixes **to the previous two monthly releases in addition to the current stable release**. (See [security releases](#security-releases).) +- Backporting bug fixes for **only the current stable release** at any given time - see [patch releases](#patch-releases) below. +- Backporting security fixes **to the previous two monthly releases in addition to the current stable release**. In some circumstances (outlined in [security releases](#security-releases) below) we may address a security vulnerability using the [patch release](#patch-releases) process or regular monthly release process, that is, providing an update to the current stable release only, with no backports. In rare cases, release managers may make an exception and backport to more than -the last two monthly releases. See +the last two monthly releases. See [Backporting to older releases](#backporting-to-older-releases) for more information. ## Versioning @@ -132,13 +132,16 @@ To request backporting to more than one stable release for consideration, raise ### Security releases Security releases are a special kind of patch release that only include security -fixes and patches (see below) for the previous two monthly releases in addition to the current stable release. +fixes and patches for the previous two monthly releases in addition to the current stable release. For very serious security issues, there is [precedent](https://about.gitlab.com/releases/2016/05/02/cve-2016-4340-patches/) to backport security fixes to even more monthly releases of GitLab. This decision is made on a case-by-case basis. +In some circumstances we may choose to address a vulnerability using the [patch release](#patch-releases) process or the regular monthly release process, that is, updating the current stable release only, with no backports. Factors influencing this decision include very low likelihood of exploitation, low impact, fix complexity and risk to stability. We will **always address +high and critical** security issues with a security release. + ## More information You may also want to read our: diff --git a/doc/raketasks/backup_gitlab.md b/doc/raketasks/backup_gitlab.md index 4629364ce3d..0a38416825a 100644 --- a/doc/raketasks/backup_gitlab.md +++ b/doc/raketasks/backup_gitlab.md @@ -35,6 +35,11 @@ WARNING: The backup command requires [additional parameters](backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer) when your installation is using PgBouncer, for either performance reasons or when using it with a Patroni cluster. +WARNING: +The backup command doesn't verify if another backup is already running, as described in +[issue 362593](https://gitlab.com/gitlab-org/gitlab/-/issues/362593). We strongly recommend +you make sure that all backups are complete before starting a new one. + Depending on your version of GitLab, use the following command if you installed GitLab using the Omnibus package: @@ -262,7 +267,7 @@ sudo -u git -H bundle exec rake gitlab:backup:create SKIP=db,uploads RAILS_ENV=p ### Skipping tar creation NOTE: -It is not possible to skip the tar creation when using [object storage](#uploading-backups-to-a-remote-cloud-storage) for backups. +It is not possible to skip the tar creation when using [object storage](#upload-backups-to-a-remote-cloud-storage) for backups. The last part of creating a backup is generation of a `.tar` file containing all the parts. In some cases (for example, if the backup is picked up by other @@ -341,12 +346,14 @@ To create an incremental backup, run: sudo gitlab-backup create INCREMENTAL=yes PREVIOUS_BACKUP=<timestamp_of_backup> ``` -Incremental backups can also be created from [an untarred backup](#skipping-tar-creation) by using `SKIP=tar`: +To create an [untarred](#skipping-tar-creation) incremental backup from a tarred backup, use `SKIP=tar`: ```shell sudo gitlab-backup create INCREMENTAL=yes SKIP=tar ``` +You can't create an incremental backup from an [untarred](#skipping-tar-creation) backup. + ### Back up specific repository storages > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86896) in GitLab 15.0. @@ -391,7 +398,7 @@ For example, to back up all repositories for all projects in **Group A** (`group sudo -u git -H bundle exec rake gitlab:backup:create REPOSITORIES_PATHS=group-a,group-b/project-c ``` -### Uploading backups to a remote (cloud) storage +### Upload backups to a remote (cloud) storage NOTE: It is not possible to [skip the tar creation](#skipping-tar-creation) when using object storage for backups. @@ -401,7 +408,7 @@ the `.tar` file it creates. In the following example, we use Amazon S3 for storage, but Fog also lets you use [other storage providers](https://fog.io/storage/). GitLab also [imports cloud drivers](https://gitlab.com/gitlab-org/gitlab/-/blob/da46c9655962df7d49caef0e2b9f6bbe88462a02/Gemfile#L113) for AWS, Google, OpenStack Swift, Rackspace, and Aliyun. A local driver is -[also available](#uploading-to-locally-mounted-shares). +[also available](#upload-to-locally-mounted-shares). [Read more about using object storage with GitLab](../administration/object_storage.md). @@ -452,8 +459,8 @@ gitlab_rails['backup_upload_storage_options'] = { ##### SSE-KMS -To enable SSE-KMS, you'll need the -[KMS key via its Amazon Resource Name (ARN) in the `arn:aws:kms:region:acct-id:key/key-id` format](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html). +To enable SSE-KMS, you'll need the +[KMS key via its Amazon Resource Name (ARN) in the `arn:aws:kms:region:acct-id:key/key-id` format](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html). Under the `backup_upload_storage_options` configuration setting, set: - `server_side_encryption` to `aws:kms`. @@ -722,7 +729,7 @@ Users of GitLab 12.1 and earlier should use the command `gitlab-rake gitlab:back ### Skip uploading backups to remote storage -If you have configured GitLab to [upload backups in a remote storage](#uploading-backups-to-a-remote-cloud-storage), +If you have configured GitLab to [upload backups in a remote storage](#upload-backups-to-a-remote-cloud-storage), you can use the `SKIP=remote` option to skip uploading your backups to the remote storage. For Omnibus GitLab packages: @@ -737,23 +744,40 @@ For installations from source: sudo -u git -H bundle exec rake gitlab:backup:create SKIP=remote RAILS_ENV=production ``` -### Uploading to locally mounted shares +### Upload to locally-mounted shares + +You can send backups to a locally-mounted share (for example, `NFS`,`CIFS`, or `SMB`) using the Fog +[`Local`](https://github.com/fog/fog-local#usage) storage provider. + +To do this, you must set the following configuration keys: + +- `backup_upload_connection.local_root`: mounted directory that backups are copied to. +- `backup_upload_remote_directory`: subdirectory of the `backup_upload_connection.local_root` directory. It is created if it doesn't exist. + If you want to copy the tarballs to the root of your mounted directory, use `.`. -You may also send backups to a mounted share (for example, `NFS`,`CIFS`, or -`SMB`) by using the Fog [`Local`](https://github.com/fog/fog-local#usage) -storage provider. The directory pointed to by the `local_root` key _must_ be -owned by the `git` user _when mounted_ (mounting with the `uid=` of the `git` -user for `CIFS` and `SMB`) or the user that you are executing the backup tasks -as (for Omnibus packages, this is the `git` user). +When mounted, the directory set in the `local_root` key must be owned by either: -The `backup_upload_remote_directory` _must_ be set in addition to the -`local_root` key. This is the sub directory inside the mounted directory that -backups are copied to, and is created if it does not exist. If the -directory that you want to copy the tarballs to is the root of your mounted -directory, use `.` instead. +- The `git` user. So, mounting with the `uid=` of the `git` user for `CIFS` and `SMB`. +- The user that you are executing the backup tasks as. For Omnibus GitLab, this is the `git` user. Because file system performance may affect overall GitLab performance, -[GitLab doesn't recommend using cloud-based file systems for storage](../administration/nfs.md#avoid-using-cloud-based-file-systems). +[we don't recommend using cloud-based file systems for storage](../administration/nfs.md#avoid-using-cloud-based-file-systems). + +#### Avoid conflicting configuration + +Don't set the following configuration keys to the same path: + +- `gitlab_rails['backup_path']` (`backup.path` for source installations). +- `gitlab_rails['backup_upload_connection'].local_root` (`backup.upload.connection.local_root` for source installations). + +The `backup_path` configuration key sets the local location of the backup file. The `upload` configuration key is +intended for use when the backup file is uploaded to a separate server, perhaps for archival purposes. + +If these configuration keys are set to the same location, the upload feature fails because a backup already exists at +the upload location. This failure causes the upload feature to delete the backup because it assumes it's a residual file +remaining after the failed upload attempt. + +#### Configure uploads to locally-mounted shares For Omnibus GitLab packages: @@ -878,7 +902,7 @@ for backups. The next time the backup task runs, backups older than the `backup_ pruned. This configuration option manages only local files. GitLab doesn't prune old -files stored in a third-party [object storage](#uploading-backups-to-a-remote-cloud-storage) +files stored in a third-party [object storage](#upload-backups-to-a-remote-cloud-storage) because the user may not have permission to list and delete files. It's recommended that you configure the appropriate retention policy for your object storage (for example, [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html)). diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 878511b3e14..03413aca2af 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -12,7 +12,7 @@ An application data backup creates an archive file that contains the database, all repositories and all attachments. You can only restore a backup to **exactly the same version and type (CE/EE)** -of GitLab on which it was created. The best way to +of GitLab on which it was created. The best way to [migrate your projects from one server to another](#migrate-to-a-new-server) is through a backup and restore. WARNING: @@ -79,10 +79,18 @@ For detailed information on restoring GitLab, see [Restore GitLab](restore_gitla ## Alternative backup strategies -If your GitLab instance contains a lot of Git repository data, you may find the -GitLab backup script to be too slow. If your GitLab instance has a lot of forked -projects, the regular backup task also duplicates the Git data for all of them. -In these cases, consider using file system snapshots as part of your backup strategy. +In the following cases, consider using file system data transfer or snapshots as part of your backup strategy: + +- Your GitLab instance contains a lot of Git repository data and the GitLab backup script is too slow. +- Your GitLab instance has a lot of forked projects and the regular backup task duplicates the Git data for all of them. +- Your GitLab instance has a problem and using the regular backup and import Rake tasks isn't possible. + +When considering using file system data transfer or snapshots: + +- Don't use these methods to migrate from one operating system to another. The operating systems of the source and destination should be as similar as possible. For example, + don't use these methods to migrate from Ubuntu to Fedora. +- Data consistency is very important. We recommend stopping GitLab with `sudo gitlab-ctl stop` before taking doing a file system transfer (with rsync, for example) or taking a + snapshot. Example: Amazon Elastic Block Store (EBS) @@ -190,7 +198,7 @@ tables will [be logged by PostgreSQL](../administration/logs/index.md#postgresql ERROR: relation "tablename" does not exist at character 123 ``` -This happens because the task uses `pg_dump`, which +This happens because the task uses `pg_dump`, which [sets a null search path and explicitly includes the schema in every SQL query](https://gitlab.com/gitlab-org/gitlab/-/issues/23211) to address [CVE-2018-1058](https://www.postgresql.org/about/news/postgresql-103-968-9512-9417-and-9322-released-1834/). @@ -318,7 +326,7 @@ To prepare the new server: ``` 1. Disable periodic background jobs: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. Under the Sidekiq dashboard, select **Cron** tab and then **Disable All**. @@ -398,7 +406,7 @@ To prepare the new server: 1. [Restore the GitLab backup](#restore-gitlab). 1. Verify that the Redis database restored correctly: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. Under the Sidekiq dashboard, verify that the numbers match with what was shown on the old server. diff --git a/doc/raketasks/restore_gitlab.md b/doc/raketasks/restore_gitlab.md index 9d9a4d7b8c8..7b3a60b436c 100644 --- a/doc/raketasks/restore_gitlab.md +++ b/doc/raketasks/restore_gitlab.md @@ -157,6 +157,8 @@ the restore target directories are empty. For both these installation types, the backup tarball has to be available in the backup location (default location is `/var/opt/gitlab/backups`). +If you use Docker Swarm, [first disable the health check](#restore-gitlab-from-backup-using-docker-swarm). + For Docker installations, the restore task can be run from host: ```shell @@ -188,6 +190,20 @@ issue. The GitLab Helm chart uses a different process, documented in [restoring a GitLab Helm chart installation](https://gitlab.com/gitlab-org/charts/gitlab/blob/master/doc/backup-restore/restore.md). +### Restore GitLab from backup using Docker Swarm + +Docker Swarm might restart the container during the restore process because Puma is shut down, +and so the container health check fails. To work around this problem, disable the health check +mechanism in Docker compose file: + +```yaml +healthcheck: + disable: true +``` + +Read more in issue #6846, +[GitLab restore can fail owing to `gitlab-healthcheck`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6846). + ## Restore for installation from source First, ensure your backup tar file is in the backup directory described in the diff --git a/doc/security/img/allowlist_v13_0.png b/doc/security/img/allowlist_v13_0.png Binary files differdeleted file mode 100644 index 973b53a57a4..00000000000 --- a/doc/security/img/allowlist_v13_0.png +++ /dev/null diff --git a/doc/security/img/outbound_requests_section_v12_2.png b/doc/security/img/outbound_requests_section_v12_2.png Binary files differdeleted file mode 100644 index 3dc99868a35..00000000000 --- a/doc/security/img/outbound_requests_section_v12_2.png +++ /dev/null diff --git a/doc/security/information_exclusivity.md b/doc/security/information_exclusivity.md index 754d5fff843..2eeb436316f 100644 --- a/doc/security/information_exclusivity.md +++ b/doc/security/information_exclusivity.md @@ -24,7 +24,7 @@ limitation. You can take steps to prevent unintentional sharing and information destruction. This limitation is the reason why only certain people are allowed to [add users to a project](../user/project/members/index.md) -and why only a GitLab administrator can +and why only a GitLab administrator can [force push a protected branch](../user/project/protected_branches.md). <!-- ## Troubleshooting diff --git a/doc/security/password_length_limits.md b/doc/security/password_length_limits.md index 04c3a5c99e1..57466d1ed5d 100644 --- a/doc/security/password_length_limits.md +++ b/doc/security/password_length_limits.md @@ -30,7 +30,7 @@ The user password length is set to a minimum of 8 characters by default. To change the minimum password length using GitLab UI: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General** and expand **Sign-up restrictions**.  diff --git a/doc/security/ssh_keys_restrictions.md b/doc/security/ssh_keys_restrictions.md index eca52c41e4f..138ebb9858a 100644 --- a/doc/security/ssh_keys_restrictions.md +++ b/doc/security/ssh_keys_restrictions.md @@ -20,7 +20,7 @@ limit the allowed SSH key algorithms. GitLab allows you to restrict the allowed SSH key technology as well as specify the minimum key length for each technology: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General** (`/admin/application_settings/general`). 1. Expand the **Visibility and access controls** section: diff --git a/doc/security/token_overview.md b/doc/security/token_overview.md index a2119c86268..e585f2caeca 100644 --- a/doc/security/token_overview.md +++ b/doc/security/token_overview.md @@ -124,17 +124,16 @@ This table shows available scopes per token. Scopes can be limited further on to ## Security considerations -Access tokens should be treated like passwords and kept secure. - -Adding them to URLs is a security risk. This is especially true when cloning or adding a remote, as Git then writes the URL to its `.git/config` file in plain text. URLs are also generally logged by proxies and application servers, which makes those credentials visible to system administrators. - -Instead, API calls can be passed an access token using headers, like [the `Private-Token` header](../api/index.md#personalprojectgroup-access-tokens). - -Tokens can also be stored using a [Git credential storage](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). - -Tokens should not be committed to your source code. Instead, consider an approach such as [using external secrets in CI](../ci/secrets/index.md). - -When creating a scoped token, consider using the most limited scope possible to reduce the impact of accidentally leaking the token. - -When creating a token, consider setting a token that expires when your task is complete. For example, if performing a one-off import, set the -token to expire after a few hours or a day. This reduces the impact of a token that is accidentally leaked because it is useless when it expires. +- Access tokens should be treated like passwords and kept secure. +- Adding access tokens to URLs is a security risk, especially when cloning or adding a remote because Git then writes the URL to its `.git/config` file in plain text. URLs are + also generally logged by proxies and application servers, which makes those credentials visible to system administrators. Instead, pass API calls an access token using + headers like [the `Private-Token` header](../api/index.md#personalprojectgroup-access-tokens). +- Tokens can also be stored using a [Git credential storage](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). +- Tokens should not be committed to your source code. Instead, consider an approach such as [using external secrets in CI](../ci/secrets/index.md). +- When creating a scoped token, consider using the most limited scope possible to reduce the impact of accidentally leaking the token. +- When creating a token, consider setting a token that expires when your task is complete. For example, if performing a one-off import, set the + token to expire after a few hours or a day. This reduces the impact of a token that is accidentally leaked because it is useless when it expires. +- Be careful not to include tokens when pasting code, console commands, or log outputs into an issue or MR description or comment. +- Don’t log credentials in the console logs. Consider [protecting](../ci/variables/index.md#protected-cicd-variables) and + [masking](../ci/variables/index.md#mask-a-cicd-variable) your credentials. +- Review all currently active access tokens of all types on a regular basis and revoke any that are no longer needed. diff --git a/doc/security/two_factor_authentication.md b/doc/security/two_factor_authentication.md index 93d77a69f0e..c808cf4e321 100644 --- a/doc/security/two_factor_authentication.md +++ b/doc/security/two_factor_authentication.md @@ -26,7 +26,7 @@ cannot leave the 2FA configuration area at `/-/profile/two_factor_auth`. To enable 2FA for all users: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General** (`/admin/application_settings/general`). 1. Expand the **Sign-in restrictions** section, where you can configure both. @@ -52,7 +52,7 @@ Prerequisites: To enforce 2FA only for certain groups: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. 1. Select **All users in this group must set up two-factor authentication**. diff --git a/doc/security/unlock_user.md b/doc/security/unlock_user.md index efe9c5784ad..041527f18af 100644 --- a/doc/security/unlock_user.md +++ b/doc/security/unlock_user.md @@ -14,7 +14,7 @@ Users are locked after ten failed sign-in attempts. These users remain locked: ## Unlock a user from the Admin Area -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Use the search bar to find the locked user. 1. From the **User administration** dropdown select **Unlock**. diff --git a/doc/security/user_email_confirmation.md b/doc/security/user_email_confirmation.md index 54920b15362..172e26db618 100644 --- a/doc/security/user_email_confirmation.md +++ b/doc/security/user_email_confirmation.md @@ -11,7 +11,7 @@ GitLab can be configured to require confirmation of a user's email address when the user signs up. When this setting is enabled, the user is unable to sign in until they confirm their email address. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General** (`/admin/application_settings/general`). 1. Expand the **Sign-up restrictions** section and look for the **Send confirmation email on sign-up** option. diff --git a/doc/security/user_file_uploads.md b/doc/security/user_file_uploads.md index 7c11d01396d..ddb8392d2be 100644 --- a/doc/security/user_file_uploads.md +++ b/doc/security/user_file_uploads.md @@ -39,7 +39,7 @@ Prerequisite: To configure authentication settings for all media files: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. Scroll to **Project visibility** and select **Require authentication to view media files**. diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md index f2066ee4a42..e1daa705355 100644 --- a/doc/security/webhooks.md +++ b/doc/security/webhooks.md @@ -7,85 +7,80 @@ type: concepts, reference, howto # Webhooks and insecure internal web services **(FREE SELF)** -NOTE: -On GitLab.com, the [maximum number of webhooks and their size](../user/gitlab_com/index.md#webhooks) per project, and per group, is limited. - -If you have non-GitLab web services running on your GitLab server or within its -local network, these may be vulnerable to exploitation via Webhooks. - -With [Webhooks](../user/project/integrations/webhooks.md), you and your project -maintainers and owners can set up URLs to be triggered when specific changes -occur in your projects. Normally, these requests are sent to external web -services specifically set up for this purpose, that process the request and its -attached data in some appropriate way. - -Things get hairy, however, when a Webhook is set up with a URL that doesn't -point to an external, but to an internal service, that may do something -completely unintended when the webhook is triggered and the POST request is -sent. - -Webhook requests are made by the GitLab server itself and use a single -(optional) secret token per hook for authorization (instead of a user or -repository-specific token). As a result, these requests may have broader access than -intended, including access to everything running on the server hosting the webhook. This -may include the GitLab server or API itself (for example, `http://localhost:123`). -Depending on the called webhook, this may also result in network access -to other servers within that webhook server's local network (for example, -`http://192.168.1.12:345`), even if these services are otherwise protected -and inaccessible from the outside world. - -If a web service does not require authentication, Webhooks can be used to -trigger destructive commands by getting the GitLab server to make POST requests -to endpoints like `http://localhost:123/some-resource/delete`. - -To prevent this type of exploitation from happening, starting with GitLab 10.6, -all Webhook requests to the current GitLab instance server address and/or in a -private network are forbidden by default. That means that all requests made -to `127.0.0.1`, `::1` and `0.0.0.0`, as well as IPv4 `10.0.0.0/8`, `172.16.0.0/12`, -`192.168.0.0/16` and IPv6 site-local (`ffc0::/10`) addresses aren't allowed. - -This behavior can be overridden: - -1. On the top bar, select **Menu > Admin**. +Users with at least the Maintainer role can set up [webhooks](../user/project/integrations/webhooks.md) that are +triggered when specific changes occur in a project. When triggered, a `POST` HTTP request is sent to a URL. A webhook is +usually configured to send data to a specific external web service, which processes the data in an appropriate way. + +However, a webhook can be configured with a URL for an internal web service instead of an external web service. +When the webhook is triggered, non-GitLab web services running on your GitLab server or in its local network could be +exploited. + +Webhook requests are made by the GitLab server itself and use a single optional secret token per hook for authorization +instead of: + +- A user token. +- A repository-specific token. + +As a result, these requests can have broader access than intended, including access to everything running on the server +that hosts the webhook including: + +- The GitLab server. +- The API itself. +- For some webhooks, network access to other servers in that webhook server's local network, even if these services + are otherwise protected and inaccessible from the outside world. + +Webhooks can be used to trigger destructive commands using web services that don't require authentication. These webhooks +can get the GitLab server to make `POST` HTTP requests to endpoints that delete resources. + +## Allow webhook and service requests to local network + +To prevent exploitation of insecure internal web services, all webhook requests to the following local network addresses are not allowed: + +- The current GitLab instance server address. +- Private network addresses, including `127.0.0.1`, `::1`, `0.0.0.0`, `10.0.0.0/8`, `172.16.0.0/12`, + `192.168.0.0/16`, and IPv6 site-local (`ffc0::/10`) addresses. + +To allow access to these addresses: + +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. -1. Expand the **Outbound requests** section: -  -1. Select **Allow requests to the local network from web hooks and services**. +1. Expand **Outbound requests**. +1. Select the **Allow requests to the local network from web hooks and services** checkbox. -NOTE: -*System hooks* are enabled to make requests to local network by default since they are -set up by administrators. However, you can turn this off by disabling the -**Allow requests to the local network from system hooks** option. +## Prevent system hook requests to local network -## Allowlist for local requests +[System hooks](../administration/system_hooks.md) are permitted to make requests to local network by default because +they are set up by administrators. To prevent system hook requests to the local network: -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/44496) in GitLab 12.2 +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > Network**. +1. Expand **Outbound requests**. +1. Clear the **Allow requests to the local network from system hooks** checkbox. -You can allow certain domains and IP addresses to be accessible to both *system hooks* -and *webhooks* even when local requests are not allowed by adding them to the -allowlist: +## Create an allowlist for local requests -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Settings > Network** (`/admin/application_settings/network`) - and expand **Outbound requests**: +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/44496) in GitLab 12.2 -  +You can allow certain domains and IP addresses to be accessible to both system hooks and webhooks, even when local +requests are forbidden. To add these domains to the allowlist: -The allowed entries can be separated by semicolons, commas or whitespaces -(including newlines) and be in different formats like hostnames, IP addresses and/or -IP ranges. IPv6 is supported. Hostnames that contain Unicode characters should -use [Internationalized Domain Names in Applications](https://www.icann.org/en/icann-acronyms-and-terms/internationalized-domain-names-in-applications-en) -(IDNA) encoding. +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > Network**. +1. Expand **Outbound requests** and add entries. -The allowlist can hold a maximum of 1000 entries. Each entry can be a maximum of -255 characters. +The entries can: -You can allow a particular port by specifying it in the allowlist entry. -For example `127.0.0.1:8080` only allows connections to port 8080 on `127.0.0.1`. -If no port is mentioned, all ports on that IP/domain are allowed. An IP range -allows all ports on all IPs in that range. +- Be separated by semicolons, commas, or whitespaces (including newlines). +- Be in different formats like hostnames, IP addresses, IP address ranges. IPv6 is supported. Hostnames that contain + Unicode characters should use [Internationalized Domain Names in Applications](https://www.icann.org/en/icann-acronyms-and-terms/internationalized-domain-names-in-applications-en) + (IDNA) encoding. +- Include ports. For example, `127.0.0.1:8080` only allows connections to port 8080 on `127.0.0.1`. If no port is specified, + all ports on that IP address or domain are allowed. An IP address range allows all ports on all IP addresses in that + range. +- Number no more than 1000 entries of no more than 255 characters for each entry. +- Not contain wildcards (for example, `*.example.com`). -Example: +For example: ```plaintext example.com;gitlab.example.com @@ -96,9 +91,6 @@ example.com;gitlab.example.com example.com:8080 ``` -NOTE: -Wildcards (`*.example.com`) are not currently supported. - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/subscriptions/gitlab_com/index.md b/doc/subscriptions/gitlab_com/index.md index 6b83a00cac1..aed8bafb780 100644 --- a/doc/subscriptions/gitlab_com/index.md +++ b/doc/subscriptions/gitlab_com/index.md @@ -48,7 +48,7 @@ Prerequisite: To see the status of your GitLab SaaS subscription: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Billing**. The following information is displayed: @@ -98,7 +98,7 @@ In this case, they would see only the features available to that subscription. To view a list of seats being used: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Usage Quotas**. 1. On the **Seats** tab, view usage information. @@ -128,7 +128,7 @@ For example: To export seat usage data as a CSV file: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Billing**. 1. Under **Seats currently in use**, select **See usage**. 1. Select **Export list**. @@ -179,7 +179,7 @@ The following is emailed to you: To remove a billable user from your subscription: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Billing**. 1. In the **Seats currently in use** section, select **See usage**. 1. In the row for the user you want to remove, on the right side, select the ellipsis and **Remove user**. @@ -299,7 +299,7 @@ for your personal or group namespace. CI/CD minutes are a **one-time purchase**, NOTE: Free namespaces are subject to a 5GB storage and 10GB transfer [soft limit](https://about.gitlab.com/pricing/). Once all storage is available to view in the usage quota workflow, GitLab will automatically enforce the namespace storage limit and the project limit will be removed. This change will be announced separately. The storage and transfer add-on can be purchased to increase the limits. -Projects have a free storage quota of 10 GB. To exceed this quota you must first +Projects have a free storage quota of 10 GB. To exceed this quota you must first [purchase one or more storage subscription units](#purchase-more-storage-and-transfer). Each unit provides 10 GB of additional storage per namespace. A storage subscription is renewed annually. For more details, see [Usage Quotas](../../user/usage_quotas.md). @@ -346,7 +346,7 @@ main quota. You can find pricing for additional storage on the To purchase additional storage for your group on GitLab SaaS: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Usage Quotas**. 1. Select **Storage** tab. 1. Select **Purchase more storage**. diff --git a/doc/subscriptions/gitlab_dedicated/index.md b/doc/subscriptions/gitlab_dedicated/index.md index 218f6b7f824..e5f75fd5a7a 100644 --- a/doc/subscriptions/gitlab_dedicated/index.md +++ b/doc/subscriptions/gitlab_dedicated/index.md @@ -20,13 +20,13 @@ GitLab Dedicated enables you to offload the operational overhead of managing the - Authentication: Support for instance-level [SAML OmniAuth](../../integration/saml.md) functionality. GitLab Dedicated acts as the service provider, and you must provide the necessary [configuration](../../integration/saml.md#general-setup) in order for GitLab to communicate with your IdP. This is provided during onboarding. SAML [request signing](../../integration/saml.md#request-signing-optional) is supported. - Networking: - - Public connectivity + - Public connectivity with support for IP Allowlists. During onboarding, you can optionally specify a list of IP addresses that can access your Dedicated instance. Subsequently, when an IP not on the allowlist tries to access your instance the connection will be refused. - Optional. Private connectivity via [AWS PrivateLink](https://aws.amazon.com/privatelink/). - You can specify an AWS IAM Principal and preferred Availability Zones during onboarding to enable this functionality. -- Upgrade strategy: + You can specify an AWS IAM Principal and preferred Availability Zones during onboarding to enable this functionality. Both Ingress and Egress Private Links are supported. When connecting to an internal service running in your VPC over https via PrivateLink, Dedicated supports the ability to use a private SSL certificate, which can be provided during onboarding. +- Upgrades: - Monthly upgrades tracking one release behind the latest (n-1), with the latest security release. - Out of band security patches provided for high severity releases. -- Backup strategy: regular backups taken and tested. +- Backups: regular backups taken and tested. - Choice of cloud region: upon onboarding, choose the cloud region where you want to deploy your instance. Some AWS regions have limited features and as a result, we are not able to deploy production instances to those regions. See below for the [full list of regions](#aws-regions-not-supported) not currently supported. - Security: Data encrypted at rest and in transit using latest encryption standards. - Application: Self-managed [Ultimate feature set](https://about.gitlab.com/pricing/self-managed/feature-comparison/) with the exception of the unsupported features [listed below](#features-not-available-at-launch). diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md index ed96fbd91ef..e71954f1968 100644 --- a/doc/subscriptions/index.md +++ b/doc/subscriptions/index.md @@ -109,7 +109,7 @@ Purchases in the Customers Portal require a credit card on record as a payment m multiple credit cards to your account, so that purchases for different products are charged to the correct card. -If you would like to use an alternative method to pay, please +If you would like to use an alternative method to pay, please [contact our Sales team](https://about.gitlab.com/sales/). To change your payment method: @@ -194,7 +194,7 @@ GitLab for Open Source Program benefits apply to an entire GitLab namespace. To To add a license: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the overview page, select **Add LICENSE**. If the license you want is not available as a license template, manually copy the entire, unaltered [text of your chosen license](https://opensource.org/licenses/alphabetical) into the `LICENSE` file. Note that GitLab defaults to **All rights reserved** if users do not perform this action. Applicants must add the correct license to each project in their respective groups or namespaces When you're sure you're using OSI-approved licenses for your projects, you can take your screenshots. @@ -211,7 +211,7 @@ Benefits of the GitLab Open Source Program apply to all projects in a GitLab nam ##### Screenshot 1: License overview -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select your project avatar. If you haven't specified an avatar for your project, the avatar displays as a single letter. 1. Take a screenshot of the project overview that clearly displays the license you've chosen for your project. @@ -219,7 +219,7 @@ Benefits of the GitLab Open Source Program apply to all projects in a GitLab nam ##### Screenshot 2: License contents -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository** and locate the project's `LICENSE` file. 1. Take a screenshot of the contents of the file. Make sure the screenshot includes the title of the license. @@ -229,7 +229,7 @@ Benefits of the GitLab Open Source Program apply to all projects in a GitLab nam To be eligible for the GitLab Open Source Program, projects must be publicly visible. To check your project's public visibility settings: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. From the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. From the **Project visibility** dropdown list, select **Public**. diff --git a/doc/subscriptions/self_managed/index.md b/doc/subscriptions/self_managed/index.md index 758b472d67b..4ae38de54bc 100644 --- a/doc/subscriptions/self_managed/index.md +++ b/doc/subscriptions/self_managed/index.md @@ -49,7 +49,7 @@ Prorated charges are not possible without a quarterly usage report. You can view users for your license and determine if you've gone over your subscription. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left menu, select **Subscription**. The lists of users are displayed. @@ -218,7 +218,7 @@ to IP address `104.18.26.123:443` (`customers.gitlab.com`). You can manually sync your subscription details at any time. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Subscription**. 1. In the **Subscription details** section, select **Sync subscription details**. @@ -241,7 +241,7 @@ instance, ensure you're purchasing enough seats to If you are an administrator, you can view the status of your subscription: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Subscription**. The **Subscription** page includes the following details: @@ -265,7 +265,7 @@ It also displays the following information: If you are an administrator, you can export your license usage into a CSV: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Subscription**. 1. In the top right, select **Export license usage file**. diff --git a/doc/topics/application_development_platform/index.md b/doc/topics/application_development_platform/index.md index fac9f963a98..524ba2aaf6d 100644 --- a/doc/topics/application_development_platform/index.md +++ b/doc/topics/application_development_platform/index.md @@ -1,67 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: '../../user/infrastructure/index.md' +remove_date: '2022-12-08' --- -# Application Development Platform **(FREE)** +This document was moved to [another location](../../user/infrastructure/index.md). -The GitLab Application Development Platform refers to the set of GitLab features used to create, configure, and manage -a complete software development environment. It provides development, operations, and security teams with a robust feature set aimed at supporting best practices out of the box. - -## Overview - -The GitLab Application Development Platform aims to: - -- Reduce and even eliminate the time it takes for an Operations team - to provide a full environment for software developers. -- Get developers up and running fast so they can focus on writing - great applications with a robust development feature set. -- Provide best-of-breed security features so that applications developed - with GitLab are not affected by vulnerabilities that may lead to security - problems and unintended use. - -It is comprised of the following high-level elements: - -1. Compute -1. Build, test, and deploy a wide range of applications -1. Security -1. Observability - -We believe the use of these common building blocks equate to big gains for teams of all sizes, resulting from the adoption -of newer, more efficient, more profitable, and less error-prone techniques for shipping software applications. - -### Compute - -Because at GitLab we are [cloud-native first](https://about.gitlab.com/handbook/product/#cloud-native-first) our -Application Development Platform initially focuses on providing robust support for Kubernetes, with other platforms -to follow. Teams can bring their own clusters and we additionally make it easy to create new infrastructure -with various cloud providers. - -### Build, test, deploy - -In order to provide modern DevOps workflows, our Application Development Platform relies on -[Auto DevOps](../autodevops/index.md) to provide those workflows. Auto DevOps works with -any Kubernetes cluster; you're not limited to running on GitLab infrastructure. Additionally, Auto DevOps offers -an incremental consumption path. Because it is [composable](../autodevops/customize.md#using-components-of-auto-devops), -you can use as much or as little of the default pipeline as you'd like, and deeply customize without having to integrate a completely different platform. - -### Security - -The Application Development Platform helps you ensure that the applications you create are not affected by vulnerabilities -that may lead to security problems and unintended use. This can be achieved by making use of the embedded security features of Auto DevOps, -which inform security teams and developers if there is something to consider changing in their apps -before it is too late to create a preventative fix. The following features are included: - -- [Auto SAST (Static Application Security Testing)](../autodevops/stages.md#auto-sast) -- [Auto Dependency Scanning](../autodevops/stages.md#auto-dependency-scanning) -- [Auto Container Scanning](../autodevops/stages.md#auto-container-scanning) -- [Auto DAST (Dynamic Application Security Testing)](../autodevops/stages.md#auto-dast) - -### Observability - -Performance is a critical aspect of the user experience, and ensuring your application is responsive and available is everyone's -responsibility. The Application Development Platform integrates key performance analytics and feedback -into GitLab, automatically. The following features are included: - -- [Auto Monitoring](../autodevops/stages.md#auto-monitoring) +<!-- This redirect file can be deleted after <2022-12-08>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_ecs.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_ecs.md index a9b466653bf..4c084f405cd 100644 --- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_ecs.md +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_ecs.md @@ -13,7 +13,7 @@ You can choose to target AWS ECS as a deployment platform instead of using Kuber To get started on Auto DevOps to AWS ECS, you must add a specific CI/CD variable. To do so, follow these steps: -1. In GitLab, on the top bar, select **Menu > Projects** and find your project. +1. In GitLab, on the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Auto DevOps**. 1. Specify which AWS platform to target during the Auto DevOps deployment diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md index 1fea573faa1..5f37ca30604 100644 --- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md @@ -138,7 +138,7 @@ While Auto DevOps is enabled by default, Auto DevOps can be disabled at both the instance level (for self-managed instances) and the group level. Complete these steps to enable Auto DevOps if it's disabled: -1. On the top bar, select **Menu > Projects** and find the application project. +1. On the top bar, select **Main menu > Projects** and find the application project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Auto DevOps**. 1. Select **Default to Auto DevOps pipeline** to display more options. diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index 7f9707a6939..4d97f012a92 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -425,6 +425,7 @@ applications. | `ROLLOUT_RESOURCE_TYPE` | Allows specification of the resource type being deployed when using a custom Helm chart. Default value is `deployment`. | | `ROLLOUT_STATUS_DISABLED` | From GitLab 12.0, used to disable rollout status check because it does not support all resource types, for example, `cronjob`. | | `STAGING_ENABLED` | Used to define a [deploy policy for staging and production environments](#deploy-policy-for-staging-and-production-environments). | +| `TRACE` | Set to any value to make Helm commands produce verbose output. You can use this setting to help diagnose Auto DevOps deployment problems. | NOTE: After you set up your replica variables using a @@ -512,7 +513,7 @@ these prefixed variables available to the deployed application as environment va To configure your application variables: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Variables**. 1. Create a CI/CD variable, ensuring the key is prefixed with diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index dc58f42f30e..c21ed5ed444 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -12,15 +12,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w GitLab Auto DevOps is a collection of pre-configured features and integrations that work together to support your software delivery process. -Auto DevOps features and integrations: +Auto DevOps detects your programming language and uses [CI/CD templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates) +to create and run default pipelines to build and test your application. Then, you can [configure deployments](requirements.md) to deploy your apps to staging +and production, and set up [Review Apps](stages.md#auto-review-apps) +to preview your changes per branch. -- Detect your code's language. -- Build and test your application. -- Measure code quality. -- Scan for vulnerabilities and security flaws. -- Check for licensing issues. -- Monitor in real time. -- Deploy your application. +You can use default settings to quickly ship your apps, and iterate and [customize](customize.md) later. + +You can also [manage Auto DevOps with APIs](customize.md#extend-auto-devops-with-the-api). <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an introduction to Auto DevOps, watch [Auto DevOps in GitLab 11.0](https://youtu.be/0Tc0YYBxqi4). @@ -58,33 +57,6 @@ Based on the DevOps [stages](stages.md), use Auto DevOps to: - [Auto Static Application Security Testing (SAST)](stages.md#auto-sast) - [Auto Secret Detection](stages.md#auto-secret-detection) -### How it works - -Auto DevOps detects your code language and uses [CI/CD templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates) -to create and run default pipelines. All you need to kick it off is to -[enable](#enable-or-disable-auto-devops) it. - -Auto DevOps starts by building and testing your application. Then, based on your -[predefined deployment configuration](requirements.md), -creates the necessary jobs to deploy your apps to staging -and/or production. It also sets up [Review Apps](stages.md#auto-review-apps) -so that you can preview your changes in a per-branch basis. - -Note that you don't need to set up the deployment upfront. Auto DevOps -still builds and tests your application. You can define the deployment later. - -Auto DevOps avoids the hassle of having to create entire pipelines manually. -Keep it simple and facilitate an iterative approach: ship your app first, -then explore the [customizations](customize.md) later. -You can also [manage Auto DevOps with APIs](customize.md#extend-auto-devops-with-the-api). - -Some of the benefits of using Auto DevOps as part of your workflow are: - -- Consistency: always start from default templates. -- Simplicity: create your pipeline with the default settings first, iterate later. -- Productivity: deploy multiple apps in a short period of time. -- Efficiency: get things done fast. - ### Comparison to application platforms and PaaS Auto DevOps provides features often included in an application @@ -146,15 +118,15 @@ you can enable it for a [group](#at-the-group-level) or an [instance](#at-the-instance-level). This can save you the time of enabling it one by one. -Only project Maintainers can enable or disable Auto DevOps at the project level. +Prerequisites: -Before enabling Auto DevOps, ensure that your project does not have a -`.gitlab-ci.yml` present. If present, your CI/CD configuration takes -precedence over the Auto DevOps pipeline. +- You must have at least the Maintainer role for the project. +- Ensure your project does not have a `.gitlab-ci.yml` present. If present, your CI/CD configuration takes + precedence over the Auto DevOps pipeline. To enable Auto DevOps for a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Auto DevOps**. 1. Select the **Default to Auto DevOps pipeline** checkbox. @@ -178,12 +150,13 @@ rather than enabling individually for each subgroup or project. When enabled for a group, you can still disable Auto DevOps for the subgroups and projects where you don't want to use it. -Only GitLab administrators and group owners can enable or disable Auto DevOps -at the group level. +Prerequisites: + +- You must have at least the Owner role for the group. To enable Auto DevOps for a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Auto DevOps**. 1. Select the **Default to Auto DevOps pipeline** checkbox. @@ -193,9 +166,9 @@ To disable Auto DevOps on the group level, follow the same process and clear the **Default to Auto DevOps pipeline** checkbox. After enabling Auto DevOps at the group level, you can trigger the -Auto DevOps pipeline for any project that belongs to that group. To do so: +Auto DevOps pipeline for any project that belongs to that group: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Make sure the project doesn't contain a `.gitlab-ci.yml` file. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. To trigger the Auto DevOps pipeline, select **Run pipeline**. @@ -207,15 +180,16 @@ instance become enabled. This is convenient when you want to run Auto DevOps by default for all projects. You can still disable Auto DevOps individually for the groups and projects where you don't want to run it. -Only GitLab administrators can enable or disable Auto DevOps at the instance -level. - -Even when disabled for an instance, group owners and project maintainers +Even when disabled for an instance, group Owners and project Maintainers can still enable Auto DevOps at the group and project levels. +Prerequisites: + +- You must be an administrator for the instance. + To enable Auto DevOps for your instance: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Auto DevOps**. 1. Select the **Default to Auto DevOps pipeline** checkbox. @@ -232,6 +206,13 @@ it remains unchanged and Auto DevOps doesn't affect it. To disable Auto DevOps in the instance level, follow the same process and clear the **Default to Auto DevOps pipeline** checkbox. +### Private registry support + +There is no guarantee that you can use a private container registry with Auto DevOps. + +Instead, use the [GitLab Container Registry](../../user/packages/container_registry/index.md) with Auto DevOps to +simplify configuration and prevent any unforeseen issues. + ### Quick start - [Use Auto DevOps to deploy to a Kubernetes cluster on Google Kubernetes Engine (GKE)](cloud_deployments/auto_devops_with_gke.md) @@ -252,16 +233,7 @@ match your new GitLab version: - Environment variables. - [Upgrading PostgreSQL](upgrading_postgresql.md). -## Limitations - -### Private registry support - -We cannot guarantee that you can use a private container registry with Auto DevOps. - -We strongly advise you to use GitLab Container Registry with Auto DevOps to -simplify configuration and prevent any unforeseen issues. - -### Install applications behind a proxy +## Install applications behind a proxy The GitLab integration with Helm does not support installing applications when behind a proxy. diff --git a/doc/topics/autodevops/prepare_deployment.md b/doc/topics/autodevops/prepare_deployment.md index 22a50df7f54..c16a6c837cd 100644 --- a/doc/topics/autodevops/prepare_deployment.md +++ b/doc/topics/autodevops/prepare_deployment.md @@ -44,7 +44,7 @@ To define the base domain, either: - In the project, group, or instance level: go to your cluster settings and add it there. - In the project or group level: add it as an environment variable: `KUBE_INGRESS_BASE_DOMAIN`. -- In the instance level: go to **Menu > Admin > Settings > CI/CD > Continuous Integration and Delivery** and add it there. +- In the instance level: go to **Main menu > Admin > Settings > CI/CD > Continuous Integration and Delivery** and add it there. The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence as other environment [variables](../../ci/variables/index.md#cicd-variable-precedence). diff --git a/doc/topics/autodevops/requirements.md b/doc/topics/autodevops/requirements.md index f3ea13ad1ce..9dffb490807 100644 --- a/doc/topics/autodevops/requirements.md +++ b/doc/topics/autodevops/requirements.md @@ -41,7 +41,7 @@ that works best for your needs: You can choose the deployment method when enabling Auto DevOps or later: -1. In GitLab, on the top bar, select **Menu > Projects** and find your project. +1. In GitLab, on the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Auto DevOps**. 1. Choose the deployment strategy. @@ -61,7 +61,7 @@ To define the base domain, either: - In the project, group, or instance level: go to your cluster settings and add it there. - In the project or group level: add it as an environment variable: `KUBE_INGRESS_BASE_DOMAIN`. -- In the instance level: go to **Menu > Admin > Settings > CI/CD > Continuous Integration and Delivery** and add it there. +- In the instance level: go to **Main menu > Admin > Settings > CI/CD > Continuous Integration and Delivery** and add it there. The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence as other environment [variables](../../ci/variables/index.md#cicd-variable-precedence). diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index 4e8d0e08eff..62356807854 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -199,7 +199,7 @@ see the documentation. ## Auto Secret Detection > - Introduced in GitLab 13.1. -> - Select functionality [made available](../../user/application_security/secret_detection/index.md#making-secret-detection-available-to-all-gitlab-tiers) in all tiers in GitLab 13.3 +> - Select functionality [made available](../../user/application_security/secret_detection/index.md#features-per-tier) in all tiers in GitLab 13.3 Secret Detection uses the [Secret Detection Docker image](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) to run Secret Detection on the current code, and checks for leaked secrets. Auto Secret Detection requires [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or above. diff --git a/doc/topics/autodevops/troubleshooting.md b/doc/topics/autodevops/troubleshooting.md index 045f843be44..bf3dc27c0e8 100644 --- a/doc/topics/autodevops/troubleshooting.md +++ b/doc/topics/autodevops/troubleshooting.md @@ -9,6 +9,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w The information in this documentation page describes common errors when using Auto DevOps, and any available workarounds. +## Trace Helm commands + +Set the CI/CD variable `TRACE` to any value to make Helm commands produce verbose output. You can use this output to diagnose Auto DevOps deployment problems. + +You can resolve some problems with Auto DevOps deployment by changing advanced Auto DevOps configuration variables. Read more about [customizing Auto DevOps CI/CD variables](customize.md#cicd-variables). + ## Unable to select a buildpack Auto Build and Auto Test may fail to detect your language or framework with the diff --git a/doc/topics/build_your_application.md b/doc/topics/build_your_application.md index d7097e55052..d48b8383271 100644 --- a/doc/topics/build_your_application.md +++ b/doc/topics/build_your_application.md @@ -12,5 +12,5 @@ code, and use CI/CD to generate your application. Include packages in your app a - [Repositories](../user/project/repository/index.md) - [Merge requests](../user/project/merge_requests/index.md) - [CI/CD](../ci/index.md) -- [Packages & Registries](../user/packages/index.md) +- [Packages and registries](../user/packages/index.md) - [Application infrastructure](../user/infrastructure/index.md) diff --git a/doc/topics/git/cherry_picking.md b/doc/topics/git/cherry_picking.md index 98458133937..d9314c3becc 100644 --- a/doc/topics/git/cherry_picking.md +++ b/doc/topics/git/cherry_picking.md @@ -17,8 +17,8 @@ and apply those changes to another branch. Cherry-picks can help you: You can cherry-pick commits from the command line. In the GitLab user interface, you can also: -- Cherry-pick [all changes from a merge request](../../user/project/merge_requests/cherry_pick_changes.md#cherry-pick-a-merge-request). -- Cherry-pick [a single commit](../../user/project/merge_requests/cherry_pick_changes.md#cherry-pick-a-commit). +- Cherry-pick [all changes from a merge request](../../user/project/merge_requests/cherry_pick_changes.md#cherry-pick-all-changes-from-a-merge-request). +- Cherry-pick [a single commit](../../user/project/merge_requests/cherry_pick_changes.md#cherry-pick-a-single-commit). - Cherry-pick [from a fork to the upstream repository](../../user/project/merge_requests/cherry_pick_changes.md#cherry-pick-into-a-project). ## Cherry-pick from the command line diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 54af1e99797..3e78e366e00 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -36,7 +36,7 @@ The following resources can help you get started with Git: - [GitLab Git Cheat Sheet (download)](https://about.gitlab.com/images/press/git-cheat-sheet.pdf) - Commits: - [Revert a commit](../../user/project/merge_requests/revert_changes.md#revert-a-commit) - - [Cherry-picking a commit](../../user/project/merge_requests/cherry_pick_changes.md#cherry-pick-a-commit) + - [Cherry-picking a commit](../../user/project/merge_requests/cherry_pick_changes.md) - [Squashing commits](../gitlab_flow.md#squashing-commits-with-rebase) - [Squash-and-merge](../../user/project/merge_requests/squash_and_merge.md) - [Signing commits](../../user/project/repository/gpg_signed_commits/index.md) diff --git a/doc/topics/git/lfs/migrate_to_git_lfs.md b/doc/topics/git/lfs/migrate_to_git_lfs.md index 864615e7264..dda15845088 100644 --- a/doc/topics/git/lfs/migrate_to_git_lfs.md +++ b/doc/topics/git/lfs/migrate_to_git_lfs.md @@ -9,7 +9,7 @@ description: "How to migrate an existing Git repository to Git LFS with BFG." WARNING: The following documentation is deprecated. We recommend using -[`git lfs migrate`](https://github.com/git-lfs/git-lfs/blob/main/docs/man/git-lfs-migrate.1.ronn) +[`git lfs migrate`](https://github.com/git-lfs/git-lfs/blob/main/docs/man/git-lfs-migrate.adoc) instead of the method documented below. Using Git LFS can help you to reduce the size of your Git diff --git a/doc/topics/git/numerous_undo_possibilities_in_git/index.md b/doc/topics/git/numerous_undo_possibilities_in_git/index.md index 9786d1399f7..0ed7b2b5e03 100644 --- a/doc/topics/git/numerous_undo_possibilities_in_git/index.md +++ b/doc/topics/git/numerous_undo_possibilities_in_git/index.md @@ -209,7 +209,7 @@ To recover from multiple incorrect commits: The commits are now `A-B-C-D-E`. Alternatively, with GitLab, -you can [cherry-pick](../../../user/project/merge_requests/cherry_pick_changes.md#cherry-pick-a-commit) +you can [cherry-pick](../../../user/project/merge_requests/cherry_pick_changes.md#cherry-pick-a-single-commit) that commit into a new merge request. NOTE: diff --git a/doc/topics/offline/quick_start_guide.md b/doc/topics/offline/quick_start_guide.md index 353ba094d1e..c00c9621756 100644 --- a/doc/topics/offline/quick_start_guide.md +++ b/doc/topics/offline/quick_start_guide.md @@ -71,7 +71,7 @@ sudo EXTERNAL_URL="http://my-host.internal" dpkg -i <gitlab_package_name>.deb ## Enabling SSL Follow these steps to enable SSL for your fresh instance. These steps reflect those for -[manually configuring SSL in Omnibus's NGINX configuration](https://docs.gitlab.com/omnibus/settings/nginx.html#manually-configuring-https): +[manually configuring SSL in Omnibus's NGINX configuration](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-https-manually): 1. Make the following changes to `/etc/gitlab/gitlab.rb`: diff --git a/doc/topics/release_your_application.md b/doc/topics/release_your_application.md index 61ca1468dca..31fc9b4dbb9 100644 --- a/doc/topics/release_your_application.md +++ b/doc/topics/release_your_application.md @@ -30,13 +30,13 @@ to Kubernetes clusters using the [GitLab agent](../user/clusters/agent/install/i #### GitOps deployments **(PREMIUM)** -With the [GitLab agent for Kubernetes](../user/clusters/agent/install/index.md), you can perform -[pull-based deployments of Kubernetes manifests](../user/clusters/agent/gitops.md). This provides a scalable, secure, +With the [GitLab agent for Kubernetes](../user/clusters/agent/install/index.md), you can perform +[pull-based deployments of Kubernetes manifests](../user/clusters/agent/gitops.md). This provides a scalable, secure, and cloud-native approach to manage Kubernetes deployments. #### Deploy to Kubernetes from GitLab CI/CD -With the [GitLab agent for Kubernetes](../user/clusters/agent/install/index.md), you can perform +With the [GitLab agent for Kubernetes](../user/clusters/agent/install/index.md), you can perform [push-based deployments](../user/clusters/agent/ci_cd_workflow.md) from GitLab CI/CD. The agent provides a secure and reliable connection between GitLab and your Kubernetes cluster. diff --git a/doc/tutorials/make_your_first_git_commit.md b/doc/tutorials/make_your_first_git_commit.md index be9023c6ae0..879257fc3b8 100644 --- a/doc/tutorials/make_your_first_git_commit.md +++ b/doc/tutorials/make_your_first_git_commit.md @@ -83,8 +83,8 @@ Here's an overview of what we're going to do: To start, create a sample project in GitLab. -1. In GitLab, on the top bar, select **Menu > Projects > Create new project**. -1. Select **Create blank project**. +1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. For **Project name**, enter `My sample project`. The project slug is generated for you. This slug is the URL you can use to access the project after it's created. 1. Ensure **Initialize repository with a README** is selected. @@ -238,7 +238,7 @@ to the default branch (`main`). NOTE: For this tutorial, you merge your branch directly to the default branch for your -repository. In GitLab, you typically use a [merge request](../user/project/merge_requests/) +repository. In GitLab, you typically use a [merge request](../user/project/merge_requests/index.md) to merge your branch. ### View your changes in GitLab diff --git a/doc/tutorials/move_personal_project_to_a_group.md b/doc/tutorials/move_personal_project_to_a_group.md index 5ebbf813ab9..49ce09a2ed4 100644 --- a/doc/tutorials/move_personal_project_to_a_group.md +++ b/doc/tutorials/move_personal_project_to_a_group.md @@ -46,8 +46,8 @@ Maintainer role for the group. If you don't have a group, create one: -1. On the top bar, select **Menu > Groups > Create group** -1. Select **Create group**. +1. On the top bar, select **Main menu > Groups > View all groups**. +1. On the right of the page, select **New group**. 1. In **Group name**, enter a name for the group. 1. In **Group URL**, enter a path for the group, which is used as the namespace. 1. Choose the [visibility level](../user/public_access.md). @@ -64,7 +64,7 @@ Before you move your project to a group: Now you're ready to move your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. Under **Transfer project**, choose the group to transfer the project to. @@ -85,7 +85,7 @@ your related resources and tools, such as websites and package managers. You can now view your project in your group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. Look for your project under **Subgroups and projects**. Start enjoying the benefits of a group! For example, as the group Owner, you can diff --git a/doc/update/deprecations.md b/doc/update/deprecations.md index b69f8de2947..44a7137f698 100644 --- a/doc/update/deprecations.md +++ b/doc/update/deprecations.md @@ -45,24 +45,91 @@ sole discretion of GitLab Inc. <div class="announcement-milestone"> -## Announced in 15.8 +## Announced in 15.4 <div class="deprecation removal-160 breaking-change"> -### Security report schemas version 14.x.x +### Container Scanning variables that reference Docker + +Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22) + +WARNING: +This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +Review the details carefully before upgrading. + +All Container Scanning variables that are prefixed by `DOCKER_` in variable name are deprecated. This includes the `DOCKER_IMAGE`, `DOCKER_PASSWORD`, `DOCKER_USER`, and `DOCKERFILE_PATH` variables. Support for these variables will be removed in the GitLab 16.0 release. Use the [new variable names](https://docs.gitlab.com/ee/user/application_security/container_scanning/#available-cicd-variables) `CS_IMAGE`, `CS_REGISTRY_PASSWORD`, `CS_REGISTRY_USER`, and `CS_DOCKERFILE_PATH` in place of the deprecated names. + +</div> + +<div class="deprecation removal-160 breaking-change"> + +### Non-expiring access tokens + +Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22) + +WARNING: +This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +Review the details carefully before upgrading. + +Access tokens that have no expiration date are valid indefinitely, which presents a security risk if the access token +is divulged. Because access tokens that have an exipiration date are better, from GitLab 15.3 we +[populate a default expiration date](https://gitlab.com/gitlab-org/gitlab/-/issues/348660). + +In GitLab 16.0, any [personal](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html), +[project](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html), or +[group](https://docs.gitlab.com/ee/user/group/settings/group_access_tokens.html) access token that does not have an +expiration date will automatically have an expiration date set at one year. + +We recommend giving your access tokens an expiration date in line with your company's security policies before the +default is applied: + +- On GitLab.com during the 16.0 milestone. +- On GitLab self-managed instances when they are upgraded to 16.0. + +</div> + +<div class="deprecation removal-160 breaking-change"> + +### Starboard directive in the config for the GitLab Agent for Kubernetes -End of Support: GitLab <span class="removal-milestone">15.8</span> (2023-01-22) Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22) WARNING: This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -All [security report schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas) versions before 15.0.0 are considered deprecated in GitLab %15.8. Specifically, all [schemas](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/ee/gitlab/ci/parsers/security/validators/schemas) that match 14.*.* will be deprecated. +GitLab's operational container scanning capabilities no longer require starboard to be installed. Consequently, use of the `starboard:` directive in the configuration file for the GitLab Agent for Kubernetes is now deprecated and is scheduled for removal in GitLab 16.0. Update your configuration file to use the `container_scanning:` directive. + +</div> + +<div class="deprecation removal-160 breaking-change"> + +### Toggle behavior of `/draft` quick action in merge requests + +Planned removal: GitLab <span class="removal-milestone">16.0</span> (2022-05-22) -Please note that any [security report scanner integration](https://docs.gitlab.com/ee/development/integrations/secure.html) with GitLab using a deprecated schema version will result in a deprecation warning as a result of [report validation](https://docs.gitlab.com/ee/development/integrations/secure.html#report-validation). +WARNING: +This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +Review the details carefully before upgrading. -See [Security report validation](https://docs.gitlab.com/ee/user/application_security/#security-report-validation) for more information. +In order to make the behavior of toggling the draft status of a merge request more clear via a quick action, we're deprecating and removing the toggle behavior of the `/draft` quick action. Beginning with the 16.0 release of GitLab, `/draft` will only set a merge request to Draft and a new `/ready` quick action will be used to remove the draft status. + +</div> + +<div class="deprecation removal-160 breaking-change"> + +### Vulnerability confidence field + +Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22) + +WARNING: +This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +Review the details carefully before upgrading. + +In GitLab 15.3, [security report schemas below version 15 were deprecated](https://docs.gitlab.com/ee/update/deprecations.html#security-report-schemas-version-14xx). +The `confidence` attribute on vulnerability findings exists only in schema versions before `15-0-0`, and therefore is effectively deprecated since GitLab 15.4 supports schema version `15-0-0`. To maintain consistency +between the reports and our public APIs, the `confidence` attribute on any vulnerability-related components of our GraphQL API is now deprecated and will be +removed in 16.0. </div> </div> @@ -88,6 +155,20 @@ next major release, GitLab 16.0. This gem sees very little use and its </div> +<div class="deprecation removal-154"> + +### Bundled Grafana deprecated + +Planned removal: GitLab <span class="removal-milestone">15.4</span> (2022-09-22) + +In GitLab 15.4, we will be swapping the bundled Grafana to a fork of Grafana maintained by GitLab. + +There was an [identified CVE for Grafana](https://nvd.nist.gov/vuln/detail/CVE-2022-31107), and to mitigate this security vulnerability, we must swap to our own fork because the older version of Grafana we were bundling is no longer receiving long-term support. + +This is not expected to cause any incompatibilities with the previous version of Grafana. Neither when using our bundled version, nor when using an external instance of Grafana. + +</div> + <div class="deprecation removal-160 breaking-change"> ### CAS OmniAuth provider @@ -121,7 +202,7 @@ The [**Maximum number of active pipelines per project** limit](https://docs.gitl ### Redis 5 deprecated -End of Support: GitLab <span class="removal-milestone">15.6</span> (2022-11-22) +End of Support: GitLab <span class="removal-milestone">15.6</span> (2022-11-22)<br /> Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22) WARNING: @@ -133,6 +214,40 @@ Redis 5 has reached the end of life in April 2022 and will no longer be supporte If you are using your own Redis 5.0 instance, you should upgrade it to Redis 6.0 or higher before upgrading to GitLab 16.0 or higher. </div> + +<div class="deprecation removal-160 breaking-change"> + +### Security report schemas version 14.x.x + +Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22) + +WARNING: +This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +Review the details carefully before upgrading. + +Version 14.x.x [security report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas) are deprecated. + +In GitLab 15.8 and later, [security report scanner integrations](https://docs.gitlab.com/ee/development/integrations/secure.html) that use schema version 14.x.x will display a deprecation warning in the pipeline's **Security** tab. + +In GitLab 16.0 and later, the feature will be removed. Security reports that use schema version 14.x.x will cause an error in the pipeline's **Security** tab. + +For more information, refer to [security report validation](https://docs.gitlab.com/ee/user/application_security/#security-report-validation). + +</div> + +<div class="deprecation removal-160 breaking-change"> + +### Use of `id` field in vulnerabilityFindingDismiss mutation + +Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22) + +WARNING: +This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +Review the details carefully before upgrading. + +You can use the vulnerabilityFindingDismiss GraphQL mutation to set the status of a vulnerability finding to `Dismissed`. Previously, this mutation used the `id` field to identify findings uniquely. However, this did not work for dismissing findings from the pipeline security tab. Therefore, using the `id` field as an identifier has been dropped in favor of the `uuid` field. Using the 'uuid' field as an identifier allows you to dismiss the finding from the pipeline security tab. + +</div> </div> <div class="announcement-milestone"> @@ -337,38 +452,6 @@ In GitLab 15.0, for Dependency Scanning, the default version of Java that the sc </div> -<div class="deprecation removal-160 breaking-change"> - -### Manual iteration management - -Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-04-22) - -WARNING: -This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -Review the details carefully before upgrading. - -Manual iteration management is deprecated and only automatic iteration cadences will be supported in the future. - -Creating and deleting iterations will be fully removed in 16.0. Updating all iteration fields except for -`description` will also be removed. - -On the GraphQL API the following mutations will be removed: - - 1. `iterationCreate` - 1. `iterationDelete` - -The update `updateIteration` mutation will only allow updating the iteration's `description`. The following -arguments will be removed: - - 1. `title` - 1. `dueDate` - 1. `startDate` - -For more information about iteration cadences, you can refer to -[the documentation of the feature](https://docs.gitlab.com/ee/user/group/iterations/#iteration-cadences). - -</div> - <div class="deprecation removal-150 breaking-change"> ### Outdated indices of Advanced Search migrations @@ -1757,17 +1840,17 @@ When checking if a runner is `paused`, API users are advised to check the boolea </div> -<div class="deprecation removal-156 breaking-change"> +<div class="deprecation removal-159 breaking-change"> ### SaaS certificate-based integration with Kubernetes -Planned removal: GitLab <span class="removal-milestone">15.6</span> (2022-11-22) +Planned removal: GitLab <span class="removal-milestone">15.9</span> (2023-02-22) WARNING: This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -The certificate-based integration with Kubernetes will be [deprecated and removed](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/). As a GitLab SaaS customer, on new namespaces, you will no longer be able to integrate GitLab and your cluster using the certificate-based approach as of GitLab 15.0. The integration for current users will be enabled per namespace. The integrations are expected to be switched off completely on GitLab SaaS around 2022 November 22. +The certificate-based integration with Kubernetes will be [deprecated and removed](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/). As a GitLab SaaS customer, on new namespaces, you will no longer be able to integrate GitLab and your cluster using the certificate-based approach as of GitLab 15.0. The integration for current users will be enabled per namespace. For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the [agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) @@ -1778,11 +1861,11 @@ GitLab self-managed customers can still use the feature [with a feature flag](ht </div> -<div class="deprecation removal-160 breaking-change"> +<div class="deprecation removal-170 breaking-change"> ### Self-managed certificate-based integration with Kubernetes -Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22) +Planned removal: GitLab <span class="removal-milestone">17.0</span> (2024-05-22) WARNING: This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). @@ -1792,7 +1875,7 @@ The certificate-based integration with Kubernetes [will be deprecated and remove As a self-managed customer, we are introducing the [feature flag](../administration/feature_flags.md#enable-or-disable-the-feature) `certificate_based_clusters` in GitLab 15.0 so you can keep your certificate-based integration enabled. However, the feature flag will be disabled by default, so this change is a **breaking change**. -In GitLab 16.0 we will remove both the feature and its related code. Until the final removal in 16.0, features built on this integration will continue to work, if you enable the feature flag. Until the feature is removed, GitLab will continue to fix security and critical issues as they arise. +In GitLab 17.0 we will remove both the feature and its related code. Until the final removal in 17.0, features built on this integration will continue to work, if you enable the feature flag. Until the feature is removed, GitLab will continue to fix security and critical issues as they arise. For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the [agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) diff --git a/doc/update/index.md b/doc/update/index.md index d4412d85355..bffecf58304 100644 --- a/doc/update/index.md +++ b/doc/update/index.md @@ -41,7 +41,7 @@ There are also instructions when you want to ### Installation from source -- [Upgrading Community Edition and Enterprise Edition from source](upgrading_from_source.md) - +- [Upgrading Community Edition and Enterprise Edition from source](upgrading_from_source.md) - The guidelines for upgrading Community Edition and Enterprise Edition from source. - [Patch versions](patch_versions.md) guide includes the steps needed for a patch version, such as 13.2.0 to 13.2.1, and apply to both Community and Enterprise @@ -147,7 +147,7 @@ Some installations [may need to run GitLab 14.0 for at least a day](#1400) to co To check the status of batched background migrations: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Migrations**.  @@ -328,7 +328,7 @@ sudo -u git -H bundle exec rake gitlab:elastic:list_pending_migrations ### What do you do if your Advanced Search migrations are stuck? In GitLab 15.0, an Advanced Search migration named `DeleteOrphanedCommit` can be permanently stuck -in a pending state across upgrades. This issue +in a pending state across upgrades. This issue [is corrected in GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89539). If you are a self-managed customer who uses GitLab 15.0 with Advanced Search, you will experience performance degradation. @@ -380,7 +380,7 @@ Find where your version sits in the upgrade path below, and upgrade GitLab accordingly, while also consulting the [version-specific upgrade instructions](#version-specific-upgrading-instructions): -`8.11.Z` -> `8.12.0` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> [`11.11.8`](#1200) -> `12.0.12` -> [`12.1.17`](#1210) -> [`12.10.14`](#12100) -> `13.0.14` -> [`13.1.11`](#1310) -> [`13.8.8`](#1388) -> [`13.12.15`](#13120) -> [`14.0.12`](#1400) -> [`14.3.6`](#1430) -> [`14.9.5`](#1490) -> [`14.10.Z`](#14100) -> [`15.0.Z`](#1500) -> [latest `15.Y.Z`](https://gitlab.com/gitlab-org/gitlab/-/releases) +`8.11.Z` -> `8.12.0` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> [`11.11.8`](#1200) -> `12.0.12` -> [`12.1.17`](#1210) -> [`12.10.14`](#12100) -> `13.0.14` -> [`13.1.11`](#1310) -> [`13.8.8`](#1388) -> [`13.12.15`](#13120) -> [`14.0.12`](#1400) -> [`14.3.6`](#1430) -> [`14.9.5`](#1490) -> [`14.10.Z`](#14100) -> [`15.0.Z`](#1500) -> [`15.4.0`](#1540) -> [latest `15.Y.Z`](https://gitlab.com/gitlab-org/gitlab/-/releases) NOTE: When not explicitly specified, upgrade GitLab to the latest available patch @@ -465,6 +465,21 @@ NOTE: Specific information that follow related to Ruby and Git versions do not apply to [Omnibus installations](https://docs.gitlab.com/omnibus/) and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with appropriate Ruby and Git versions and are not using system binaries for Ruby and Git. There is no need to install Ruby or Git when utilizing these two approaches. +### 15.4.0 + +- GitLab 15.4.0 includes a [batched background migration](#batched-background-migrations) to [remove incorrect values from `expire_at` in `ci_job_artifacts` table](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89318). + This migration might take hours or days to complete on larger GitLab instances. + +### 15.3.3 + +- In GitLab 15.3.3, [SAML Group Links](../api/groups.md#saml-group-links) API `access_level` attribute type changed to `integer`. See +[valid access levels](../api/members.md#valid-access-levels) documentation. + +### 15.3.0 + +- [Incorrect deletion of object storage files on Geo secondary sites]((https://gitlab.com/gitlab-org/gitlab/-/issues/371397)) can occur in certain situations. See [Geo: Incorrect object storage LFS file deletion on secondary site issue in GitLab 15.0.0 to 15.3.2](#geo-incorrect-object-storage-lfs-file-deletion-on-secondary-sites-in-gitlab-1500-to-1532). +- LFS transfers can [redirect to the primary from secondary site mid-session](https://gitlab.com/gitlab-org/gitlab/-/issues/371571) causing failed pull and clone requests when [Geo proxying](../administration/geo/secondary_proxy/index.md) is enabled. Geo proxying is enabled by default in GitLab 15.1 and later. See [Geo: LFS transfer redirect to primary from secondary site mid-session issue in GitLab 15.1.0 to 15.3.2](#geo-lfs-transfers-redirect-to-primary-from-secondary-site-mid-session-in-gitlab-1510-to-1532) for more details. + ### 15.2.0 - GitLab installations that have multiple web nodes should be @@ -472,6 +487,19 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap configuration change in Rails that can result in inconsistent ETag key generation. - Some Sidekiq workers were renamed in this release. To avoid any disruption, [run the Rake tasks to migrate any pending jobs](../administration/sidekiq/sidekiq_job_migration.md#future-jobs) before starting the upgrade to GitLab 15.2.0. +- Gitaly now executes its binaries in a [runtime location](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4670). By default on Omnibus GitLab, + this path is `/var/opt/gitlab/gitaly/run/`. If this location is mounted with `noexec`, merge requests generate the following error: + + ```plaintext + fork/exec /var/opt/gitlab/gitaly/run/gitaly-<nnnn>/gitaly-git2go-v15: permission denied + ``` + + To resolve this, remove the `noexec` option from the filesystem mount. An alternative is to change the Gitaly runtime directory: + + 1. Add `gitaly['runtime_dir'] = '<PATH_WITH_EXEC_PERM>'` to `/etc/gitlab/gitlab.rb` and specify a location without `noexec` set. + 1. Run `sudo gitlab-ctl reconfigure`. +- [Incorrect deletion of object storage files on Geo secondary sites]((https://gitlab.com/gitlab-org/gitlab/-/issues/371397)) can occur in certain situations. See [Geo: Incorrect object storage LFS file deletion on secondary site issue in GitLab 15.0.0 to 15.3.2](#geo-incorrect-object-storage-lfs-file-deletion-on-secondary-sites-in-gitlab-1500-to-1532). +- LFS transfers can [redirect to the primary from secondary site mid-session](https://gitlab.com/gitlab-org/gitlab/-/issues/371571) causing failed pull and clone requests when [Geo proxying](../administration/geo/secondary_proxy/index.md) is enabled. Geo proxying is enabled by default in GitLab 15.1 and later. See [Geo: LFS transfer redirect to primary from secondary site mid-session issue in GitLab 15.1.0 to 15.3.2](#geo-lfs-transfers-redirect-to-primary-from-secondary-site-mid-session-in-gitlab-1510-to-1532) for more details. ### 15.1.0 @@ -490,6 +518,8 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap - Unauthenticated requests to the [`ciConfig` GraphQL field](../api/graphql/reference/index.md#queryciconfig) are no longer supported. Before you upgrade to GitLab 15.1, add an [access token](../api/index.md#authentication) to your requests. The user creating the token must have [permission](../user/permissions.md) to create pipelines in the project. +- [Incorrect deletion of object storage files on Geo secondary sites]((https://gitlab.com/gitlab-org/gitlab/-/issues/371397)) can occur in certain situations. See [Geo: Incorrect object storage LFS file deletion on secondary site issue in GitLab 15.0.0 to 15.3.2](#geo-incorrect-object-storage-lfs-file-deletion-on-secondary-sites-in-gitlab-1500-to-1532). +- LFS transfers can [redirect to the primary from secondary site mid-session](https://gitlab.com/gitlab-org/gitlab/-/issues/371571) causing failed pull and clone requests when [Geo proxying](../administration/geo/secondary_proxy/index.md) is enabled. Geo proxying is enabled by default in GitLab 15.1 and later. See [Geo: LFS transfer redirect to primary from secondary site mid-session issue in GitLab 15.1.0 to 15.3.2](#geo-lfs-transfers-redirect-to-primary-from-secondary-site-mid-session-in-gitlab-1510-to-1532) for more details. ### 15.0.0 @@ -500,6 +530,12 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap - The use of encrypted S3 buckets with storage-specific configuration is no longer supported after [removing support for using `background_upload`](removals.md#background-upload-for-object-storage). - The [certificate-based Kubernetes integration (DEPRECATED)](../user/infrastructure/clusters/index.md#certificate-based-kubernetes-integration-deprecated) is disabled by default, but you can be re-enable it through the [`certificate_based_clusters` feature flag](../administration/feature_flags.md#how-to-enable-and-disable-features-behind-flags) until GitLab 16.0. - When you use the GitLab Helm Chart project with a custom `serviceAccount`, ensure it has `get` and `list` permissions for the `serviceAccount` and `secret` resources. +- The [`custom_hooks_dir`](../administration/server_hooks.md#create-global-server-hooks-for-all-repositories) setting for configuring global server hooks is now configured in + Gitaly. The previous implementation in GitLab Shell was removed in GitLab 15.0. With this change, global server hooks are stored only inside a subdirectory named after the + hook type. Global server hooks can no longer be a single hook file in the root of the custom hooks directory. For example, you must use `<custom_hooks_dir>/<hook_name>.d/*` rather + than `<custom_hooks_dir>/<hook_name>`. +- [Incorrect deletion of object storage files on Geo secondary sites]((https://gitlab.com/gitlab-org/gitlab/-/issues/371397)) can occur in certain situations. See [Geo: Incorrect object storage LFS file deletion on secondary site issue in GitLab 15.0.0 to 15.3.2](#geo-incorrect-object-storage-lfs-file-deletion-on-secondary-sites-in-gitlab-1500-to-1532). +- The `FF_GITLAB_REGISTRY_HELPER_IMAGE` [feature flag](../administration/feature_flags.md#enable-or-disable-the-feature) is removed and helper images are always pulled from GitLab Registry. ### 14.10.0 @@ -1040,7 +1076,7 @@ In 13.1.0, you must upgrade to either: Failure to do so results in internal errors in the Gitaly service in some RPCs due to the use of the new `--end-of-options` Git flag. -Additionally, in GitLab 13.1.0, the version of +Additionally, in GitLab 13.1.0, the version of [Rails was upgraded from 6.0.3 to 6.0.3.1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33454). The Rails upgrade included a change to CSRF token generation which is not backwards-compatible - GitLab servers with the new Rails version @@ -1121,7 +1157,7 @@ for more information. When [Maintenance mode](../administration/maintenance_mode/index.md) is enabled, users cannot sign in with SSO, SAML, or LDAP. -Users who were signed in before Maintenance mode was enabled, continue to be signed in. If the administrator who enabled Maintenance mode loses their session, then they can't disable Maintenance mode via the UI. In that case, you can [disable Maintenance mode via the API or Rails console](../administration/maintenance_mode/#disable-maintenance-mode). +Users who were signed in before Maintenance mode was enabled, continue to be signed in. If the administrator who enabled Maintenance mode loses their session, then they can't disable Maintenance mode via the UI. In that case, you can [disable Maintenance mode via the API or Rails console](../administration/maintenance_mode/index.md#disable-maintenance-mode). [This bug](https://gitlab.com/gitlab-org/gitlab/-/issues/329261) was fixed in GitLab 14.5.0 and backported into 14.4.3 and 14.3.5. @@ -1145,6 +1181,27 @@ by a database engine bug that causes a segmentation fault. Read more [in the issue](https://gitlab.com/gitlab-org/gitlab/-/issues/364763). +### Geo: Incorrect object storage LFS file deletion on secondary sites in GitLab 15.0.0 to 15.3.2 + +[Incorrect deletion of object storage files on Geo secondary sites]((https://gitlab.com/gitlab-org/gitlab/-/issues/371397)) +can occur in GitLab 15.0.0 to 15.3.2 in the following situations: + +- GitLab-managed object storage replication is disabled, and LFS objects are created while importing a project with object storage enabled. +- GitLab-managed replication to sync object storage is enabled and subsequently disabled. + +This issue is resolved in 15.3.3. Customers who have both LFS enabled and LFS objects being replicated across Geo sites +should upgrade directly to 15.3.3 to reduce the risk of data loss on secondary sites. + +### Geo: LFS transfers redirect to primary from secondary site mid-session in GitLab 15.1.0 to 15.3.2 + +LFS transfers can [redirect to the primary from secondary site mid-session](https://gitlab.com/gitlab-org/gitlab/-/issues/371571) causing failed pull and clone requests in GitLab 15.1.0 to 15.3.2 when [Geo proxying](../administration/geo/secondary_proxy/index.md) is enabled. Geo proxying is enabled by default in GitLab 15.1 and later. + +This issue is resolved in GitLab 15.3.3, so customers with the following configuration should upgrade to 15.3.3 or later: + +- LFS is enabled. +- LFS objects are being replicated across Geo sites. +- Repositories are being pulled by using a Geo secondary site. + ## Miscellaneous - [MySQL to PostgreSQL](mysql_to_postgresql.md) guides you through migrating diff --git a/doc/update/package/index.md b/doc/update/package/index.md index 12a8b6f3190..06a56f49cc1 100644 --- a/doc/update/package/index.md +++ b/doc/update/package/index.md @@ -330,3 +330,10 @@ To fix this error: ```shell sudo gitlab-ctl reconfigure ``` + +1. Hot reload `puma` and `sidekiq` services: + + ```shell + sudo gitlab-ctl hup puma + sudo gitlab-ctl restart sidekiq + ``` diff --git a/doc/update/plan_your_upgrade.md b/doc/update/plan_your_upgrade.md index 0947bab855f..03bdf161756 100644 --- a/doc/update/plan_your_upgrade.md +++ b/doc/update/plan_your_upgrade.md @@ -132,7 +132,7 @@ to your instance and then upgrade it for any relevant features you're using. [turning on maintenance mode](../administration/maintenance_mode/index.md) during the upgrade. - About PostgreSQL: - - On the top bar, select **Menu > Admin**, and look for the version of + - On the top bar, select **Main menu > Admin**, and look for the version of PostgreSQL you are using. If [a PostgreSQL upgrade is needed](../administration/package_information/postgresql_versions.md), account for the relevant @@ -172,6 +172,9 @@ If you have Kubernetes clusters connected with GitLab, [upgrade your GitLab agen #### Elasticsearch +Before updating GitLab, confirm Advanced Search migrations are complete by +[checking for pending advanced search migrations](index.md#checking-for-pending-advanced-search-migrations). + After updating GitLab, you may have to upgrade [Elasticsearch if the new version breaks compatibility](../integration/advanced_search/elasticsearch.md#version-requirements). Updating Elasticsearch is **out of scope for GitLab Support**. diff --git a/doc/update/removals.md b/doc/update/removals.md index cdb35b5faa0..9b5596d67f2 100644 --- a/doc/update/removals.md +++ b/doc/update/removals.md @@ -31,6 +31,49 @@ For removal reviewers (Technical Writers only): https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-removals-doc --> +## Removed in 15.4 + +### SAST analyzer consolidation and CI/CD template changes + +WARNING: +This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +Review the details carefully before upgrading. + +We have replaced the GitLab SAST [analyzers](https://docs.gitlab.com/ee/user/application_security/sast/analyzers/) for certain languages in GitLab 15.4 as part of our long-term strategy to deliver a more consistent user experience, faster scan times, and reduced CI minute usage. + +Starting from GitLab 15.4, the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) uses [Semgrep-based scanning](https://docs.gitlab.com/ee/user/application_security/sast/analyzers.html#transition-to-semgrep-based-scanning) instead of the following analyzers: + +- [ESLint](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) for JavaScript, TypeScript, React +- [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) for Go +- [Bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) for Python +- [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) for Java + +We will no longer make any updates to the ESLint-, Gosec-, and Bandit-based analyzers. +The SpotBugs-based analyzer will continue to be used for Groovy, Kotlin, and Scala scanning. + +We won't delete container images previously published for these analyzers, so older versions of the CI/CD template will continue to work. + +If you changed the default GitLab SAST configuration, you may need to update your configuration as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352554#actions-required). + +## Removed in 15.3 + +### Support for Debian 9 + +Long term service and support (LTSS) for [Debian 9 Stretch ended in July 2022](https://wiki.debian.org/LTS). Therefore, we will no longer support the Debian 9 distribution for the GitLab package. Users can upgrade to Debian 10 or Debian 11. + +### Vulnerability Report sort by State + +The ability to sort the Vulnerability Report by the `State` column was disabled and put behind a feature flag in GitLab 14.10 due to a refactor +of the underlying data model. The feature flag has remained off by default as further refactoring will be required to ensure sorting +by this value remains performant. Due to very low usage of the `State` column for sorting, the feature flag is instead removed in 15.3 to simplify the codebase and prevent any unwanted performance degradation. + +### Vulnerability Report sort by Tool + +The ability to sort the Vulnerability Report by the `Tool` column (scan type) was disabled and put behind a feature flag in GitLab 14.10 due to a refactor +of the underlying data model. The feature flag has remained off by default as further refactoring will be required to ensure sorting +by this value remains performant. Due to very low usage of the `Tool` column for sorting, the feature flag is instead removed in +GitLab 15.3 to simplify the codebase and prevent any unwanted performance degradation. + ## Removed in 15.2 ### Support for older browsers @@ -121,8 +164,8 @@ If you have set a prefix, you can use a workaround to revert to background uploa gitlab_rails['env'] = { 'GITLAB_LEGACY_BACKGROUND_UPLOADS' => 'artifacts,external_diffs,lfs,uploads,packages,dependency_proxy,terraform_state,pages' } ``` -Prefixes will be supported officially in [GitLab 15.2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91307). -This workaround will be dropped, so we encourage migrating to consolidated object storage. +Support for prefixes was restored in GitLab 15.2 via [this MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91307). +Support for setting `GITLAB_LEGACY_BACKGROUND_UPLOADS` will be removed in GitLab 15.4. ### Container Network and Host Security diff --git a/doc/update/upgrading_from_source.md b/doc/update/upgrading_from_source.md index 2df11e8f741..4f54e69d8c9 100644 --- a/doc/update/upgrading_from_source.md +++ b/doc/update/upgrading_from_source.md @@ -119,7 +119,7 @@ rm go1.17.10.linux-amd64.tar.gz To check you are running the minimum required Git version, see [Git versions](../install/installation.md#software-requirements). -From GitLab 13.6, we recommend you use the +From GitLab 13.6, we recommend you use the [Git version provided by Gitaly](https://gitlab.com/gitlab-org/gitaly/-/issues/2729) that: diff --git a/doc/update/zero_downtime.md b/doc/update/zero_downtime.md index 0abdd769a6b..29bb1c4bcb4 100644 --- a/doc/update/zero_downtime.md +++ b/doc/update/zero_downtime.md @@ -53,7 +53,7 @@ migrating data. Background migrations are only added in the monthly releases. Certain major/minor releases may require a set of background migrations to be finished. To guarantee this, such a release processes any remaining jobs before continuing the upgrading procedure. While this doesn't require downtime -(if the above conditions are met) we require that you +(if the above conditions are met) we require that you [wait for background migrations to complete](index.md#checking-for-background-migrations-before-upgrading) between each major/minor release upgrade. The time necessary to complete these migrations can be reduced by @@ -492,9 +492,11 @@ You can only upgrade one minor release at a time. The order of steps is important. While following these steps, make sure you follow them in the right order, on the correct node. +### Update the Geo primary site + Log in to your **primary** node, executing the following: -1. Create an empty file at `/etc/gitlab/skip-auto-reconfigure`. This prevents upgrades from running `gitlab-ctl reconfigure`, which by default automatically stops GitLab, runs all database migrations, and restarts GitLab. +1. Create an empty file at `/etc/gitlab/skip-auto-reconfigure`. This prevents upgrades from running `gitlab-ctl reconfigure`, which by default automatically stops GitLab, runs all database migrations, and restarts GitLab: ```shell sudo touch /etc/gitlab/skip-auto-reconfigure @@ -512,7 +514,7 @@ Log in to your **primary** node, executing the following: sudo gitlab-ctl reconfigure ``` -1. Update the GitLab package +1. Update the GitLab package: ```shell # Debian/Ubuntu @@ -522,18 +524,13 @@ Log in to your **primary** node, executing the following: sudo yum install gitlab-ee ``` -1. To get the database migrations and latest code in place, run +1. To get the database migrations and latest code in place, run: ```shell sudo SKIP_POST_DEPLOYMENT_MIGRATIONS=true gitlab-ctl reconfigure ``` -1. Hot reload `puma` and `sidekiq` services - - ```shell - sudo gitlab-ctl hup puma - sudo gitlab-ctl restart sidekiq - ``` +### Update the Geo secondary site On each **secondary** node, executing the following: @@ -555,7 +552,7 @@ On each **secondary** node, executing the following: sudo gitlab-ctl reconfigure ``` -1. Update the GitLab package +1. Update the GitLab package: ```shell # Debian/Ubuntu @@ -565,26 +562,20 @@ On each **secondary** node, executing the following: sudo yum install gitlab-ee ``` -1. To get the database migrations and latest code in place, run +1. To get the database migrations and latest code in place, run: ```shell sudo SKIP_POST_DEPLOYMENT_MIGRATIONS=true gitlab-ctl reconfigure ``` -1. Hot reload `puma`, `sidekiq` and restart `geo-logcursor` services - - ```shell - sudo gitlab-ctl hup puma - sudo gitlab-ctl restart sidekiq - sudo gitlab-ctl restart geo-logcursor - ``` - -1. Run post-deployment database migrations, specific to the Geo database +1. Run post-deployment database migrations, specific to the Geo database: ```shell sudo gitlab-rake db:migrate:geo ``` +### Finalize the update + After all **secondary** nodes are updated, finalize the update on the **primary** node: @@ -594,6 +585,16 @@ the update on the **primary** node: sudo gitlab-rake db:migrate ``` +- After the update is finalized on the primary node, hot reload `puma` and +restart `sidekiq` and `geo-logcursor` services on **all primary and secondary** +nodes: + + ```shell + sudo gitlab-ctl hup puma + sudo gitlab-ctl restart sidekiq + sudo gitlab-ctl restart geo-logcursor + ``` + After updating all nodes (both **primary** and all **secondaries**), check their status: - Verify Geo configuration and dependencies diff --git a/doc/user/admin_area/analytics/dev_ops_reports.md b/doc/user/admin_area/analytics/dev_ops_reports.md index 2ad18d5f70e..b4471a4602a 100644 --- a/doc/user/admin_area/analytics/dev_ops_reports.md +++ b/doc/user/admin_area/analytics/dev_ops_reports.md @@ -12,7 +12,7 @@ from planning to monitoring. To see DevOps Reports: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Analytics > DevOps Reports**. ## DevOps Score diff --git a/doc/user/admin_area/analytics/index.md b/doc/user/admin_area/analytics/index.md index 9315b926acc..9147a48aa8e 100644 --- a/doc/user/admin_area/analytics/index.md +++ b/doc/user/admin_area/analytics/index.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Administrators have access to instance-wide analytics: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Analytics**. There are several kinds of statistics: diff --git a/doc/user/admin_area/analytics/usage_trends.md b/doc/user/admin_area/analytics/usage_trends.md index 5f46908adb0..f21ee627f7f 100644 --- a/doc/user/admin_area/analytics/usage_trends.md +++ b/doc/user/admin_area/analytics/usage_trends.md @@ -19,7 +19,7 @@ Usage Trends data refreshes daily. To view Usage Trends: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Analytics > Usage Trends**. ## Total counts diff --git a/doc/user/admin_area/appearance.md b/doc/user/admin_area/appearance.md index 4c3bdde223b..fbc2f8d1827 100644 --- a/doc/user/admin_area/appearance.md +++ b/doc/user/admin_area/appearance.md @@ -10,12 +10,12 @@ disqus_identifier: 'https://docs.gitlab.com/ee/customization/branded_login_page. Several options are available for customizing the appearance of a self-managed instance of GitLab. To access these settings: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Appearance**. -## Navigation bar +## Top bar -By default, the navigation bar has the GitLab logo, but this can be customized with +By default, the **top bar** has the GitLab logo, but this can be customized with any image desired. It is optimized for images 28px high (any width), but any image can be used (less than 1 MB) and it is automatically resized. diff --git a/doc/user/admin_area/broadcast_messages.md b/doc/user/admin_area/broadcast_messages.md index a6e6a839912..b508b71ddac 100644 --- a/doc/user/admin_area/broadcast_messages.md +++ b/doc/user/admin_area/broadcast_messages.md @@ -57,7 +57,7 @@ To display messages to users on your GitLab instance, add a broadcast message. To add a broadcast message: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Messages**. 1. Add the text for the message to the **Message** field. You can style a message's content using Markdown, emoji, and the `a` and `br` HTML tags. The `br` tag inserts a line break. The `a` HTML tag accepts `class` and `style` attributes with the following CSS properties: @@ -83,7 +83,7 @@ If you must make changes to a broadcast message, you can edit it. To edit a broadcast message: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Messages**. 1. From the list of broadcast messages, select the edit button for the message. 1. After making the required changes, select **Update broadcast message**. @@ -97,7 +97,7 @@ You can delete a broadcast message while it's active. To delete a broadcast message: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Messages**. 1. From the list of broadcast messages, select the delete button for the message. diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md index 4308b45df78..02c4cd05b23 100644 --- a/doc/user/admin_area/credentials_inventory.md +++ b/doc/user/admin_area/credentials_inventory.md @@ -31,7 +31,7 @@ You can also [revoke](#revoke-a-users-personal-access-token) and [delete](#delet To access the Credentials inventory: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Credentials**. ## Revoke a user's personal access token diff --git a/doc/user/admin_area/custom_project_templates.md b/doc/user/admin_area/custom_project_templates.md index c3c580a91c3..0c03652bfb5 100644 --- a/doc/user/admin_area/custom_project_templates.md +++ b/doc/user/admin_area/custom_project_templates.md @@ -34,7 +34,7 @@ To set project templates at the group level, see [Custom group-level project tem To select the group to use as the source for the project templates: -1. On the top bar, navigate to **Menu > Admin > Settings > Templates**. +1. On the top bar, navigate to **Main menu > Admin > Settings > Templates**. 1. Expand **Custom project templates**. 1. Select a group to use. 1. Select **Save changes**. diff --git a/doc/user/admin_area/diff_limits.md b/doc/user/admin_area/diff_limits.md index b50748ca97e..f646d9f95fd 100644 --- a/doc/user/admin_area/diff_limits.md +++ b/doc/user/admin_area/diff_limits.md @@ -33,7 +33,7 @@ set values are presented as **Too large** are cannot be expanded in the UI. To configure these values: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Diff limits**. 1. Enter a value for the diff limit. diff --git a/doc/user/admin_area/email_from_gitlab.md b/doc/user/admin_area/email_from_gitlab.md index 52f27ed48e0..09643fc290e 100644 --- a/doc/user/admin_area/email_from_gitlab.md +++ b/doc/user/admin_area/email_from_gitlab.md @@ -22,7 +22,7 @@ For information about email notifications originating from GitLab, read ## Sending emails to users from GitLab -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select **Send email to users**. diff --git a/doc/user/admin_area/geo_sites.md b/doc/user/admin_area/geo_sites.md index e577fdf60f1..fecc0e7c541 100644 --- a/doc/user/admin_area/geo_sites.md +++ b/doc/user/admin_area/geo_sites.md @@ -11,7 +11,7 @@ You can configure various settings for GitLab Geo sites. For more information, s On either the primary or secondary site: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. ## Common settings @@ -71,7 +71,7 @@ the primary uses the secondary's internal URL to contact it directly. The internal URL defaults to external URL. To change it: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. 1. Select **Edit** on the site you want to customize. 1. Edit the internal URL. diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index c689d61ad68..207b7e6f2d8 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -12,7 +12,7 @@ self-managed instances. If you are an administrator, you can access the Admin Ar by visiting `/admin` on your self-managed instance. You can also access it through the UI: -- GitLab versions 14.0 and later: on the top bar, select **Menu > Admin**. +- GitLab versions 14.0 and later: on the top bar, select **Main menu > Admin**. - GitLab versions 13.12 and earlier: on the top bar, select the Admin Area icon (**{admin}**). NOTE: @@ -47,7 +47,7 @@ The Dashboard provides statistics and system information about the GitLab instan To access the Dashboard, either: -- On the top bar, select **Menu > Admin**. +- On the top bar, select **Main menu > Admin**. - Visit `/admin` on your self-managed instance. The Dashboard is the default view of the Admin Area, and is made up of the following sections: @@ -71,7 +71,7 @@ You can administer all projects in the GitLab instance from the Admin Area's Pro To access the Projects page: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Projects**. 1. Select the **All**, **Private**, **Internal**, or **Public** tab to list only projects of that criteria. @@ -112,7 +112,7 @@ You can combine the filter options. For example, to list only public projects wi You can administer all users in the GitLab instance from the Admin Area's Users page: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. To list users matching a specific criteria, select one of the following tabs on the **Users** page: @@ -157,7 +157,7 @@ This allows the administrator to "see what the user sees," and take actions on b You can impersonate a user in the following ways: - Through the UI: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. From the list of users, select a user. 1. Select **Impersonate**. @@ -175,7 +175,7 @@ By default, impersonation is enabled. GitLab can be configured to [disable imper When using authentication providers, administrators can see the identities for a user: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. From the list of users, select a user. 1. Select **Identities**. @@ -221,7 +221,7 @@ GitLab billing is based on the number of [**Billable users**](../../subscription You must be an administrator to manually add emails to users: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users** (`/admin/users`). 1. Locate the user and select them. 1. Select **Edit**. @@ -237,7 +237,7 @@ The [Cohorts](user_cohorts.md) tab displays the monthly cohorts of new users and By default, users can create groups. To prevent a user from creating a top level group: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users** (`/admin/users`). 1. Locate the user and select them. 1. Select **Edit**. @@ -252,7 +252,7 @@ You can administer all groups in the GitLab instance from the Admin Area's Group To access the Groups page: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Groups**. For each group, the page displays their name, description, size, number of projects in the group, @@ -277,7 +277,7 @@ GitLab instance from the Admin Area's Topics page. To access the Topics page: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Topics**. For each topic, the page displays its name and the number of projects labeled with the topic. @@ -288,6 +288,8 @@ To edit a topic, select **Edit** in that topic's row. To remove a topic, select **Remove** in that topic's row. +To remove a topic and move all assigned projects to another topic, select **Merge topics**. + To search for topics by name, enter your criteria in the search box. The topic search is case insensitive and applies partial matching. @@ -302,7 +304,7 @@ You can administer all jobs in the GitLab instance from the Admin Area's Jobs pa To access the Jobs page: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Jobs**. All jobs are listed, in descending order of job ID. 1. Select the **All** tab to list all jobs. Select the **Pending**, **Running**, or **Finished** tab to list only jobs of that status. @@ -328,7 +330,7 @@ You can administer all runners in the GitLab instance from the Admin Area's **Ru To access the **Runners** page: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Runners**. #### Search and filter runners @@ -347,6 +349,22 @@ You can also filter runners by status, type, and tag. To filter:  +#### Bulk delete runners + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370241) in GitLab 15.4 [with a flag](../../administration/feature_flags.md) named `admin_runners_bulk_delete`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `admin_runners_bulk_delete`. On GitLab.com, this feature is not available but can be enabled by GitLab.com administrators. + +You can delete multiple runners at the same time. + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Overview > Runners**. +1. To the left of the runners you want to delete, select the checkbox. + To select all of the runners on the page, select the checkbox above + the list. +1. Select **Delete selected**. + #### Runner attributes For each runner, the following attributes are listed: @@ -369,7 +387,7 @@ page. For more details, see [Gitaly](../../administration/gitaly/index.md). To access the **Gitaly Servers** page: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Gitaly Servers**. For each Gitaly server, the following details are listed: diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index ac5e5ded859..a5338aa35b8 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -28,7 +28,7 @@ To activate your instance with an activation code: - Your subscription confirmation email. - The [Customers Portal](https://customers.gitlab.com/customers/sign_in), on the **Manage Purchases** page. 1. Sign in to your GitLab self-managed instance. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Subscription**. 1. Paste the activation code in **Activation code**. 1. Read and accept the terms of service. diff --git a/doc/user/admin_area/license_file.md b/doc/user/admin_area/license_file.md index 99669b2a4d3..352d79ee381 100644 --- a/doc/user/admin_area/license_file.md +++ b/doc/user/admin_area/license_file.md @@ -18,7 +18,7 @@ link to the **Add license** page should be displayed. Otherwise, to add your license: 1. Sign in to GitLab as an administrator. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. In the **Add License** area, add a license by either uploading the file or entering the key. 1. Select the **Terms of Service** checkbox. @@ -79,7 +79,7 @@ To go back to Free features, [delete all expired licenses](#remove-a-license). To remove a license from a self-managed instance: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Subscription**. 1. Select **Remove license**. @@ -89,7 +89,7 @@ Repeat these steps to remove all licenses, including those applied in the past. To view your license details: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Subscription**. You can add and view more than one license, but only the latest license in diff --git a/doc/user/admin_area/merge_requests_approvals.md b/doc/user/admin_area/merge_requests_approvals.md index e090d4e7f88..38b177766cf 100644 --- a/doc/user/admin_area/merge_requests_approvals.md +++ b/doc/user/admin_area/merge_requests_approvals.md @@ -19,7 +19,7 @@ and can no longer be changed: To enable merge request approval settings for an instance: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **{push-rules}** **Push Rules**, and expand **Merge request approvals**. 1. Choose the required options. 1. Select **Save changes**. diff --git a/doc/user/admin_area/moderate_users.md b/doc/user/admin_area/moderate_users.md index ab581cd3aa8..6d632a6bdb6 100644 --- a/doc/user/admin_area/moderate_users.md +++ b/doc/user/admin_area/moderate_users.md @@ -40,7 +40,7 @@ sign in. To view user sign ups pending approval: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Pending approval** tab. @@ -50,7 +50,7 @@ A user sign up pending approval can be approved or rejected from the Admin Area. To approve or reject a user sign up: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Pending approval** tab. 1. Optional. Select a user. @@ -75,7 +75,7 @@ administrators can choose to block the user. Users can be blocked [via an abuse report](review_abuse_reports.md#blocking-users), by removing them in LDAP, or directly from the Admin Area. To do this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Optional. Select a user. 1. Select the **{settings}** **User administration** dropdown. @@ -98,7 +98,7 @@ Users can also be blocked using the [GitLab API](../../api/users.md#block-user). A blocked user can be unblocked from the Admin Area. To do this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Blocked** tab. 1. Optional. Select a user. @@ -114,7 +114,7 @@ Users can also be unblocked using the [GitLab API](../../api/users.md#unblock-us The unblock option may be unavailable for LDAP users. To enable the unblock option, the LDAP identity first needs to be deleted: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Blocked** tab. 1. Select a user. @@ -153,7 +153,7 @@ Users are notified about account deactivation if A user can be deactivated from the Admin Area. To do this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Optional. Select a user. 1. Select the **{settings}** **User administration** dropdown. @@ -169,19 +169,21 @@ Users can also be deactivated using the [GitLab API](../../api/users.md#deactiva ### Automatically deactivate dormant users -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/320875) in GitLab 14.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/320875) in GitLab 14.0. +> - Customizable time period [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336747) in GitLab 15.4 Administrators can enable automatic deactivation of users who either: - Were created more than a week ago and have not signed in. -- Have no activity in the last 90 days. +- Have no activity for a specified period of time (defaults to 90 days). To do this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Account and limit** section. 1. Under **Dormant users**, check **Deactivate dormant users after 90 days of inactivity**. +1. Under **Period of inactivity (days)**, enter a period of time before deactivation. 1. Select **Save changes**. When this feature is enabled, GitLab runs a job once a day to deactivate the dormant users. @@ -196,7 +198,7 @@ A deactivated user can be activated from the Admin Area. To do this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Deactivated** tab. 1. Optional. Select a user. @@ -229,7 +231,7 @@ To block a user and hide their contributions, administrators can ban the user. Users can be banned using the Admin Area. To do this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Optional. Select a user. 1. Select the **{settings}** **User administration** dropdown. @@ -241,7 +243,7 @@ The banned user does not consume a [seat](../../subscriptions/self_managed/index A banned user can be unbanned using the Admin Area. To do this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Banned** tab. 1. Optional. Select a user. @@ -255,7 +257,7 @@ The user's state is set to active and they consume a Use the Admin Area to delete users. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Banned** tab. 1. Optional. Select a user. @@ -269,7 +271,7 @@ You can only delete a user if there are inherited or direct owners of a group. Y You can also delete a user and their contributions, such as merge requests, issues, and groups of which they are the only group owner. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Banned** tab. 1. Optional. Select a user. diff --git a/doc/user/admin_area/monitoring/background_migrations.md b/doc/user/admin_area/monitoring/background_migrations.md index 87374849674..092a49dc7e9 100644 --- a/doc/user/admin_area/monitoring/background_migrations.md +++ b/doc/user/admin_area/monitoring/background_migrations.md @@ -59,7 +59,8 @@ Use the following database queries to see the state of the current batched backg ```sql SELECT - id job_class_name, + id, + job_class_name, table_name, column_name, job_arguments diff --git a/doc/user/admin_area/reporting/git_abuse_rate_limit.md b/doc/user/admin_area/reporting/git_abuse_rate_limit.md index ad3ecfa3a5a..11b0e2403aa 100644 --- a/doc/user/admin_area/reporting/git_abuse_rate_limit.md +++ b/doc/user/admin_area/reporting/git_abuse_rate_limit.md @@ -19,7 +19,7 @@ When both flags are enabled, the administrator receives an email when a user is ## Configure Git abuse rate limiting -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Reporting**. 1. Expand **Git abuse rate limit**. 1. Update the Git abuse rate limit settings: diff --git a/doc/user/admin_area/reporting/spamcheck.md b/doc/user/admin_area/reporting/spamcheck.md index 2360c1f2899..670b0521732 100644 --- a/doc/user/admin_area/reporting/spamcheck.md +++ b/doc/user/admin_area/reporting/spamcheck.md @@ -40,7 +40,7 @@ Spamcheck is only available for package-based installations: ## Configure GitLab to use Spamcheck -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Reporting**. 1. Expand **Spam and Anti-bot Protection**. 1. Update the Spam Check settings: diff --git a/doc/user/admin_area/review_abuse_reports.md b/doc/user/admin_area/review_abuse_reports.md index a5e7fcb1b8e..16295f340d8 100644 --- a/doc/user/admin_area/review_abuse_reports.md +++ b/doc/user/admin_area/review_abuse_reports.md @@ -16,7 +16,7 @@ reports in the Admin Area. To receive notifications of new abuse reports by email: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Reporting**. 1. Expand the **Abuse reports** section. 1. Provide an email address and select **Save changes**. @@ -26,14 +26,14 @@ The notification email address can also be set and retrieved ## Reporting abuse -To find out more about reporting abuse, see +To find out more about reporting abuse, see [abuse reports user documentation](../report_abuse.md). ## Resolving abuse reports To access abuse reports: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Abuse Reports**. There are 3 ways to resolve an abuse report, with a button for each method: diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index e33cf4a9082..e8f80cfb40f 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -16,7 +16,7 @@ the [project limits for existing users](#projects-limit-for-a-user). To configure the maximum number of projects in personal namespaces for new users: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. 1. Increase or decrease that **Default projects limit** value. @@ -28,7 +28,7 @@ in their users personal namespace. However, projects can still be created in a g You can edit a specific user, and change the maximum number of projects this user can create in their personal namespace: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview** > **Users**. 1. From the list of users, select a user. 1. Select **Edit**. @@ -39,7 +39,7 @@ can create in their personal namespace: The maximum file size for attachments in GitLab comments and replies is 10 MB. To change the maximum attachment size: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. 1. Increase or decrease by changing the value in **Maximum attachment size (MB)**. @@ -53,7 +53,7 @@ For GitLab.com repository size limits, read [accounts and limit settings](../../ You can change the maximum push size for your instance: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. 1. Increase or decrease by changing the value in **Maximum push size (MB)**. @@ -72,7 +72,7 @@ Use [Git LFS](../../../topics/git/lfs/index.md) to add large files to a reposito To modify the maximum file size for exports in GitLab: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. 1. Increase or decrease by changing the value in **Maximum export size (MB)**. @@ -82,7 +82,7 @@ To modify the maximum file size for exports in GitLab: To modify the maximum file size for imports in GitLab: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. 1. Increase or decrease by changing the value in **Maximum import size (MB)**. @@ -108,7 +108,7 @@ The default prefix is `glpat-` but administrators can change it. To change the default global prefix: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Account and limit** section. 1. Fill in the **Personal Access Token prefix** field. @@ -154,7 +154,7 @@ These settings can be found in: 1. Fill in the **Repository size limit (MB)** field in the **Naming, visibility** section. 1. Select **Save changes**. - GitLab global settings: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Account and limit** section. 1. Fill in the **Size limit per repository (MB)** field. @@ -182,7 +182,7 @@ GitLab administrators can choose to customize the session duration (in minutes) To set a limit on how long these sessions are valid: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Account and limit** section. 1. Fill in the **Session duration for Git operations when 2FA is enabled (minutes)** field. @@ -209,7 +209,7 @@ there are no restrictions. To set a lifetime on how long SSH keys are valid: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Account and limit** section. 1. Fill in the **Maximum allowable lifetime for SSH keys (days)** field. @@ -225,18 +225,6 @@ Once a lifetime for SSH keys is set, GitLab: NOTE: When a user's SSH key becomes invalid they can delete and re-add the same key again. -<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> -## Allow expired SSH keys to be used (removed) **(ULTIMATE SELF)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250480) in GitLab 13.9. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/320970) in GitLab 14.0. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 14.8. -> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 15.0. - -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 14.8. -This feature was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 15.0. -<!--- end_remove --> - ## Limit the lifetime of access tokens **(ULTIMATE SELF)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3649) in GitLab 12.6. @@ -257,7 +245,7 @@ there are no restrictions. To set a lifetime on how long access tokens are valid: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Account and limit** section. 1. Fill in the **Maximum allowable lifetime for access tokens (days)** field. @@ -291,7 +279,7 @@ To maintain integrity of user details in [Audit Events](../../../administration/ To do this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. 1. Select the **Prevent users from changing their profile name** checkbox. diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 638b61f6197..dab3c78d9d1 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -15,7 +15,7 @@ job artifacts. To enable (or disable) [Auto DevOps](../../../topics/autodevops/index.md) for all projects: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Check (or uncheck to disable) the box that says **Default to Auto DevOps pipeline for all projects**. 1. Optionally, set up the [Auto DevOps base domain](../../../topics/autodevops/requirements.md#auto-devops-base-domain) @@ -32,7 +32,7 @@ If you want to disable it for a specific project, you can do so in You can set all new projects to have the instance's shared runners available by default. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Continuous Integration and Deployment**. 1. Select the **Enable shared runners for new projects** checkbox. @@ -48,7 +48,7 @@ limit on the number of [CI/CD minutes](../../../ci/pipelines/cicd_minutes.md) yo To enable a specific runner for one or more projects: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. From the left sidebar, select **Overview > Runners**. 1. Select the runner you want to edit. 1. In the top right, select **Edit** (**{pencil}**). @@ -61,7 +61,7 @@ To enable a specific runner for one or more projects: To display details about the instance's shared runners in all projects' runner settings: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Continuous Integration and Deployment**. 1. Enter text, including Markdown if you want, in the **Shared runner details** field. For example: @@ -70,7 +70,7 @@ runner settings: To view the rendered details: -1. On the top bar, select **Menu > Project** and select any group or project. +1. On the top bar, select **Main menu > Projects** or **Main menu > Groups** and find your project or group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. @@ -90,7 +90,7 @@ The value is in MB and the default is 100MB per job. To change it at the: - Instance level: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD > Continuous Integration and Deployment**. 1. Change the value of **Maximum artifacts size (MB)**. 1. Select **Save changes** for the changes to take effect. @@ -117,7 +117,7 @@ can be set in the Admin Area of your GitLab instance. The syntax of duration is described in [`artifacts:expire_in`](../../../ci/yaml/index.md#artifactsexpire_in) and the default value is `30 days`. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Change the value of default expiration time. 1. Select **Save changes** for the changes to take effect. @@ -148,7 +148,7 @@ If disabled at the instance level, you cannot enable this per-project. To disable the setting: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Continuous Integration and Deployment**. 1. Clear the **Keep the latest artifacts for all jobs in the latest successful pipelines** checkbox. @@ -168,7 +168,7 @@ but persisting the traces and artifacts for auditing purposes. To set the duration for which the jobs are considered as old and expired: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand the **Continuous Integration and Deployment** section. 1. Set the value of **Archive jobs**. @@ -185,7 +185,7 @@ For the value set for GitLab.com, see [Scheduled job archiving](../../gitlab_com To set all new [CI/CD variables](../../../ci/variables/index.md) as [protected](../../../ci/variables/index.md#protected-cicd-variables) by default: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Select **Protect CI/CD variables by default**. @@ -196,7 +196,7 @@ To set all new [CI/CD variables](../../../ci/variables/index.md) as The default CI/CD configuration file and path for new projects can be set in the Admin Area of your GitLab instance (`.gitlab-ci.yml` if not set): -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Input the new file and path in the **Default CI/CD configuration file** field. 1. Hit **Save changes** for the changes to take effect. @@ -210,7 +210,7 @@ It is also possible to specify a [custom CI/CD configuration file for a specific You can configure some [CI/CD limits](../../../administration/instance_limits.md#cicd-limits) from the Admin Area: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand the **Continuous Integration and Deployment** section. 1. In the **CI/CD limits** section, you can set the following limits: @@ -232,7 +232,7 @@ walkthrough on how to add one. To enable or disable the banner: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Select or clear the **Enable pipeline suggestion banner** checkbox. 1. Select **Save changes**. @@ -267,7 +267,7 @@ in the pipeline editor. To select a CI/CD template for the required pipeline configuration: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand the **Required pipeline configuration** section. 1. Select a CI/CD template from the dropdown. @@ -275,13 +275,25 @@ To select a CI/CD template for the required pipeline configuration: ## Package Registry configuration +### Maven Forwarding **(PREMIUM SELF)** + +GitLab administrators can disable the forwarding of Maven requests to [Maven Central](https://search.maven.org/). + +To disable forwarding Maven requests: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand the **Package Registry** section. +1. Clear the checkbox **Forward Maven package requests to the Maven Registry if the packages are not found in the GitLab Package Registry**. +1. Select **Save changes**. + ### npm Forwarding **(PREMIUM SELF)** GitLab administrators can disable the forwarding of npm requests to [npmjs.com](https://www.npmjs.com/). To disable it: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand the **Package Registry** section. 1. Clear the checkbox **Forward npm package requests to the npm Registry if the packages are not found in the GitLab Package Registry**. @@ -293,7 +305,7 @@ GitLab administrators can disable the forwarding of PyPI requests to [pypi.org]( To disable it: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand the **Package Registry** section. 1. Clear the checkbox **Forward PyPI package requests to the PyPI Registry if the packages are not found in the GitLab Package Registry**. @@ -305,7 +317,7 @@ GitLab administrators can adjust the maximum allowed file size for each package To set the maximum file size: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand the **Package Registry** section. 1. Find the package type you would like to adjust. @@ -325,7 +337,7 @@ By default, all members of a project and group are able to register runners. To change this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runner registration**. 1. Clear the checkbox if you don't want to display runner registration diff --git a/doc/user/admin_area/settings/deprecated_api_rate_limits.md b/doc/user/admin_area/settings/deprecated_api_rate_limits.md index d651e445a95..7298d55b051 100644 --- a/doc/user/admin_area/settings/deprecated_api_rate_limits.md +++ b/doc/user/admin_area/settings/deprecated_api_rate_limits.md @@ -34,7 +34,7 @@ Prerequisites: To override the general user and IP rate limits for requests to deprecated API endpoints: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Deprecated API Rate Limits**. 1. Select the check boxes for the types of rate limits you want to enable: diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md index e4fc3b6e6d4..96fab153e85 100644 --- a/doc/user/admin_area/settings/email.md +++ b/doc/user/admin_area/settings/email.md @@ -11,7 +11,7 @@ You can customize some of the content in emails sent from your GitLab instance. ## Custom logo -The logo in the header of some emails can be customized, see the [logo customization section](../appearance.md#navigation-bar). +The logo in the header of some emails can be customized, see the [logo customization section](../appearance.md#top-bar). ## Include author name in email notification email body **(PREMIUM SELF)** @@ -21,7 +21,7 @@ address in the body of the email instead. To include the author's email address in the email body: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences** (`/admin/application_settings/preferences`). 1. Expand **Email**. 1. Select the **Include author name in email notification email body** checkbox. @@ -33,7 +33,7 @@ GitLab can send email in multipart format (HTML and plain text) or plain text on To enable multipart email: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences** (`/admin/application_settings/preferences`). 1. Expand **Email**. 1. Select **Enable multipart email**. @@ -48,7 +48,7 @@ This configuration option sets the email hostname for [private commit emails](.. To change the hostname used in private commit emails: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences** (`/admin/application_settings/preferences`). 1. Expand **Email**. 1. Enter the desired hostname in the **Custom hostname (for private commit emails)** field. @@ -66,7 +66,7 @@ can be used for legal, auditing, or compliance reasons, for example. To add additional text to emails: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences** (`/admin/application_settings/preferences`). 1. Expand **Email**. 1. Enter your text in the **Additional text** field. @@ -78,7 +78,7 @@ GitLab sends email notifications to users when their account has been deactivate To disable these notifications: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences** (`/admin/application_settings/preferences`). 1. Expand **Email**. 1. Clear the **Enable user deactivation emails** checkbox. diff --git a/doc/user/admin_area/settings/external_authorization.md b/doc/user/admin_area/settings/external_authorization.md index d6e6deb0274..9db85eb6ea8 100644 --- a/doc/user/admin_area/settings/external_authorization.md +++ b/doc/user/admin_area/settings/external_authorization.md @@ -47,7 +47,7 @@ Alternatively, learn where to install custom certificates by using The external authorization service can be enabled by an administrator: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **External authorization**. 1. Complete the fields. diff --git a/doc/user/admin_area/settings/files_api_rate_limits.md b/doc/user/admin_area/settings/files_api_rate_limits.md index 544c81e0583..963bca096a4 100644 --- a/doc/user/admin_area/settings/files_api_rate_limits.md +++ b/doc/user/admin_area/settings/files_api_rate_limits.md @@ -30,7 +30,7 @@ Prerequisite: To override the general user and IP rate limits for requests to the Repository files API: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Files API Rate Limits**. 1. Select the check boxes for the types of rate limits you want to enable: diff --git a/doc/user/admin_area/settings/floc.md b/doc/user/admin_area/settings/floc.md index dccab461b85..b7d3d8bfa20 100644 --- a/doc/user/admin_area/settings/floc.md +++ b/doc/user/admin_area/settings/floc.md @@ -22,10 +22,10 @@ Permissions-Policy: interest-cohort=() To enable it: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. -1. Expand **Federated Learning of Cohorts**. -1. Check the box. +1. Expand **Federated Learning of Cohorts (FLoC)**. +1. Select the **Participate in FLoC** checkbox. 1. Select **Save changes**. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/settings/git_lfs_rate_limits.md b/doc/user/admin_area/settings/git_lfs_rate_limits.md index c10300baeef..519bbbef4e4 100644 --- a/doc/user/admin_area/settings/git_lfs_rate_limits.md +++ b/doc/user/admin_area/settings/git_lfs_rate_limits.md @@ -21,7 +21,7 @@ rate limits. Git LFS rate limits are disabled by default. If enabled and configured, these limits supersede the [general user and IP rate limits](user_and_ip_rate_limits.md): -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Main menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Git LFS Rate Limits**. 1. Select **Enable authenticated Git LFS request rate limit**. diff --git a/doc/user/admin_area/settings/gitaly_timeouts.md b/doc/user/admin_area/settings/gitaly_timeouts.md index 8866a044241..dcf7574136a 100644 --- a/doc/user/admin_area/settings/gitaly_timeouts.md +++ b/doc/user/admin_area/settings/gitaly_timeouts.md @@ -11,7 +11,7 @@ configured to make sure that long-running Gitaly calls don't needlessly take up To access Gitaly timeout settings: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand the **Gitaly timeouts** section. diff --git a/doc/user/admin_area/settings/help_page.md b/doc/user/admin_area/settings/help_page.md index 0bf5dc1d37a..38161d0607c 100644 --- a/doc/user/admin_area/settings/help_page.md +++ b/doc/user/admin_area/settings/help_page.md @@ -16,7 +16,7 @@ the GitLab sign-in page. You can add a help message, which is shown at the top of the GitLab `/help` page (for example, <https://gitlab.com/help>): -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Sign-in and Help page**. 1. In **Additional text to show on the Help page**, enter the information you want to display on `/help`. @@ -31,10 +31,9 @@ is restricted, `/help` is visible only to signed-in users. ## Add a help message to the sign-in page -You can add a help message, which is shown on the GitLab sign-in page. The message appears in a new -section titled **Need Help?**, located below the sign-in page message: +You can add a help message, which is shown on the GitLab sign-in page. The message appears on the sign-in page: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Sign-in and Help page**. 1. In **Additional text to show on the sign-in page**, enter the information you want to @@ -47,7 +46,7 @@ You can now see the message on the sign-in page. GitLab marketing-related entries are occasionally shown on the Help page. To hide these entries: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Sign-in and Help page**. 1. Select the **Hide marketing-related entries from the Help page** checkbox. @@ -60,7 +59,7 @@ You can specify a custom URL to which users are directed when they: - Select **Support** from the Help dropdown. - Select **See our website for help** on the Help page. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Sign-in and Help page**. 1. In the **Support page URL** field, enter the URL. @@ -81,7 +80,7 @@ You can redirect these `/help` links to either: - The more navigable and searchable version published at [`docs.gitlab.com`](https://docs.gitlab.com). - A destination that meets [necessary requirements](#destination-requirements). -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Sign-in and Help page**. 1. In the **Documentation pages URL** field, enter the URL. diff --git a/doc/user/admin_area/settings/import_export_rate_limits.md b/doc/user/admin_area/settings/import_export_rate_limits.md index 7d5a928eedf..cfe4f49388e 100644 --- a/doc/user/admin_area/settings/import_export_rate_limits.md +++ b/doc/user/admin_area/settings/import_export_rate_limits.md @@ -13,7 +13,7 @@ You can configure the rate limits for imports and exports of projects and groups To change a rate limit: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**, then expand **Import and export rate limits**. 1. Change the value of any rate limit. The rate limits are per minute per user, not per IP address. Set to `0` to disable a rate limit. diff --git a/doc/user/admin_area/settings/incident_management_rate_limits.md b/doc/user/admin_area/settings/incident_management_rate_limits.md index ed2d707af0a..f9e91d26db5 100644 --- a/doc/user/admin_area/settings/incident_management_rate_limits.md +++ b/doc/user/admin_area/settings/incident_management_rate_limits.md @@ -30,7 +30,7 @@ Requests that exceed the limit are logged into `auth.log`. To set inbound incident management alert limits: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Incident Management Limits**. 1. Select the **Enable Incident Management inbound alert limit** checkbox. diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index 2e27b213f16..7a63f2059ba 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -19,7 +19,7 @@ read [GitLab.com settings](../../gitlab_com/index.md). To access the **Admin Area**: 1. Sign in to your GitLab instance as an administrator. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings**, and the group of settings to view: - [General](#general) - [Geo](#geo) @@ -197,6 +197,6 @@ The **Templates** settings contain: You can change the [Default first day of the week](../../profile/preferences.md) for the entire GitLab instance: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Scroll to the **Localization** section, and select your desired first day of the week. diff --git a/doc/user/admin_area/settings/instance_template_repository.md b/doc/user/admin_area/settings/instance_template_repository.md index 51695ef7fd2..0d63fe5db7f 100644 --- a/doc/user/admin_area/settings/instance_template_repository.md +++ b/doc/user/admin_area/settings/instance_template_repository.md @@ -20,7 +20,7 @@ while the project remains secure. To select a project to serve as the custom template repository: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Templates**. 1. Expand **Templates** 1. From the dropdown list, select the project to use as the template repository. diff --git a/doc/user/admin_area/settings/package_registry_rate_limits.md b/doc/user/admin_area/settings/package_registry_rate_limits.md index 1aeb011d880..62cddd2781c 100644 --- a/doc/user/admin_area/settings/package_registry_rate_limits.md +++ b/doc/user/admin_area/settings/package_registry_rate_limits.md @@ -30,7 +30,7 @@ no difference in functionality compared to the general user and IP rate limits. To enable the unauthenticated request rate limit: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**, and expand **Package registry rate limits**. 1. Select **Enable unauthenticated request rate limit**. @@ -43,7 +43,7 @@ To enable the unauthenticated request rate limit: To enable the authenticated API request rate limit: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**, and expand **Package registry rate limits**. 1. Select **Enable authenticated API request rate limit**. diff --git a/doc/user/admin_area/settings/project_integration_management.md b/doc/user/admin_area/settings/project_integration_management.md index 010ba6a12fc..58afc6a104c 100644 --- a/doc/user/admin_area/settings/project_integration_management.md +++ b/doc/user/admin_area/settings/project_integration_management.md @@ -22,7 +22,7 @@ Only the complete settings for an integration can be inherited. Per-field inheri > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2137) in GitLab 13.3 for project-level integrations. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2543) in GitLab 13.6 for group-level integrations. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Integrations**. 1. Select an integration. 1. Enter configuration details and select **Save changes**. @@ -54,7 +54,7 @@ integration on all non-configured groups and projects by default. ### Remove an instance-level default setting -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Integrations**. 1. Select an integration. 1. Select **Reset** and confirm. @@ -68,7 +68,7 @@ Resetting an instance-level default setting removes the integration from all pro You can view which projects in your instance use custom settings that [override the instance-level default settings](#use-custom-settings-for-a-group-or-project-integration) for an integration. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Integrations**. 1. Select an integration. 1. Select the **Projects using custom settings** tab. diff --git a/doc/user/admin_area/settings/push_event_activities_limit.md b/doc/user/admin_area/settings/push_event_activities_limit.md index 760ce96d987..cd982bb4aa3 100644 --- a/doc/user/admin_area/settings/push_event_activities_limit.md +++ b/doc/user/admin_area/settings/push_event_activities_limit.md @@ -26,7 +26,7 @@ the activity feed. To modify this setting: - In the Admin Area: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**, then expand **Performance optimization**. - Through the [Application settings API](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls) as `push_event_activities_limit`. diff --git a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md index 6c0c15243da..8d08da8246a 100644 --- a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md +++ b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md @@ -12,7 +12,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w This setting allows you to rate limit the requests to the issue and epic creation endpoints. To can change its value: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Issues Rate Limits**. 1. Under **Max requests per minute**, enter the new value. diff --git a/doc/user/admin_area/settings/rate_limit_on_notes_creation.md b/doc/user/admin_area/settings/rate_limit_on_notes_creation.md index 0a07cf095ee..f55c2f3bd4a 100644 --- a/doc/user/admin_area/settings/rate_limit_on_notes_creation.md +++ b/doc/user/admin_area/settings/rate_limit_on_notes_creation.md @@ -13,7 +13,7 @@ You can configure the per-user rate limit for requests to the note creation endp To change the note creation rate limit: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Notes rate limit**. 1. In the **Maximum requests per minute** box, enter the new value. diff --git a/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md b/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md index fce6179f5cf..ba383d74701 100644 --- a/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md +++ b/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can set a limit so that users and processes can't request more than a certain number of pipelines each minute. This limit can help save resources and improve stability. -For example, if you set a limit of `10`, and `11` requests are sent to the [trigger API](../../../ci/triggers/) within one minute, +For example, if you set a limit of `10`, and `11` requests are sent to the [trigger API](../../../ci/triggers/index.md) within one minute, the eleventh request is blocked. Access to the endpoint is allowed again after one minute. This limit is: @@ -26,7 +26,7 @@ Requests that exceed the limit are logged in the `application_json.log` file. To limit the number of pipeline requests: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Pipelines Rate Limits**. 1. Under **Max requests per minute**, enter a value greater than `0`. diff --git a/doc/user/admin_area/settings/rate_limit_on_users_api.md b/doc/user/admin_area/settings/rate_limit_on_users_api.md index 5eed989f73f..9792fd1000d 100644 --- a/doc/user/admin_area/settings/rate_limit_on_users_api.md +++ b/doc/user/admin_area/settings/rate_limit_on_users_api.md @@ -13,7 +13,7 @@ You can configure the per user rate limit for requests to [Users API](../../../a To change the rate limit: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Users API rate limit**. 1. In the **Maximum requests per 10 minutes** text box, enter the new value. diff --git a/doc/user/admin_area/settings/rate_limits_on_raw_endpoints.md b/doc/user/admin_area/settings/rate_limits_on_raw_endpoints.md index 028d5e4c2f3..cb3ca0fe756 100644 --- a/doc/user/admin_area/settings/rate_limits_on_raw_endpoints.md +++ b/doc/user/admin_area/settings/rate_limits_on_raw_endpoints.md @@ -11,7 +11,7 @@ type: reference This setting defaults to `300` requests per minute, and allows you to rate limit the requests to raw endpoints: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Performance optimization**. diff --git a/doc/user/admin_area/settings/sidekiq_job_limits.md b/doc/user/admin_area/settings/sidekiq_job_limits.md index 750665285b4..c1990572aee 100644 --- a/doc/user/admin_area/settings/sidekiq_job_limits.md +++ b/doc/user/admin_area/settings/sidekiq_job_limits.md @@ -17,7 +17,7 @@ Redis. To avoid excessive memory for Redis, we: To access Sidekiq job size limits: -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Sidekiq job size limits**. 1. Adjust the compression threshold or size limit. The compression can diff --git a/doc/user/admin_area/settings/sign_in_restrictions.md b/doc/user/admin_area/settings/sign_in_restrictions.md index 7316b1bdbb8..bdffe87b75c 100644 --- a/doc/user/admin_area/settings/sign_in_restrictions.md +++ b/doc/user/admin_area/settings/sign_in_restrictions.md @@ -12,7 +12,7 @@ You can use **Sign-in restrictions** to customize authentication restrictions fo To access sign-in restriction settings: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Sign-in restrictions** section. @@ -26,7 +26,7 @@ You can restrict the password authentication for web interface and Git over HTTP - **Git over HTTP(S)**: When this feature is disabled, a [Personal Access Token](../../profile/personal_access_tokens.md) or LDAP password must be used to authenticate. -In the event of an external authentication provider outage, use the [GitLab Rails console](../../../administration/operations/rails_console.md) to [re-enable the standard web sign-in form](../../../administration/troubleshooting/gitlab_rails_cheat_sheet.md#re-enable-standard-web-sign-in-form). This configuration can also be changed over the [Application settings REST API](../../../api/settings.md#change-application-settings) while authenticating with an administrator account's personal access token. +In the event of an external authentication provider outage, use the [GitLab Rails console](../../../administration/operations/rails_console.md) to [re-enable the standard web sign-in form](#re-enable-standard-web-sign-in-form-in-rails-console). This configuration can also be changed over the [Application settings REST API](../../../api/settings.md#change-application-settings) while authenticating with an administrator account's personal access token. ## Admin Mode @@ -121,21 +121,21 @@ For example, if you include the following information in the noted text box: To access this text box: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, and expand the **Sign-in restrictions** section. ``` Your users see the **Custom sign-in text** when they navigate to the sign-in screen for your GitLab instance. -<!-- ## Troubleshooting +## Troubleshooting -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### Re-enable standard web sign-in form in rails console -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +Re-enable the standard username and password-based sign-in form if it was disabled as a [Sign-in restriction](#password-authentication-enabled). + +You can use this method through the [rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session) when a configured external authentication provider (through SSO or an LDAP configuration) is facing an outage and direct sign-in access to GitLab is required. + +```ruby +Gitlab::CurrentSettings.update!(password_authentication_enabled_for_web: true) +``` diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index 56abb3d4701..a0ec964e8db 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -22,7 +22,7 @@ you do not expect public users to sign up for an account. To disable sign ups: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, and expand **Sign-up restrictions**. 1. Clear the **Sign-up enabled** checkbox, then select **Save changes**. @@ -38,7 +38,7 @@ enabled by default for new GitLab instances. It is only applicable if sign ups a To require administrator approval for new sign ups: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, and expand **Sign-up restrictions**. 1. Select the **Require admin approval for new sign-ups** checkbox, then select **Save changes**. @@ -58,7 +58,7 @@ their email address before they are allowed to sign in. To enforce confirmation of the email address used for new sign ups: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, and expand **Sign-up restrictions**. 1. Select the **Send confirmation email on sign-up** checkbox, then select **Save changes**. @@ -76,7 +76,7 @@ user cap, the users in pending approval state are automatically approved in a ba ### Set the user cap number -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Sign-up restrictions**. 1. Enter a number in **User cap**. @@ -86,7 +86,7 @@ New user sign ups are subject to the user cap restriction. ## Remove the user cap -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Sign-up restrictions**. 1. Remove the number from **User cap**. @@ -130,7 +130,7 @@ You can add additional complexity requirements. Changes to password complexity r Existing passwords are unaffected. To change password complexity requirements: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Sign-up restrictions**. 1. Under **Minimum password length (number of characters)**, select additional password complexity requirements. You can require numbers, uppercase letters, lowercase letters, @@ -159,7 +159,7 @@ reduce the risk of malicious users creating spam accounts with disposable email To create an email domain allowlist or denylist: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, and expand **Sign-up restrictions**. 1. For the allowlist, you must enter the list manually. For the denylist, you can enter the list manually or upload a `.txt` file that contains list entries. diff --git a/doc/user/admin_area/settings/terms.md b/doc/user/admin_area/settings/terms.md index 870fa7ad18e..d26ace161bb 100644 --- a/doc/user/admin_area/settings/terms.md +++ b/doc/user/admin_area/settings/terms.md @@ -17,7 +17,7 @@ for example `https://gitlab.example.com/-/users/terms`. To enforce acceptance of a Terms of Service and Privacy Policy: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Terms of Service and Privacy Policy** section. 1. Check the **All users must accept the Terms of Service and Privacy Policy to access GitLab** checkbox. diff --git a/doc/user/admin_area/settings/third_party_offers.md b/doc/user/admin_area/settings/third_party_offers.md index 04ec9ed652f..59c100dc016 100644 --- a/doc/user/admin_area/settings/third_party_offers.md +++ b/doc/user/admin_area/settings/third_party_offers.md @@ -18,7 +18,7 @@ questions when creating a group. To toggle the display of customer experience improvement content and third-party offers: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings**, and expand **Customer experience improvement & third-party offers**. 1. Select **Do not display content for customer experience improvement and offers from third parties**. 1. Select **Save changes**. diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index afb937494e0..ad018abf809 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -56,7 +56,7 @@ Registration is not yet required for participation, but may be added in a future ### Enable registration features 1. Sign in as a user with administrator access. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Expand the **Usage statistics** section. 1. If not enabled, select the **Enable Service Ping** checkbox. @@ -113,7 +113,7 @@ If your GitLab instance is behind a proxy, set the appropriate To enable or disable Service Ping and version check: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Expand **Usage statistics**. 1. Select or clear the **Enable version check** and **Enable Service Ping** checkboxes. @@ -165,7 +165,7 @@ the Admin Area: You can view the exact JSON payload sent to GitLab Inc. in the Admin Area. To view the payload: 1. Sign in as a user with administrator access. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Expand the **Usage statistics** section. 1. Select **Preview payload**. @@ -183,7 +183,7 @@ or if the Service Ping [cron job](../../../development/service_ping/index.md#how To upload the payload manually: 1. Sign in as a user with administrator access. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Service** usage data. 1. Select **Download payload**. 1. Save the JSON file. diff --git a/doc/user/admin_area/settings/user_and_ip_rate_limits.md b/doc/user/admin_area/settings/user_and_ip_rate_limits.md index a35cbe5381a..86676e4a63e 100644 --- a/doc/user/admin_area/settings/user_and_ip_rate_limits.md +++ b/doc/user/admin_area/settings/user_and_ip_rate_limits.md @@ -31,7 +31,7 @@ counted as web traffic. To enable the unauthenticated request rate limit: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**, and expand **User and IP rate limits**. 1. Select **Enable unauthenticated API request rate limit**. @@ -44,7 +44,7 @@ To enable the unauthenticated request rate limit: To enable the unauthenticated request rate limit: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**, and expand **User and IP rate limits**. 1. Select **Enable unauthenticated web request rate limit**. @@ -57,7 +57,7 @@ To enable the unauthenticated request rate limit: To enable the authenticated API request rate limit: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**, and expand **User and IP rate limits**. 1. Select **Enable authenticated API request rate limit**. @@ -70,7 +70,7 @@ To enable the authenticated API request rate limit: To enable the unauthenticated request rate limit: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**, and expand **User and IP rate limits**. 1. Select **Enable authenticated web request rate limit**. @@ -88,7 +88,7 @@ plain-text body, which by default is `Retry later`. To use a custom response: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Network**, and expand **User and IP rate limits**. 1. In the **Plain-text response to send to clients that hit a rate limit** text box, add the plain-text response message. diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index 118d375da01..87c24f04a1c 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -13,7 +13,7 @@ specific controls on branches, projects, snippets, groups, and more. To access the visibility and access control options: 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. @@ -24,7 +24,7 @@ Instance-level protections for project creation define which roles can on the instance. To alter which roles have permission to create projects: 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. For **Default project creation protection**, select the desired roles: @@ -40,7 +40,7 @@ on the instance. To alter which roles have permission to create projects: By default both administrators and anyone with the **Owner** role can delete a project. To restrict project deletion to only administrators: 1. Sign in to GitLab as a user with administrator access. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Scroll to: @@ -78,7 +78,7 @@ deleted groups will remain restorable within a retention period. To configure delayed project deletion: 1. Sign in to GitLab as a user with administrator access. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Scroll to: @@ -112,7 +112,7 @@ Alternatively, projects that are marked for removal can be deleted immediately. To set the default [visibility levels for new projects](../../public_access.md): 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Select the desired default project visibility: @@ -127,7 +127,7 @@ To set the default [visibility levels for new projects](../../public_access.md): To set the default visibility levels for new [snippets](../../snippets.md): 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Select the desired default snippet visibility. @@ -141,7 +141,7 @@ For more details on snippet visibility, read To set the default visibility levels for new groups: 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Select the desired default group visibility: @@ -158,7 +158,7 @@ For more details on group visibility, see To restrict visibility levels for projects, snippets, and selected pages: 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. In the **Restricted visibility levels** section, select the desired visibility levels to restrict. @@ -177,7 +177,7 @@ For more details on project visibility, see You can specify from which hosting sites users can [import their projects](../../project/import/index.md): 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Select each of **Import sources** to allow. @@ -189,7 +189,7 @@ To enable the export of [projects and their data](../../project/settings/import_export.md#export-a-project-and-its-data): 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Select **Project export enabled**. @@ -205,7 +205,7 @@ The GitLab restrictions apply at the application level. To specify the enabled Git access protocols: 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Select the desired Git access protocols: @@ -277,13 +277,8 @@ work in every repository. They can only be re-enabled by an administrator user o ## Configure globally-allowed IP address ranges -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87579) in GitLab 15.1 [with a flag](../../../administration/feature_flags.md) named `group_ip_restrictions_allow_global`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available -per group, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) -named `group_ip_restrictions_allow_global`. -On GitLab.com, this feature is available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87579) in GitLab 15.1 [with a flag](../../../administration/feature_flags.md) named `group_ip_restrictions_allow_global`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/366445) in GitLab 15.4. [Feature flag `group_ip_restrictions_allow_global`](https://gitlab.com/gitlab-org/gitlab/-/issues/366445) removed. This setting allows you to set IP address ranges to be combined with group-level IP allowlists. It helps administrators prevent aspects of the GitLab installation from being blocked @@ -296,7 +291,7 @@ daemon to be unable to fetch artifacts from the pipeline runs. To add a IP address range to the group-level allowlist: 1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. In **Globally-allowed IP ranges**, provide a value. diff --git a/doc/user/admin_area/user_cohorts.md b/doc/user/admin_area/user_cohorts.md index 28376923810..816e629f496 100644 --- a/doc/user/admin_area/user_cohorts.md +++ b/doc/user/admin_area/user_cohorts.md @@ -10,7 +10,7 @@ You can analyze your users' GitLab activities over time. To view user cohorts: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Cohorts** tab. diff --git a/doc/user/analytics/ci_cd_analytics.md b/doc/user/analytics/ci_cd_analytics.md index f4075c3420b..b92d68222f5 100644 --- a/doc/user/analytics/ci_cd_analytics.md +++ b/doc/user/analytics/ci_cd_analytics.md @@ -32,7 +32,7 @@ View pipeline duration history: To view CI/CD analytics: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > CI/CD Analytics**. ## View deployment frequency chart **(ULTIMATE)** @@ -50,7 +50,7 @@ The deployment frequency chart is available for groups and projects. To view the deployment frequency chart: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > CI/CD Analytics**. 1. Select the **Deployment frequency** tab. @@ -68,11 +68,11 @@ merge requests to be deployed to a production environment. This chart is availab - For time periods in which no merge requests were deployed, the charts render a red, dashed line. - lead time for changes is one of the four [DORA metrics](index.md#devops-research-and-assessment-dora-key-metrics) that DevOps teams use for measuring excellence in software delivery. + Lead time for changes is one of the four [DORA metrics](index.md#devops-research-and-assessment-dora-key-metrics) that DevOps teams use for measuring excellence in software delivery. To view the lead time for changes chart: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > CI/CD Analytics**. 1. Select the **Lead time** tab. @@ -88,7 +88,7 @@ Time to restore service is one of the four [DORA metrics](index.md#devops-resear To view the time to restore service chart: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > CI/CD Analytics**. 1. Select the **Time to restore service** tab. @@ -104,6 +104,6 @@ Change failure rate is one of the four [DORA metrics](index.md#devops-research-a To view the change failure rate chart: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > CI/CD Analytics**. 1. Select the **Change failure rate** tab. diff --git a/doc/user/analytics/code_review_analytics.md b/doc/user/analytics/code_review_analytics.md index dc02512702a..dd985b6b090 100644 --- a/doc/user/analytics/code_review_analytics.md +++ b/doc/user/analytics/code_review_analytics.md @@ -6,53 +6,32 @@ info: To determine the technical writer assigned to the Stage/Group associated w --- -# Code Review Analytics **(PREMIUM)** +# Code review analytics **(PREMIUM)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38062) in GitLab 12.7. -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. -Use Code Review Analytics to view the longest-running reviews among open merge -requests, and: +Use code review analytics to view review metrics per merge request and +make improvements to your code review process: -- Take action on individual merge requests. -- Reduce overall cycle time. +- A high number of comments or commits may indicate: + - The code is too complex. + - Authors who require more training. +- A long review time may indicate: + - Types of work that move slower than other types. + - Opportunities to accelerate your development cycle. +- Fewer comments and approvers may indicate staffing requirements. -Code Review Analytics is available to users with at least the Reporter role, and displays a table of open merge requests that have at least one non-author comment. The review time is measured from the time the first non-author comment was submitted. +Code review analytics displays a table of open merge requests that have at least one non-author comment. +The review time is measured from when the first non-author comment was submitted. -NOTE: -Initially, no data appears. Data is populated as users comment on open merge requests. +## View code review analytics - +Prerequisite: -The table is sorted by: +- You must have at least the Reporter role. -- **Review time**: Helping you to quickly find the longest-running reviews which may need intervention - or to be broken down into smaller parts. -- Other columns: Display the author, approvers, comment count, and line change (-/+) counts. +To view code review analytics: -## View Code Review Analytics - -To view Code Review Analytics: - -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Code Review**. 1. Filter merge requests by milestone and label. - -## Use cases - -This feature is designed for [development team leaders](https://about.gitlab.com/handbook/product/personas/#delaney-development-team-lead) -and others who want to understand broad code review dynamics, and identify patterns to explain them. - -You can use Code Review Analytics to: - -- Expose your team's unique challenges with code review. -- Identify improvements that might substantially accelerate your development cycle. -- Your team agrees that code review is moving too slow. -- The [Value Stream Analytics feature](value_stream_analytics.md) shows that reviews are your team's most time-consuming step. -- Analyze the patterns and trends of different types of work that are moving slow. - -For example: - -- Lots of comments or commits? Maybe the code is too complex. -- A particular author is involved? Maybe more training is required. -- Few comments and approvers? Maybe your team is understaffed. diff --git a/doc/user/analytics/img/code_review_analytics_v13_11.png b/doc/user/analytics/img/code_review_analytics_v13_11.png Binary files differdeleted file mode 100644 index b559b934a89..00000000000 --- a/doc/user/analytics/img/code_review_analytics_v13_11.png +++ /dev/null diff --git a/doc/user/analytics/issue_analytics.md b/doc/user/analytics/issue_analytics.md index 62fff443073..86ae45a02b8 100644 --- a/doc/user/analytics/issue_analytics.md +++ b/doc/user/analytics/issue_analytics.md @@ -15,7 +15,7 @@ prior. To access the chart: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Issue**. Hover over each bar to see the total number of issues. diff --git a/doc/user/analytics/merge_request_analytics.md b/doc/user/analytics/merge_request_analytics.md index 038b2f0c97e..4f40575a4ef 100644 --- a/doc/user/analytics/merge_request_analytics.md +++ b/doc/user/analytics/merge_request_analytics.md @@ -28,7 +28,7 @@ You must have at least the Reporter role to view merge request analytics. To view merge request analytics: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Merge request**. ## View the number of merge requests in a date range @@ -38,7 +38,7 @@ To view merge request analytics: To view the number of merge requests merged during a specific date range: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Merge request**. 1. Optional. Filter results: 1. Select the filter bar. @@ -63,6 +63,6 @@ created and when it's merged. Closed and un-merged merge requests are not includ To view **Mean time to merge**: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Merge request**. The **Mean time to merge** number is shown on the dashboard. diff --git a/doc/user/analytics/productivity_analytics.md b/doc/user/analytics/productivity_analytics.md index f8b28ead155..f0841dc979b 100644 --- a/doc/user/analytics/productivity_analytics.md +++ b/doc/user/analytics/productivity_analytics.md @@ -26,7 +26,7 @@ Prerequisite: - You must have at least the Reporter role for the group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Productivity**. 1. Optional. Filter results: 1. Select a project from the dropdown list. @@ -44,7 +44,7 @@ Use the following charts in productivity analytics to view the velocity of your merge requests to merge after they were created. - **Trendline**: number of merge requests that were merged in a specific time period. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Productivity**. To filter time metrics: @@ -56,7 +56,7 @@ To filter time metrics: To view commit statistics for your group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Productivity**. 1. Under the **Trendline** scatterplot, view the commit statistics: - The left histogram shows the number of hours between commits, comments, and merges. diff --git a/doc/user/analytics/repository_analytics.md b/doc/user/analytics/repository_analytics.md index 5fd4a567b58..7723da7397d 100644 --- a/doc/user/analytics/repository_analytics.md +++ b/doc/user/analytics/repository_analytics.md @@ -30,7 +30,7 @@ Commits in a project's [wiki](../project/wiki/index.md#track-wiki-events) are no To review repository analytics for a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Repository**. ## How repository analytics chart data is updated diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index a71136628cf..d5756c5f822 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -32,7 +32,7 @@ Value stream analytics is also available for [groups](../group/value_stream_anal To view value stream analytics for your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Value stream**. 1. To view metrics for a particular stage, select a stage below the **Filter results** text box. 1. Optional. Filter the results: @@ -61,7 +61,7 @@ Value stream analytics shows the median time spent by issues or merge requests i To view the median time spent in each stage: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Value stream**. 1. Optional. Filter the results: 1. Select the **Filter results** text box. @@ -81,7 +81,7 @@ Value stream analytics shows the lead time and cycle time for issues in your pro To view the lead time and cycle time for issues: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Value stream**. 1. Optional. Filter the results: 1. Select the **Filter results** text box. @@ -101,7 +101,7 @@ Lead time for changes is the median duration between when a merge request is mer To view the lead time for changes for merge requests in your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Value stream**. 1. Optional. Filter the results: 1. Select the **Filter results** text box. @@ -113,19 +113,26 @@ To view the lead time for changes for merge requests in your project: The **Lead Time for Changes** metrics display below the **Filter results** text box. -## View number of successful deployments **(PREMIUM)** +## View number of successful deployments **(FREE)** -To view deployment metrics, you must have a +Prerequisites: + +- To view deployment metrics, you must have a [production environment configured](../../ci/environments/index.md#deployment-tier-of-environments). -Value stream analytics shows the following deployment metrics for your project: +Value stream analytics shows the following deployment metrics for your project within the specified date range: - Deploys: The number of successful deployments in the date range. - Deployment Frequency: The average number of successful deployments per day in the date range. +If you have a GitLab Premium or Ultimate subscription: + +- The number of successful deployments is calculated with DORA data. +- The data is filtered based on environment and environment tier. + To view deployment metrics for your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Value stream**. 1. Optional. Filter the results: 1. Select the **Filter results** text box. diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index 80e4700c34c..8e371ed4dc6 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -39,6 +39,7 @@ or other scanners) during a scan could cause inaccurate results. You can run a Web API fuzzing scan using the following methods: - [OpenAPI Specification](#openapi-specification) - version 2, and 3. +- [GraphQL Schema](#graphql-schema) - [HTTP Archive](#http-archive-har) (HAR) - [Postman Collection](#postman-collection) - version 2.0 or 2.1 @@ -76,6 +77,7 @@ To enable Web API fuzzing: - For manual configuration instructions, see the respective section, depending on the API type: - [OpenAPI Specification](#openapi-specification) + - [GraphQL Schema](#graphql-schema) - [HTTP Archive (HAR)](#http-archive-har) - [Postman Collection](#postman-collection) - Otherwise, see [Web API fuzzing configuration form](#web-api-fuzzing-configuration-form). @@ -95,7 +97,7 @@ a YAML snippet that you can paste in your GitLab CI/CD configuration. To generate an API Fuzzing configuration snippet: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **API Fuzzing** row, select **Enable API Fuzzing**. 1. Complete the fields. For details see [Available CI/CD variables](#available-cicd-variables). @@ -262,7 +264,7 @@ Example `.gitlab-ci.yml` file using a HAR file: FUZZAPI_TARGET_URL: http://test-deployment/ ``` -This is a minimal configuration for API fuzzing. From here you can: +This example is a minimal configuration for API fuzzing. From here you can: - [Run your first scan](#running-your-first-scan). - [Add authentication](#authentication). @@ -270,6 +272,118 @@ This is a minimal configuration for API fuzzing. From here you can: For details of API fuzzing configuration options, see [Available CI/CD variables](#available-cicd-variables). +### GraphQL Schema + +> Support for GraphQL Schema was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352780) in GitLab 15.4. + +GraphQL is a query language for your API and an alternative to REST APIs. +API Fuzzing supports testing GraphQL endpoints multiple ways: + +- Test using the GraphQL Schema. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352780) in GitLab 15.4. +- Test using a recording (HAR) of GraphQL queries. +- Test using a Postman Collection containing GraphQL queries. + +This section documents how to test using a GraphQL schema. The GraphQL schema support in +API Fuzzing is able to query the schema from endpoints that support introspection. +Introspection is enabled by default to allow tools like GraphiQL to work. + +#### API Fuzzing scanning with a GraphQL endpoint URL + +The GraphQL support in API Fuzzing is able to query a GraphQL endpoint for the schema. + +NOTE: +The GraphQL endpoint must support introspection queries for this method to work correctly. + +To configure API Fuzzing to use an GraphQL endpoint URL that provides information about the target API to test: + +1. [Include](../../../ci/yaml/index.md#includetemplate) + the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) in your `.gitlab-ci.yml` file. + +1. Provide the GraphQL endpoint path, for example `/api/graphql`. Specify the path by adding the `FUZZAPI_GRAPHQL` variable. + +1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. See the [dynamic environment solutions](#dynamic-environment-solutions) section of our documentation for more information. + +Complete example configuration of using a GraphQL endpoint URL: + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +apifuzzer_fuzz: + variables: + FUZZAPI_GRAPHQL: /api/graphql + FUZZAPI_TARGET_URL: http://test-deployment/ +``` + +This example is a minimal configuration for API Fuzzing. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +#### API Fuzzing with a GraphQL Schema file + +To configure API Fuzzing to use a GraphQl schema file that provides information about the target API to test: + +1. [Include](../../../ci/yaml/index.md#includetemplate) + the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) in your `.gitlab-ci.yml` file. + +1. Provide the GraphQL endpoint path, for example `/api/graphql`. Specify the path by adding the `FUZZAPI_GRAPHQL` variable. + +1. Provide the location of the GraphQL schema file. You can provide the location as a file path + or URL. Specify the location by adding the `FUZZAPI_GRAPHQL_SCHEMA` variable. + +1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. See the [dynamic environment solutions](#dynamic-environment-solutions) section of our documentation for more information. + +Complete example configuration of using an GraphQL schema file: + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +apifuzzer_fuzz: + variables: + FUZZAPI_GRAPHQL: /api/graphql + FUZZAPI_GRAPHQL_SCHEMA: test-api-graphql.schema + FUZZAPI_TARGET_URL: http://test-deployment/ +``` + +Complete example configuration of using an GraphQL schema file URL: + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +apifuzzer_fuzz: + variables: + FUZZAPI_GRAPHQL: /api/graphql + FUZZAPI_GRAPHQL_SCHEMA: http://file-store/files/test-api-graphql.schema + FUZZAPI_TARGET_URL: http://test-deployment/ +``` + +This example is a minimal configuration for API Fuzzing. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + ### Postman Collection The [Postman API Client](https://www.postman.com/product/api-client/) is a popular tool that @@ -344,34 +458,264 @@ For details of API fuzzing configuration options, see [Available CI/CD variables #### Postman variables +> - Support for Postman Environment file format was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356312) in GitLab 15.1. +> - Support for multiple variable files was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356312) in GitLab 15.1. +> - Support for Postman variable scopes: Global and Environment was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356312) in GitLab 15.1. + +##### Variables in Postman Client + Postman allows the developer to define placeholders that can be used in different parts of the -requests. These placeholders are called variables, as explained in the Postman documentation, -[Using variables](https://learning.postman.com/docs/sending-requests/variables/). +requests. These placeholders are called variables, as explained in [using variables](https://learning.postman.com/docs/sending-requests/variables/). You can use variables to store and reuse values in your requests and scripts. For example, you can edit the collection to add variables to the document:  +Or alternatively, you can add variables in an environment: + + + You can then use the variables in sections such as URL, headers, and others:  -Variables can be defined at different [scopes](https://learning.postman.com/docs/sending-requests/variables/#variable-scopes) -(for example, Global, Collection, Environment, Local, and Data). In this example, they're defined at -the Environment scope: +Postman has grown from a basic client tool with a nice UX experience to a more complex ecosystem that allows testing APIs with scripts, creating complex collections that trigger secondary requests, and setting variables along the way. Not every feature in the Postman ecosystem is supported. For example, scripts are not supported. The main focus of the Postman support is to ingest Postman Collection definitions that are used by the Postman Client and their related variables defined in the workspace, environments, and the collections themselves. - +Postman allows creating variables in different scopes. Each scope has a different level of visibility in the Postman tools. For example, you can create a variable in a _global environment_ scope that is seen by every operation definition and workspace. You can also create a variable in a specific _environment_ scope that is only visible and used when that specific environment is selected for use. Some scopes are not always available, for example in the Postman ecosystem you can create requests in the Postman Client, these requests do not have a _local_ scope, but test scripts do. -When you export a Postman collection, only Postman collection variables are exported into the -Postman file. For example, Postman does not export environment-scoped variables into the Postman -file. +Variable scopes in Postman can be a daunting topic and not everyone is familiar with it. We strongly recommend that you read [Variable Scopes](https://learning.postman.com/docs/sending-requests/variables/#variable-scopes) from Postman documentation before moving forward. -By default, the API fuzzer uses the Postman file to resolve Postman variable values. If a JSON file -is set in a GitLab CI/CD variable `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`, then the JSON -file takes precedence to get Postman variable values. +As mentioned above, there are different variable scopes, and each of them has a purpose and can be used to provide more flexibility to your Postman document. There is an important note on how values for variables are computed, as per Postman documentation: -WARNING: -Although Postman can export environment variables into a JSON file, the format is not compatible with the JSON expected by `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`. +> If a variable with the same name is declared in two different scopes, the value stored in the variable with narrowest scope is used. For example, if there is a global variable named `username` and a local variable named `username`, the local value is used when the request runs. + +The following is a summary of the variable scopes supported by the Postman Client and API Fuzzing: + +- **Global Environment (Global) scope** is a special pre-defined environment that is available throughout a workspace. We can also refer to the _global environment_ scope as the _global_ scope. The Postman Client allows exporting the global environment into a JSON file, which can be used with API Fuzzing. +- **Environment scope** is a named group of variables created by a user in the Postman Client. +The Postman Client supports a single active environment along with the global environment. The variables defined in an active user-created environment take precedence over variables defined in the global environment. The Postman Client allows exporting your environment into a JSON file, which can be used with API Fuzzing. +- **Collection scope** is a group of variables declared in a given collection. The collection variables are available to the collection where they have been declared and the nested requests or collections. Variables defined in the collection scope take precedence over the _global environment_ scope and also the _environment_ scope. +The Postman Client can export one or more collections into a JSON file, this JSON file contains selected collections, requests, and collection variables. +- **API Fuzzing Scope** is a new scope added by API Fuzzing to allow users to provide extra variables, or override variables defined in other supported scopes. This scope is not supported by Postman. The _API Fuzzing Scope_ variables are provided using a [custom JSON file format](#api-fuzzing-scope-custom-json-file-format). + - Override values defined in the environment or collection + - Defining variables from scripts + - Define a single row of data from the unsupported _data scope_ +- **Data scope** is a group of variables in which their name and values come from JSON or CSV files. A Postman collection runner like [Newman](https://learning.postman.com/docs/running-collections/using-newman-cli/command-line-integration-with-newman/) or [Postman Collection Runner](https://learning.postman.com/docs/running-collections/intro-to-collection-runs/) executes the requests in a collection as many times as entries have the JSON or CSV file. A good use case for these variables is to automate tests using scripts in Postman. +API Fuzzing does **not** support reading data from a CSV or JSON file. +- **Local scope** are variables that are defined in Postman scripts. API Fuzzing does **not** support Postman scripts and by extension, variables defined in scripts. You can still provide values for the script-defined variables by defining them in one of the supported scopes, or our custom JSON format. + +Not all scopes are supported by API Fuzzing and variables defined in scripts are not supported. The following table is sorted by broadest scope to narrowest scope. + +| Scope |Postman | API Fuzzing | Comment | +| ------------------ |:---------:|:------------:| :--------| +| Global Environment | Yes | Yes | Special pre-defined environment | +| Environment | Yes | Yes | Named environments | +| Collection | Yes | Yes | Defined in your postman collection | +| API Fuzzing Scope | No | Yes | Custom scope added by API Fuzzing | +| Data | Yes | No | External files in CSV or JSON format | +| Local | Yes | No | Variables defined in scripts | + +For more details on how to define variables and export variables in different scopes, see: + +- [Defining collection variables](https://learning.postman.com/docs/sending-requests/variables/#defining-collection-variables) +- [Defining environment variables](https://learning.postman.com/docs/sending-requests/variables/#defining-environment-variables) +- [Defining global variables](https://learning.postman.com/docs/sending-requests/variables/#defining-global-variables) + +##### Exporting from Postman Client + +The Postman Client lets you export different file formats, for instance, you can export a Postman collection or a Postman environment. +The exported environment can be the global environment (which is always available) or can be any custom environment you previously have created. When you export a Postman Collection, it may contain only declarations for _collection_ and _local_ scoped variables; _environment_ scoped variables are not included. + +To get the declaration for _environment_ scoped variables, you have to export a given environment at the time. Each exported file only includes variables from the selected environment. + +For more details on exporting variables in different supported scopes, see: + +- [Exporting collections](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-collections) +- [Exporting environments](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) +- [Downloading global environments](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) + +##### API Fuzzing Scope, custom JSON file format + +Our custom JSON file format is a JSON object where each object property represents a variable name and the property value represents the variable value. This file can be created using your favorite text editor, or it can be produced by an earlier job in your pipeline. + +This example defines two variables `base_url` and `token` in the API Fuzzing scope: + +```json +{ + "base_url": "http://127.0.0.1/", + "token": "Token 84816165151" +} +``` + +##### Using scopes with API Fuzzing + +The scopes: _global_, _environment_, _collection_, and _GitLab API Fuzzing_ are supported in [GitLab 15.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/356312). GitLab 15.0 and earlier, supports only the _collection_, and _GitLab API Fuzzing_ scopes. + +The following table provides a quick reference for mapping scope files/URLs to API Fuzzing configuration variables: + +| Scope | How to Provide | +| ------------------ | --------------- | +| Global Environment | FUZZAPI_POSTMAN_COLLECTION_VARIABLES | +| Environment | FUZZAPI_POSTMAN_COLLECTION_VARIABLES | +| Collection | FUZZAPI_POSTMAN_COLLECTION | +| API Fuzzing Scope | FUZZAPI_POSTMAN_COLLECTION_VARIABLES | +| Data | Not supported | +| Local | Not supported | + +The Postman Collection document automatically includes any _collection_ scoped variables. The Postman Collection is provided with the configuration variable `FUZZAPI_POSTMAN_COLLECTION`. This variable can be set to a single [exported Postman collection](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-collections). + +Variables from other scopes are provided through the `FUZZAPI_POSTMAN_COLLECTION_VARIABLES` configuration variable. The configuration variable supports a comma (`,`) delimited file list in [GitLab 15.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/356312). GitLab 15.0 and earlier, supports only one single file. The order of the files provided is not important as the files provide the needed scope information. + +The configuration variable `FUZZAPI_POSTMAN_COLLECTION_VARIABLES` can be set to: + +- [Exported Global environment](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) +- [Exported environments](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) +- [API Fuzzing Custom JSON format](#api-fuzzing-scope-custom-json-file-format) + +##### Undefined Postman variables + +There is a chance that API Fuzzing engine does not find all variables references that your Postman collection file is using. Some cases can be: + +- You are using _data_ or _local_ scoped variables, and as stated previously these scopes are not supported by API Fuzzing. Thus, assuming the values for these variables have not been provided through [the API Fuzzing scope](#api-fuzzing-scope-custom-json-file-format), then the values of the _data_ and _local_ scoped variables are undefined. +- A variable name was typed incorrectly, and the name does not match the defined variable. +- Postman Client supports a new dynamic variable that is not supported by API Fuzzing. + +When possible, API Fuzzing follows the same behavior as the Postman Client does when dealing with undefined variables. The text of the variable reference remains the same, and there is no text substitution. The same behavior also applies to any unsupported dynamic variables. + +For example, if a request definition in the Postman Collection references the variable `{{full_url}}` and the variable is not found it is left unchanged with the value `{{full_url}}`. + +##### Dynamic Postman variables + +In addition to variables that a user can define at various scope levels, Postman has a set of pre-defined variables called _dynamic_ variables. The [_dynamic_ variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) are already defined and their name is prefixed with a dollar sign (`$`), for instance, `$guid`. _Dynamic_ variables can be used like any other variable, and in the Postman Client, they produce random values during the request/collection run. + +An important difference between API Fuzzing and Postman is that API Fuzzing returns the same value for each usage of the same dynamic variables. This differs from the Postman Client behavior which returns a random value on each use of the same dynamic variable. In other words, API Fuzzing uses static values for dynamic variables while Postman uses random values. + +The supported dynamic variables during the scanning process are: + +| Variable | Value | +| ----------- | ----------- | +| `$guid` | `611c2e81-2ccb-42d8-9ddc-2d0bfa65c1b4` | +| `$isoTimestamp` | `2020-06-09T21:10:36.177Z` | +| `$randomAbbreviation` | `PCI` | +| `$randomAbstractImage` | `http://no-a-valid-host/640/480/abstract` | +| `$randomAdjective` | `auxiliary` | +| `$randomAlphaNumeric` | `a` | +| `$randomAnimalsImage` | `http://no-a-valid-host/640/480/animals` | +| `$randomAvatarImage` | `https://no-a-valid-host/path/to/some/image.jpg` | +| `$randomBankAccount` | `09454073` | +| `$randomBankAccountBic` | `EZIAUGJ1` | +| `$randomBankAccountIban` | `MU20ZPUN3039684000618086155TKZ` | +| `$randomBankAccountName` | `Home Loan Account` | +| `$randomBitcoin` | `3VB8JGT7Y4Z63U68KGGKDXMLLH5` | +| `$randomBoolean` | `true` | +| `$randomBs` | `killer leverage schemas` | +| `$randomBsAdjective` | `viral` | +| `$randomBsBuzz` | `repurpose` | +| `$randomBsNoun` | `markets` | +| `$randomBusinessImage` | `http://no-a-valid-host/640/480/business` | +| `$randomCatchPhrase` | `Future-proofed heuristic open architecture` | +| `$randomCatchPhraseAdjective` | `Business-focused` | +| `$randomCatchPhraseDescriptor` | `bandwidth-monitored` | +| `$randomCatchPhraseNoun` | `superstructure` | +| `$randomCatsImage` | `http://no-a-valid-host/640/480/cats` | +| `$randomCity` | `Spinkahaven` | +| `$randomCityImage` | `http://no-a-valid-host/640/480/city` | +| `$randomColor` | `fuchsia` | +| `$randomCommonFileExt` | `wav` | +| `$randomCommonFileName` | `well_modulated.mpg4` | +| `$randomCommonFileType` | `audio` | +| `$randomCompanyName` | `Grady LLC` | +| `$randomCompanySuffix` | `Inc` | +| `$randomCountry` | `Kazakhstan` | +| `$randomCountryCode` | `MD` | +| `$randomCreditCardMask` | `3622` | +| `$randomCurrencyCode` | `ZMK` | +| `$randomCurrencyName` | `Pound Sterling` | +| `$randomCurrencySymbol` | `£` | +| `$randomDatabaseCollation` | `utf8_general_ci` | +| `$randomDatabaseColumn` | `updatedAt` | +| `$randomDatabaseEngine` | `Memory` | +| `$randomDatabaseType` | `text` | +| `$randomDateFuture` | `Tue Mar 17 2020 13:11:50 GMT+0530 (India Standard Time)` | +| `$randomDatePast` | `Sat Mar 02 2019 09:09:26 GMT+0530 (India Standard Time)` | +| `$randomDateRecent` | `Tue Jul 09 2019 23:12:37 GMT+0530 (India Standard Time)` | +| `$randomDepartment` | `Electronics` | +| `$randomDirectoryPath` | `/usr/local/bin` | +| `$randomDomainName` | `trevor.info` | +| `$randomDomainSuffix` | `org` | +| `$randomDomainWord` | `jaden` | +| `$randomEmail` | `Iva.Kovacek61@no-a-valid-host.com` | +| `$randomExampleEmail` | `non-a-valid-user@example.net` | +| `$randomFashionImage` | `http://no-a-valid-host/640/480/fashion` | +| `$randomFileExt` | `war` | +| `$randomFileName` | `neural_sri_lanka_rupee_gloves.gdoc` | +| `$randomFilePath` | `/home/programming_chicken.cpio` | +| `$randomFileType` | `application` | +| `$randomFirstName` | `Chandler` | +| `$randomFoodImage` | `http://no-a-valid-host/640/480/food` | +| `$randomFullName` | `Connie Runolfsdottir` | +| `$randomHexColor` | `#47594a` | +| `$randomImageDataUri` | `data:image/svg+xml;charset=UTF-8,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20version%3D%221.1%22%20baseProfile%3D%22full%22%20width%3D%22undefined%22%20height%3D%22undefined%22%3E%20%3Crect%20width%3D%22100%25%22%20height%3D%22100%25%22%20fill%3D%22grey%22%2F%3E%20%20%3Ctext%20x%3D%220%22%20y%3D%2220%22%20font-size%3D%2220%22%20text-anchor%3D%22start%22%20fill%3D%22white%22%3Eundefinedxundefined%3C%2Ftext%3E%20%3C%2Fsvg%3E` | +| `$randomImageUrl` | `http://no-a-valid-host/640/480` | +| `$randomIngverb` | `navigating` | +| `$randomInt` | `494` | +| `$randomIP` | `241.102.234.100` | +| `$randomIPV6` | `dbe2:7ae6:119b:c161:1560:6dda:3a9b:90a9` | +| `$randomJobArea` | `Mobility` | +| `$randomJobDescriptor` | `Senior` | +| `$randomJobTitle` | `International Creative Liaison` | +| `$randomJobType` | `Supervisor` | +| `$randomLastName` | `Schneider` | +| `$randomLatitude` | `55.2099` | +| `$randomLocale` | `ny` | +| `$randomLongitude` | `40.6609` | +| `$randomLoremLines` | `Ducimus in ut mollitia.\nA itaque non.\nHarum temporibus nihil voluptas.\nIste in sed et nesciunt in quaerat sed.` | +| `$randomLoremParagraph` | `Ab aliquid odio iste quo voluptas voluptatem dignissimos velit. Recusandae facilis qui commodi ea magnam enim nostrum quia quis. Nihil est suscipit assumenda ut voluptatem sed. Esse ab voluptas odit qui molestiae. Rem est nesciunt est quis ipsam expedita consequuntur.` | +| `$randomLoremParagraphs` | `Voluptatem rem magnam aliquam ab id aut quaerat. Placeat provident possimus voluptatibus dicta velit non aut quasi. Mollitia et aliquam expedita sunt dolores nam consequuntur. Nam dolorum delectus ipsam repudiandae et ipsam ut voluptatum totam. Nobis labore labore recusandae ipsam quo.` | +| `$randomLoremSentence` | `Molestias consequuntur nisi non quod.` | +| `$randomLoremSentences` | `Et sint voluptas similique iure amet perspiciatis vero sequi atque. Ut porro sit et hic. Neque aspernatur vitae fugiat ut dolore et veritatis. Ab iusto ex delectus animi. Voluptates nisi iusto. Impedit quod quae voluptate qui.` | +| `$randomLoremSlug` | `eos-aperiam-accusamus, beatae-id-molestiae, qui-est-repellat` | +| `$randomLoremText` | `Quisquam asperiores exercitationem ut ipsum. Aut eius nesciunt. Et reiciendis aut alias eaque. Nihil amet laboriosam pariatur eligendi. Sunt ullam ut sint natus ducimus. Voluptas harum aspernatur soluta rem nam.` | +| `$randomLoremWord` | `est` | +| `$randomLoremWords` | `vel repellat nobis` | +| `$randomMACAddress` | `33:d4:68:5f:b4:c7` | +| `$randomMimeType` | `audio/vnd.vmx.cvsd` | +| `$randomMonth` | `February` | +| `$randomNamePrefix` | `Dr.` | +| `$randomNameSuffix` | `MD` | +| `$randomNatureImage` | `http://no-a-valid-host/640/480/nature` | +| `$randomNightlifeImage` | `http://no-a-valid-host/640/480/nightlife` | +| `$randomNoun` | `bus` | +| `$randomPassword` | `t9iXe7COoDKv8k3` | +| `$randomPeopleImage` | `http://no-a-valid-host/640/480/people` | +| `$randomPhoneNumber` | `700-008-5275` | +| `$randomPhoneNumberExt` | `27-199-983-3864` | +| `$randomPhrase` | `You can't program the monitor without navigating the mobile XML program!` | +| `$randomPrice` | `531.55` | +| `$randomProduct` | `Pizza` | +| `$randomProductAdjective` | `Unbranded` | +| `$randomProductMaterial` | `Steel` | +| `$randomProductName` | `Handmade Concrete Tuna` | +| `$randomProtocol` | `https` | +| `$randomSemver` | `7.0.5` | +| `$randomSportsImage` | `http://no-a-valid-host/640/480/sports` | +| `$randomStreetAddress` | `5742 Harvey Streets` | +| `$randomStreetName` | `Kuhic Island` | +| `$randomTransactionType` | `payment` | +| `$randomTransportImage` | `http://no-a-valid-host/640/480/transport` | +| `$randomUrl` | `https://no-a-valid-host.net` | +| `$randomUserAgent` | `Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.9.8; rv:15.6) Gecko/20100101 Firefox/15.6.6` | +| `$randomUserName` | `Jarrell.Gutkowski` | +| `$randomUUID` | `6929bb52-3ab2-448a-9796-d6480ecad36b` | +| `$randomVerb` | `navigate` | +| `$randomWeekday` | `Thursday` | +| `$randomWord` | `withdrawal` | +| `$randomWords` | `Samoa Synergistic sticky copying Grocery` | +| `$timestamp` | `1562757107` | + +##### Example: Global Scope + +In this example, [the _global_ scope is exported](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) from the Postman Client as `global-scope.json` and provided to API Fuzzing through the `FUZZAPI_POSTMAN_COLLECTION_VARIABLES` configuration variable. Here is an example of using `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`: @@ -384,21 +728,171 @@ include: variables: FUZZAPI_PROFILE: Quick-10 - FUZZAPI_POSTMAN_COLLECTION: postman-collection_serviceA.json + FUZZAPI_POSTMAN_COLLECTION: postman-collection.json + FUZZAPI_POSTMAN_COLLECTION_VARIABLES: global-scope.json + FUZZAPI_TARGET_URL: http://test-deployment/ +``` + +##### Example: Environment Scope + +In this example, [the _environment_ scope is exported](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) from the Postman Client as `environment-scope.json` and provided to API Fuzzing through the `FUZZAPI_POSTMAN_COLLECTION_VARIABLES` configuration variable. + +Here is an example of using `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`: + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_POSTMAN_COLLECTION: postman-collection.json + FUZZAPI_POSTMAN_COLLECTION_VARIABLES: environment-scope.json + FUZZAPI_TARGET_URL: http://test-deployment/ +``` + +##### Example: Collection Scope + +The _collection_ scope variables are included in the exported Postman Collection file and provided through the `FUZZAPI_POSTMAN_COLLECTION` configuration variable. + +Here is an example of using `FUZZAPI_POSTMAN_COLLECTION`: + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_POSTMAN_COLLECTION: postman-collection.json FUZZAPI_TARGET_URL: http://test-deployment/ FUZZAPI_POSTMAN_COLLECTION_VARIABLES: variable-collection-dictionary.json ``` -The file `variable-collection-dictionary.json` is a JSON document. This JSON is an object with -key-value pairs for properties. The keys are the variables' names, and the values are the variables' +##### Example: API Fuzzing Scope + +The API Fuzzing Scope is used for two main purposes, defining _data_ and _local_ scope variables that are not supported by API Fuzzing, and changing the value of an existing variable defined in another scope. The API Fuzzing Scope is provided through the `FUZZAPI_POSTMAN_COLLECTION_VARIABLES` configuration variable. + +Here is an example of using `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`: + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_POSTMAN_COLLECTION: postman-collection.json + FUZZAPI_POSTMAN_COLLECTION_VARIABLES: api-fuzzing-scope.json + FUZZAPI_TARGET_URL: http://test-deployment/ +``` + +The file `api-fuzzing-scope.json` uses our [custom JSON file format](#api-fuzzing-scope-custom-json-file-format). This JSON is an object with key-value pairs for properties. The keys are the variables' names, and the values are the variables' values. For example: - ```json - { - "base_url": "http://127.0.0.1/", - "token": "Token 84816165151" - } - ``` +```json +{ + "base_url": "http://127.0.0.1/", + "token": "Token 84816165151" +} +``` + +##### Example: Multiple Scopes + +In this example, a _global_ scope, _environment_ scope, and _collection_ scope are configured. The first step is to export our various scopes. + +- [Export the _global_ scope](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) as `global-scope.json` +- [Export the _environment_ scope](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) as `environment-scope.json` +- Export the Postman Collection which includes the _collection_ scope as `postman-collection.json` + +The Postman Collection is provided using the `FUZZAPI_POSTMAN_COLLECTION` variable, while the other scopes are provided using the `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`. API Fuzzing can identify which scope the provided files match using data provided in each file. + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_POSTMAN_COLLECTION: postman-collection.json + FUZZAPI_POSTMAN_COLLECTION_VARIABLES: global-scope.json,environment-scope.json + FUZZAPI_TARGET_URL: http://test-deployment/ +``` + +##### Example: Changing a Variables Value + +When using exported scopes, it's often the case that the value of a variable must be changed for use with API Fuzzing. For example, a _collection_ scoped variable might contain a variable named `api_version` with a value of `v2`, while your test needs a value of `v1`. Instead of modifying the exported collection to change the value, the API Fuzzing scope can be used to change its value. This works because the _API Fuzzing_ scope takes precedence over all other scopes. + +The _collection_ scope variables are included in the exported Postman Collection file and provided through the `FUZZAPI_POSTMAN_COLLECTION` configuration variable. + +The API Fuzzing Scope is provided through the `FUZZAPI_POSTMAN_COLLECTION_VARIABLES` configuration variable, but first, we must create the file. +The file `api-fuzzing-scope.json` uses our [custom JSON file format](#api-fuzzing-scope-custom-json-file-format). This JSON is an object with key-value pairs for properties. The keys are the variables' names, and the values are the variables' +values. For example: + +```json +{ + "api_version": "v1" +} +``` + +Our CI definition: + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_POSTMAN_COLLECTION: postman-collection.json + FUZZAPI_POSTMAN_COLLECTION_VARIABLES: api-fuzzing-scope.json + FUZZAPI_TARGET_URL: http://test-deployment/ +``` + +##### Example: Changing a Variables Value with Multiple Scopes + +When using exported scopes, it's often the case that the value of a variable must be changed for use with API Fuzzing. For example, an _environment_ scope might contain a variable named `api_version` with a value of `v2`, while your test needs a value of `v1`. Instead of modifying the exported file to change the value, the API Fuzzing scope can be used. This works because the _API Fuzzing_ scope takes precedence over all other scopes. + +In this example, a _global_ scope, _environment_ scope, _collection_ scope, and _API Fuzzing_ scope are configured. The first step is to export and create our various scopes. + +- [Export the _global_ scope](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) as `global-scope.json` +- [Export the _environment_ scope](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) as `environment-scope.json` +- Export the Postman Collection which includes the _collection_ scope as `postman-collection.json` + +The API Fuzzing scope is used by creating a file `api-fuzzing-scope.json` using our [custom JSON file format](#api-fuzzing-scope-custom-json-file-format). This JSON is an object with key-value pairs for properties. The keys are the variables' names, and the values are the variables' +values. For example: + +```json +{ + "api_version": "v1" +} +``` + +The Postman Collection is provided using the `FUZZAPI_POSTMAN_COLLECTION` variable, while the other scopes are provided using the `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`. API Fuzzing can identify which scope the provided files match using data provided in each file. + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_POSTMAN_COLLECTION: postman-collection.json + FUZZAPI_POSTMAN_COLLECTION_VARIABLES: global-scope.json,environment-scope.json,api-fuzzing-scope.json + FUZZAPI_TARGET_URL: http://test-deployment/ +``` ## API fuzzing configuration @@ -420,21 +914,23 @@ provide a script that performs an authentication flow or calculates the token. #### HTTP Basic Authentication [HTTP basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) -is an authentication method built in to the HTTP protocol and used in conjunction with +is an authentication method built into the HTTP protocol and used in conjunction with [transport layer security (TLS)](https://en.wikipedia.org/wiki/Transport_Layer_Security). -To use HTTP basic authentication, two CI/CD variables are added to your `.gitlab-ci.yml` file: -- `FUZZAPI_HTTP_USERNAME`: The username for authentication. -- `FUZZAPI_HTTP_PASSWORD`: The password for authentication. +We recommended that you [create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables) +for the password (for example, `TEST_API_PASSWORD`), and set it to be masked. You can create CI/CD +variables from the GitLab project's page at **Settings > CI/CD**, in the **Variables** section. +Because of the [limitations on masked variables](../../../ci/variables/index.md#mask-a-cicd-variable), +you should Base64-encode the password before adding it as a variable. -For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables) -(for example, `TEST_API_PASSWORD`) set to the password. You can create CI/CD variables from the -GitLab projects page at **Settings > CI/CD**, in the **Variables** section. Use that variable -as the value for `FUZZAPI_HTTP_PASSWORD`: +Finally, add two CI/CD variables to your `.gitlab-ci.yml` file: + +- `FUZZAPI_HTTP_USERNAME`: The username for authentication. +- `FUZZAPI_HTTP_PASSWORD_BASE64`: The Base64-encoded password for authentication. ```yaml stages: - - fuzz + - fuzz include: - template: API-Fuzzing.gitlab-ci.yml @@ -444,9 +940,13 @@ variables: FUZZAPI_HAR: test-api-recording.har FUZZAPI_TARGET_URL: http://test-deployment/ FUZZAPI_HTTP_USERNAME: testuser - FUZZAPI_HTTP_PASSWORD: $TEST_API_PASSWORD + FUZZAPI_HTTP_PASSWORD_BASE64: $TEST_API_PASSWORD ``` +#### Raw password + +If you do not want to Base64-encode the password (or if you are using GitLab 15.3 or earlier) you can provide the raw password `FUZZAPI_HTTP_PASSWORD`, instead of using `FUZZAPI_HTTP_PASSWORD_BASE64`. + #### Bearer Tokens Bearer tokens are used by several different authentication mechanisms, including OAuth2 and JSON Web @@ -493,7 +993,7 @@ Follow these steps to provide the bearer token with `FUZZAPI_OVERRIDES_ENV`: ##### Token generated at test runtime If the bearer token must be generated and doesn't expire during testing, you can provide to API -fuzzing a file containing the token. A prior stage and job, or part of the API fuzzing job, can +fuzzing with a file containing the token. A prior stage and job, or part of the API fuzzing job, can generate this file. API fuzzing expects to receive a JSON file with the following structure: @@ -605,7 +1105,10 @@ profile increases as the number of tests increases. |[`FUZZAPI_OPENAPI_ALL_MEDIA_TYPES`](#openapi-specification) | Use all supported media types instead of one when generating requests. Causes test duration to be longer. Default is disabled. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333304) in GitLab 14.10. | |[`FUZZAPI_OPENAPI_MEDIA_TYPES`](#openapi-specification) | Colon (`:`) separated media types accepted for testing. Default is disabled. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333304) in GitLab 14.10. | |[`FUZZAPI_HAR`](#http-archive-har) | HTTP Archive (HAR) file. | +|[`FUZZAPI_GRAPHQL`](#graphql-schema) | Path to GraphQL endpoint, for example `/api/graphql`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352780) in GitLab 15.4. | +|[`FUZZAPI_GRAPHQL_SCHEMA`](#graphql-schema) | A URL or filename for a GraphQL schema in JSON format. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352780) in GitLab 15.4. | |[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. | +|[`FUZZAPI_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract Postman variable values. The support for comma-separated (`,`) files was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356312) in GitLab 15.1. | |[`FUZZAPI_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract Postman variable values. | |[`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | |[`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | @@ -616,6 +1119,7 @@ profile increases as the number of tests increases. |[`FUZZAPI_OVERRIDES_INTERVAL`](#overrides) | How often to run overrides command in seconds. Defaults to `0` (once). | |[`FUZZAPI_HTTP_USERNAME`](#http-basic-authentication) | Username for HTTP authentication. | |[`FUZZAPI_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. | +|[`FUZZAPI_HTTP_PASSWORD_BASE64`](#http-basic-authentication) | Password for HTTP authentication, Base64-encoded. [Introduced](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing-src/-/merge_requests/702) in GitLab 15.4. | ### Overrides @@ -1062,21 +1566,21 @@ This example excludes the `/auth` resource. This does not exclude child resource ```yaml variables: - FUZZAPI_EXCLUDE_PATHS=/auth + FUZZAPI_EXCLUDE_PATHS: /auth ``` To exclude `/auth`, and child resources (`/auth/child`), we use a wildcard. ```yaml variables: - FUZZAPI_EXCLUDE_PATHS=/auth* + FUZZAPI_EXCLUDE_PATHS: /auth* ``` To exclude multiple paths we can use the `;` character. In this example we exclude `/auth*` and `/v1/*`. ```yaml variables: - FUZZAPI_EXCLUDE_PATHS=/auth*;/v1/* + FUZZAPI_EXCLUDE_PATHS: /auth*;/v1/* ``` ### Exclude parameters @@ -1336,7 +1840,15 @@ Each value in `FUZZAPI_EXCLUDE_URLS` is a regular expression. Characters such as The following example excludes the URL `http://target/api/auth` and its child resources. ```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + variables: + FUZZAPI_TARGET_URL: http://target/ + FUZZAPI_OPENAPI: test-api-specification.json FUZZAPI_EXCLUDE_URLS: http://target/api/auth ``` @@ -1345,7 +1857,15 @@ variables: To exclude the URLs `http://target/api/buy` and `http://target/api/sell` but allowing to scan their child resources, for instance: `http://target/api/buy/toy` or `http://target/api/sell/chair`. You could use the value `http://target/api/buy/$,http://target/api/sell/$`. This value is using two regular expressions, each of them separated by a `,` character. Hence, it contains `http://target/api/buy$` and `http://target/api/sell$`. In each regular expression, the trailing `$` character points out where the matching URL should end. ```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + variables: + FUZZAPI_TARGET_URL: http://target/ + FUZZAPI_OPENAPI: test-api-specification.json FUZZAPI_EXCLUDE_URLS: http://target/api/buy/$,http://target/api/sell/$ ``` @@ -1354,7 +1874,15 @@ variables: In order to exclude the URLs: `http://target/api/buy` and `http://target/api/sell`, and their child resources. To provide multiple URLs we use the `,` character as follows: ```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + variables: + FUZZAPI_TARGET_URL: http://target/ + FUZZAPI_OPENAPI: test-api-specification.json FUZZAPI_EXCLUDE_URLS: http://target/api/buy,http://target/api/sell ``` @@ -1363,7 +1891,15 @@ variables: In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. ```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + variables: + FUZZAPI_TARGET_URL: http://target/ + FUZZAPI_OPENAPI: test-api-specification.json FUZZAPI_EXCLUDE_URLS: https://target/api/v.*/user/create$ ``` @@ -1679,11 +2215,11 @@ For more information, see [Offline environments](../offline_deployments/index.md ### `Error waiting for API Security 'http://127.0.0.1:5000' to become available` -A bug exists in versions of the API Fuzzing analyzer prior to v1.6.196 that can cause a background process to fail under certain conditions. The solution is to update to a newer version of the DAST API analyzer. +A bug exists in versions of the API Fuzzing analyzer prior to v1.6.196 that can cause a background process to fail under certain conditions. The solution is to update to a newer version of the API Fuzzing analyzer. The version information can be found in the job details for the `apifuzzer_fuzz` job. -If the issue is occurring with versions v1.6.196 or greater, please contact Support and provide the following information: +If the issue is occurring with versions v1.6.196 or greater, contact Support and provide the following information: 1. Reference this troubleshooting section and ask for the issue to be escalated to the Dynamic Analysis Team. 1. The full console output of the job. @@ -1750,12 +2286,15 @@ This solution is for pipelines in which the target API URL doesn't change (is st For environments where the target API remains the same, we recommend you specify the target URL by using the `FUZZAPI_TARGET_URL` environment variable. In your `.gitlab-ci.yml` file, add a variable `FUZZAPI_TARGET_URL`. The variable must be set to the base URL of API testing target. For example: ```yaml +stages: + - fuzz + include: - - template: API-Fuzzing.gitlab-ci.yml + - template: API-Fuzzing.gitlab-ci.yml - variables: - FUZZAPI_TARGET_URL: http://test-deployment/ - FUZZAPI_OPENAPI: test-api-specification.json +variables: + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OPENAPI: test-api-specification.json ``` #### Dynamic environment solutions @@ -1793,7 +2332,7 @@ TODO ### Use OpenAPI with an invalid schema -There are cases where the document is autogenerated with an invalid schema or cannot be edited manually in a timely manner. In those scenarios, the API Security is able to perform a relaxed validation by setting the variable `FUZZAPI_OPENAPI_RELAXED_VALIDATION`. We recommend providing a fully compliant OpenAPI document to prevent unexpected behaviors. +There are cases where the document is autogenerated with an invalid schema or cannot be edited manually in a timely manner. In those scenarios, the API Fuzzing is able to perform a relaxed validation by setting the variable `FUZZAPI_OPENAPI_RELAXED_VALIDATION`. We recommend providing a fully compliant OpenAPI document to prevent unexpected behaviors. #### Edit a non-compliant OpenAPI file @@ -1810,25 +2349,25 @@ If your OpenAPI document is generated manually, load your document in the editor Relaxed validation is meant for cases when the OpenAPI document cannot meet OpenAPI specifications, but it still has enough content to be consumed by different tools. A validation is performed but less strictly in regards to document schema. -API Security can still try to consume an OpenAPI document that does not fully comply with OpenAPI specifications. To instruct API Security to perform a relaxed validation, set the variable `FUZZAPI_OPENAPI_RELAXED_VALIDATION` to any value, for example: +API Fuzzing can still try to consume an OpenAPI document that does not fully comply with OpenAPI specifications. To instruct API Fuzzing analyzer to perform a relaxed validation, set the variable `FUZZAPI_OPENAPI_RELAXED_VALIDATION` to any value, for example: ```yaml - stages: - - fuzz +stages: + - fuzz - include: - - template: API-Fuzzing.gitlab-ci.yml +include: + - template: API-Fuzzing.gitlab-ci.yml - variables: - FUZZAPI_PROFILE: Quick-10 - FUZZAPI_TARGET_URL: http://test-deployment/ - FUZZAPI_OPENAPI: test-api-specification.json - FUZZAPI_OPENAPI_RELAXED_VALIDATION: On +variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_OPENAPI_RELAXED_VALIDATION: 'On' ``` ### `No operation in the OpenAPI document is consuming any supported media type` -API Security uses the specified media types in the OpenAPI document to generate requests. If no request can be created due to the lack of supported media types, then an error will be thrown. +API Fuzzing uses the specified media types in the OpenAPI document to generate requests. If no request can be created due to the lack of supported media types, then an error will be thrown. **Error message** diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 9ca1a6f125f..32a523a1871 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -29,7 +29,7 @@ all security features are configured by default. To view a project's security configuration: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. Select **Configuration history** to see the `.gitlab-ci.yml` file's history. @@ -56,7 +56,7 @@ You can configure the following security controls: - Can be configured by adding a configuration block to your agent configuration. For more details, read [Operational Container Scanning](../../clusters/agent/vulnerabilities.md#enable-operational-container-scanning). - [Secret Detection](../secret_detection/index.md) - Select **Configure with a merge request** to create a merge request with the changes required to - enable Secret Detection. For more details, read [Enable Secret Detection via an automatic merge request](../secret_detection/index.md#enable-secret-detection-via-an-automatic-merge-request). + enable Secret Detection. For more details, read [Enable Secret Detection using a merge request](../secret_detection/index.md#enable-secret-detection-using-a-merge-request). - [API Fuzzing](../api_fuzzing/index.md) - Select **Enable API Fuzzing** to use API Fuzzing for the current project. For more details, read [API Fuzzing](../../../user/application_security/api_fuzzing/index.md#enable-web-api-fuzzing). - [Coverage Fuzzing](../coverage_fuzzing/index.md) diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 7bb3cb4f64c..961ccf6b563 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -1,7 +1,7 @@ --- type: reference, howto -stage: Protect -group: Container Security +stage: Secure +group: Composition Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -14,6 +14,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Integration with Grype as an alternative scanner [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326279) in GitLab 14.0. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86092) the major analyzer version from `4` to `5` in GitLab 15.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86783) from GitLab Ultimate to GitLab Free in 15.0. +> - Container Scanning variables that reference Docker [renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/357264) in GitLab 15.4. Your application's Docker image may itself be based on Docker images that contain known vulnerabilities. By including an extra Container Scanning job in your pipeline that scans for those @@ -68,7 +69,7 @@ information directly in the merge request. | [Solutions for vulnerabilities (auto-remediation)](#solutions-for-vulnerabilities-auto-remediation) | No | Yes | | Support for the [vulnerability allow list](#vulnerability-allowlisting) | No | Yes | | [Access to Security Dashboard page](#security-dashboard) | No | Yes | -| [Access to Dependency List page](../dependency_list/) | No | Yes | +| [Access to Dependency List page](../dependency_list/index.md) | No | Yes | ## Requirements @@ -83,7 +84,7 @@ To enable container scanning in your pipeline, you need the following: - [Build and push](../../packages/container_registry/index.md#build-and-push-by-using-gitlab-cicd) the Docker image to your project's container registry. - If you're using a third-party container registry, you might need to provide authentication - credentials through the `DOCKER_USER` and `DOCKER_PASSWORD` [configuration variables](#available-cicd-variables). + credentials through the `CS_REGISTRY_USER` and `CS_REGISTRY_PASSWORD` [configuration variables](#available-cicd-variables). For more details on how to use these variables, see [authenticate to a remote registry](#authenticate-to-a-remote-registry). ## Configuration @@ -157,13 +158,13 @@ include: container_scanning: variables: - DOCKER_IMAGE: example.com/user/image:tag + CS_IMAGE: example.com/user/image:tag ``` ##### Authenticate to a remote registry -Scanning an image in a private registry requires authentication. Provide the username in the `DOCKER_USER` -variable, and the password in the `DOCKER_PASSWORD` configuration variable. +Scanning an image in a private registry requires authentication. Provide the username in the `CS_REGISTRY_USER` +variable, and the password in the `CS_REGISTRY_PASSWORD` configuration variable. For example, to scan an image from AWS Elastic Container Registry: @@ -178,9 +179,9 @@ container_scanning: include: - template: Security/Container-Scanning.gitlab-ci.yml - DOCKER_IMAGE: <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<image>:<tag> - DOCKER_USER: AWS - DOCKER_PASSWORD: "$AWS_ECR_PASSWORD" + CS_IMAGE: <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<image>:<tag> + CS_REGISTRY_USER: AWS + CS_REGISTRY_PASSWORD: "$AWS_ECR_PASSWORD" ``` Authenticating to a remote registry is not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. @@ -190,7 +191,7 @@ Authenticating to a remote registry is not supported when [FIPS mode](../../../d > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. The `CS_DISABLE_DEPENDENCY_LIST` CI/CD variable controls whether the scan creates a -[Dependency List](../dependency_list/) +[Dependency List](../dependency_list/index.md) report. This variable is currently only supported when the `trivy` analyzer is used. The variable's default setting of `"false"` causes the scan to create the report. To disable the report, set the variable to `"true"`: @@ -230,10 +231,10 @@ container_scanning: ``` When you enable this feature, you may see [duplicate findings](../terminology/index.md#duplicate-finding) -in the [Vulnerability Report](../vulnerability_report/) -if [Dependency Scanning](../dependency_scanning/) +in the [Vulnerability Report](../vulnerability_report/index.md) +if [Dependency Scanning](../dependency_scanning/index.md) is enabled for your project. This happens because GitLab can't automatically deduplicate findings -across different types of scanning tools. Please reference [this comparison](../dependency_scanning/#dependency-scanning-compared-to-container-scanning) +across different types of scanning tools. Please reference [this comparison](../dependency_scanning/index.md#dependency-scanning-compared-to-container-scanning) between GitLab Dependency Scanning and Container Scanning for more details on which types of dependencies are likely to be duplicated. #### Available CI/CD variables @@ -251,7 +252,7 @@ including a large number of false positives. | `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | All | | `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | All | | `CS_ANALYZER_IMAGE` | `registry.gitlab.com/security-products/container-scanning:5` | Docker image of the analyzer. | All | -| `CS_DEFAULT_BRANCH_IMAGE` | `""` | The name of the `DOCKER_IMAGE` on the default branch. See [Setting the default branch image](#setting-the-default-branch-image) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338877) in GitLab 14.5. | All | +| `CS_DEFAULT_BRANCH_IMAGE` | `""` | The name of the `CS_IMAGE` on the default branch. See [Setting the default branch image](#setting-the-default-branch-image) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338877) in GitLab 14.5. | All | | `CS_DISABLE_DEPENDENCY_LIST` | `"false"` | Disable Dependency Scanning for packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | | `CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` | `"true"` | Disable scanning for language-specific packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | | `CS_DOCKER_INSECURE` | `"false"` | Allow access to secure Docker registries using HTTPS without validating the certificates. | All | @@ -259,10 +260,14 @@ including a large number of false positives. | `CS_IGNORE_UNFIXED` | `"false"` | Ignore vulnerabilities that are not fixed. | All | | `CS_REGISTRY_INSECURE` | `"false"` | Allow access to insecure registries (HTTP only). Should only be set to `true` when testing the image locally. Works with all scanners, but the registry must listen on port `80/tcp` for Trivy to work. | All | | `CS_SEVERITY_THRESHOLD` | `UNKNOWN` | Severity level threshold. The scanner outputs vulnerabilities with severity level higher than or equal to this threshold. Supported levels are Unknown, Low, Medium, High, and Critical. | Trivy | -| `DOCKER_IMAGE` | `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` | The Docker image to be scanned. If set, this variable overrides the `$CI_APPLICATION_REPOSITORY` and `$CI_APPLICATION_TAG` variables. | All | -| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | Password for accessing a Docker registry requiring authentication. The default is only set if `$DOCKER_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | -| `DOCKER_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. The default is only set if `$DOCKER_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | -| `DOCKERFILE_PATH` | `Dockerfile` | The path to the `Dockerfile` to use for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project. You should configure this variable only if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | All | +| <!-- start_remove The following content will be removed on remove_date: '2023-08-22' --> `DOCKER_IMAGE` | `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_IMAGE`. The Docker image to be scanned. If set, this variable overrides the `$CI_APPLICATION_REPOSITORY` and `$CI_APPLICATION_TAG` variables. | All | +| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_REGISTRY_PASSWORD`. Password for accessing a Docker registry requiring authentication. The default is only set if `$DOCKER_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | +| `DOCKER_USER` | `$CI_REGISTRY_USER` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_REGISTRY_USER`. Username for accessing a Docker registry requiring authentication. The default is only set if `$DOCKER_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | +| `DOCKERFILE_PATH` | `Dockerfile` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_DOCKERFILE_PATH`. The path to the `Dockerfile` to use for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project. You should configure this variable only if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | All <!-- end_remove --> | +| `CS_IMAGE` | `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` | The Docker image to be scanned. If set, this variable overrides the `$CI_APPLICATION_REPOSITORY` and `$CI_APPLICATION_TAG` variables. | All | +| `CS_REGISTRY_PASSWORD` | `$CI_REGISTRY_PASSWORD` | Password for accessing a Docker registry requiring authentication. The default is only set if `$CS_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | +| `CS_REGISTRY_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. The default is only set if `$CS_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | +| `CS_DOCKERFILE_PATH` | `Dockerfile` | The path to the `Dockerfile` to use for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project. You should configure this variable only if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | All | | `SECURE_LOG_LEVEL` | `info` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. | All | ### Supported distributions @@ -309,7 +314,7 @@ Starting with GitLab 14.10, `-fips` is automatically added to `CS_ANALYZER_IMAGE enabled in the GitLab instance. Container scanning of images in authenticated registries is not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) -is enabled. When `CI_GITLAB_FIPS_MODE` is `"true"`, and `DOCKER_USER` or `DOCKER_PASSWORD` is set, +is enabled. When `CI_GITLAB_FIPS_MODE` is `"true"`, and `CS_REGISTRY_USER` or `CS_REGISTRY_PASSWORD` is set, the analyzer exits with an error and does not perform the scan. ### Enable Container Scanning through an automatic merge request @@ -426,14 +431,14 @@ container_scanning: variables: CS_DEFAULT_BRANCH_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA before_script: - - export DOCKER_IMAGE="$CI_REGISTRY_IMAGE/$CI_COMMIT_BRANCH:$CI_COMMIT_SHA" + - export CS_IMAGE="$CI_REGISTRY_IMAGE/$CI_COMMIT_BRANCH:$CI_COMMIT_SHA" - | if [ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]; then - export DOCKER_IMAGE="$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA" + export CS_IMAGE="$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA" fi ``` -`CS_DEFAULT_BRANCH_IMAGE` should remain the same for a given `DOCKER_IMAGE`. If it changes, then a +`CS_DEFAULT_BRANCH_IMAGE` should remain the same for a given `CS_IMAGE`. If it changes, then a duplicate set of vulnerabilities are created, which must be manually dismissed. When using [Auto DevOps](../../../topics/autodevops/index.md), `CS_DEFAULT_BRANCH_IMAGE` is @@ -500,7 +505,7 @@ This example excludes from `gl-container-scanning-report.json`: - `generalallowlist` block allows you to specify CVE IDs globally. All vulnerabilities with matching CVE IDs are excluded from the scan report. -- `images` block allows you to specify CVE IDs for each container image independently. All vulnerabilities from the given image with matching CVE IDs are excluded from the scan report. The image name is retrieved from one of the environment variables used to specify the Docker image to be scanned, such as `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` or `DOCKER_IMAGE`. The image provided in this block **must** match this value and **must not** include the tag value. For example, if you specify the image to be scanned using `DOCKER_IMAGE=alpine:3.7`, then you would use `alpine` in the `images` block, but you cannot use `alpine:3.7`. +- `images` block allows you to specify CVE IDs for each container image independently. All vulnerabilities from the given image with matching CVE IDs are excluded from the scan report. The image name is retrieved from one of the environment variables used to specify the Docker image to be scanned, such as `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` or `CS_IMAGE`. The image provided in this block **must** match this value and **must not** include the tag value. For example, if you specify the image to be scanned using `CS_IMAGE=alpine:3.7`, then you would use `alpine` in the `images` block, but you cannot use `alpine:3.7`. You can specify container image in multiple ways: @@ -649,8 +654,8 @@ you're using a non-GitLab Docker registry, you must change the `$CI_REGISTRY` va To scan an image in an external private registry, you must configure access credentials so the container scanning analyzer can authenticate itself before attempting to access the image to scan. -If you use the GitLab [Container Registry](../../packages/container_registry/), -the `DOCKER_USER` and `DOCKER_PASSWORD` [configuration variables](#available-cicd-variables) +If you use the GitLab [Container Registry](../../packages/container_registry/index.md), +the `CS_REGISTRY_USER` and `CS_REGISTRY_PASSWORD` [configuration variables](#available-cicd-variables) are set automatically and you can skip this configuration. This example shows the configuration needed to scan images in a private [Google Container Registry](https://cloud.google.com/container-registry/): @@ -661,12 +666,12 @@ include: container_scanning: variables: - DOCKER_USER: _json_key - DOCKER_PASSWORD: "$GCP_CREDENTIALS" - DOCKER_IMAGE: "gcr.io/path-to-you-registry/image:tag" + CS_REGISTRY_USER: _json_key + CS_REGISTRY_PASSWORD: "$GCP_CREDENTIALS" + CS_IMAGE: "gcr.io/path-to-you-registry/image:tag" ``` -Before you commit this configuration, [add a CI/CD variable](../../../ci/variables/#add-a-cicd-variable-to-a-project) +Before you commit this configuration, [add a CI/CD variable](../../../ci/variables/index.md#add-a-cicd-variable-to-a-project) for `GCP_CREDENTIALS` containing the JSON key, as described in the [Google Cloud Platform Container Registry documentation](https://cloud.google.com/container-registry/docs/advanced-authentication#json-key). Also: @@ -706,12 +711,12 @@ The results are stored in `gl-container-scanning-report.json`. ## Reports JSON format The container scanning tool emits JSON reports which the [GitLab Runner](https://docs.gitlab.com/runner/) -recognizes through the [`artifacts:reports`](../../../ci/yaml/#artifactsreports) +recognizes through the [`artifacts:reports`](../../../ci/yaml/index.md#artifactsreports) keyword in the CI configuration file. Once the CI job finishes, the Runner uploads these reports to GitLab, which are then available in the CI Job artifacts. In GitLab Ultimate, these reports can be viewed in the corresponding [pipeline](../vulnerability_report/pipeline.md) -and become part of the [Vulnerability Report](../vulnerability_report/). +and become part of the [Vulnerability Report](../vulnerability_report/index.md). These reports must follow a format defined in the [security report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/). See: @@ -772,7 +777,7 @@ Some vulnerabilities can be fixed by applying the solution that GitLab automatically generates. To enable remediation support, the scanning tool _must_ have access to the `Dockerfile` specified by -the [`DOCKERFILE_PATH`](#available-cicd-variables) CI/CD variable. To ensure that the scanning tool +the [`CS_DOCKERFILE_PATH`](#available-cicd-variables) CI/CD variable. To ensure that the scanning tool has access to this file, it's necessary to set [`GIT_STRATEGY: fetch`](../../../ci/runners/configure_runners.md#git-strategy) in your `.gitlab-ci.yml` file by following the instructions described in this document's diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index 154884c16e7..bec242a0426 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -53,7 +53,7 @@ You can use the following fuzzing engines to test the specified languages. To confirm the status of coverage-guided fuzz testing: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **Coverage Fuzzing** section the status is: - **Not configured** @@ -174,7 +174,7 @@ artifacts files you can download from the CI/CD pipeline. To view details of the corpus registry: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **Coverage Fuzzing** section, select **Manage corpus**. @@ -202,7 +202,7 @@ provided by the `COVFUZZ_CORPUS_NAME` variable. The corpus is updated on every p To upload an existing corpus file: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **Coverage Fuzzing** section, select **Manage corpus**. 1. Select **New corpus**. @@ -277,7 +277,7 @@ For a complete example, read the [Go coverage-guided fuzzing example](https://gi It's also possible to run the coverage-guided fuzzing jobs longer and without blocking your main pipeline. This configuration uses the GitLab -[parent-child pipelines](../../../ci/pipelines/parent_child_pipelines.md). +[parent-child pipelines](../../../ci/pipelines/downstream_pipelines.md#parent-child-pipelines). The suggested workflow in this scenario is to have long-running, asynchronous fuzzing jobs on the main or development branch, and short synchronous fuzzing jobs on all other branches and MRs. This @@ -376,4 +376,4 @@ corpus file extracts into a folder named `corpus`. If you see this error message when running the fuzzing job with `COVFUZZ_USE_REGISTRY` set to `true`, ensure that duplicates are allowed. For more details, see -[duplicate Generic packages](../../packages/generic_packages/#do-not-allow-duplicate-generic-packages). +[duplicate Generic packages](../../packages/generic_packages/index.md#do-not-allow-duplicate-generic-packages). diff --git a/doc/user/application_security/cve_id_request.md b/doc/user/application_security/cve_id_request.md index 5ffd47527c5..6f076bbe3f9 100644 --- a/doc/user/application_security/cve_id_request.md +++ b/doc/user/application_security/cve_id_request.md @@ -1,6 +1,6 @@ --- type: tutorial -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/user/application_security/dast/checks/16.7.md b/doc/user/application_security/dast/checks/16.7.md index 2e6607575db..a052149ee4d 100644 --- a/doc/user/application_security/dast/checks/16.7.md +++ b/doc/user/application_security/dast/checks/16.7.md @@ -25,7 +25,7 @@ Only three directives are applicable for the `Strict-Transport-Security` header. Note that invalid directives, or the `Strict-Transport-Security` header appearing more than once (if the values are different) is considered invalid. -Prior to adding to this security configuration to your website, it is recommended you review the hstspreload.org +Prior to adding to this security configuration to your website, it is recommended you review the hstspreload.org [Deployment Recommendations](https://hstspreload.org/#deployment-recommendations). ## Details diff --git a/doc/user/application_security/dast/checks/209.1.md b/doc/user/application_security/dast/checks/209.1.md index 2e4163bdec0..f2713a70afd 100644 --- a/doc/user/application_security/dast/checks/209.1.md +++ b/doc/user/application_security/dast/checks/209.1.md @@ -9,17 +9,17 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Description The application was found to return error data such as stack traces. Depending on the data contained within the error message, -this information could be used by an attacker to conduct further attacks. While stack traces are helpful during development -and debugging, they should not be presented to users when an error occurs. +this information could be used by an attacker to conduct further attacks. While stack traces are helpful during development +and debugging, they should not be presented to users when an error occurs. ## Remediation Applications should handle exception conditions internally and map known failure types to error codes that can be displayed to a user. These error codes should be customized to the application and returned along with the relevant HTTP error code. -When an error occurs, the application identifies the error type or class, and displays a numerical value to the -user. Requests should also be tracked so when a user is presented with an error code, it has a corresponding request ID. -Support teams can then correlate the HTTP error, the customized error code, and the request ID in the log files to +When an error occurs, the application identifies the error type or class, and displays a numerical value to the +user. Requests should also be tracked so when a user is presented with an error code, it has a corresponding request ID. +Support teams can then correlate the HTTP error, the customized error code, and the request ID in the log files to determine the root cause of the error without leaking details to the end user. Example of returning customized errors: diff --git a/doc/user/application_security/dast/dast_troubleshooting.md b/doc/user/application_security/dast/dast_troubleshooting.md index 0c7a9806c72..4e87f1898cc 100644 --- a/doc/user/application_security/dast/dast_troubleshooting.md +++ b/doc/user/application_security/dast/dast_troubleshooting.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference, howto --- -# Dynamic Application Security Testing (DAST) Troubleshooting **(ULTIMATE)** +# Troubleshooting Dynamic Application Security Testing (DAST) **(ULTIMATE)** The following troubleshooting scenarios have been collected from customer support cases. If you experience a problem not addressed here, or the information here does not fix your problem, create a diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index a49dd8fd646..0f446ddee3e 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -280,7 +280,7 @@ page. You can enable or configure DAST settings using the UI. The generated settings are formatted so they can be conveniently pasted into the `.gitlab-ci.yml` file. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **Dynamic Application Security Testing (DAST)** section, select **Enable DAST** or **Configure DAST**. @@ -357,13 +357,9 @@ variables: #### Import API specification from a file If your API specification file is in your repository, you can provide its filename as the target. -The API specification file must be in the `/zap/wrk` directory. ```yaml dast: - before_script: - - mkdir -p /zap/wrk - - cp api-specification.yml /zap/wrk/api-specification.yml variables: GIT_STRATEGY: fetch DAST_API_SPECIFICATION: api-specification.yml @@ -1075,7 +1071,7 @@ The on-demand DAST scan runs and the project's dashboard shows the results. To run a saved on-demand scan: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > On-demand Scans**. 1. Select the **Scan library** tab. 1. In the scan's row, select **Run scan**. @@ -1094,7 +1090,7 @@ The on-demand DAST scan runs, and the project's dashboard shows the results. To schedule a scan: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > On-demand Scans**. 1. Select **New scan**. 1. Complete the **Scan name** and **Description** text boxes. @@ -1143,14 +1139,16 @@ To delete an on-demand scan: 1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Delete**. 1. Select **Delete** to confirm the deletion. -### Site profile +## Site profile -A site profile describes the attributes of a web site to scan on demand with DAST. A site profile is -required for an on-demand DAST scan. +A site profile defines the attributes and configuration details of the deployed application, +website, or API to be scanned by DAST. A site profile can be referenced in `.gitlab-ci.yml` and +on-demand scans. -A site profile contains the following: +A site profile contains: -- **Profile name**: A name you assign to the site to be scanned. +- **Profile name**: A name you assign to the site to be scanned. While a site profile is referenced + in either `.gitlab-ci.yml` or an on-demand scan, it **cannot** be renamed. - **Site type**: The type of target to be scanned, either website or API scan. - **Target URL**: The URL that DAST runs against. - **Excluded URLs**: A comma-separated list of URLs to exclude from the scan. @@ -1168,7 +1166,7 @@ When an API site type is selected, a [host override](#host-override) is used to When configured, request headers and password fields are encrypted using [`aes-256-gcm`](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) before being stored in the database. This data can only be read and decrypted with a valid secrets file. -#### Site profile validation +### Site profile validation > - Site profile validation [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233020) in GitLab 13.8. > - Meta tag validation [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6460) in GitLab 14.2. @@ -1192,7 +1190,7 @@ All these methods are equivalent in functionality. Use whichever is feasible. In [GitLab 14.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/324990), site profile validation happens in a CI job using the [GitLab Runner](../../../ci/runners/index.md). -#### Create a site profile +### Create a site profile To create a site profile: @@ -1203,7 +1201,7 @@ To create a site profile: The site profile is created. -#### Edit a site profile +### Edit a site profile If a site profile is linked to a security policy, a user cannot edit the profile from this page. See [Scan execution policies](../policies/scan-execution-policies.md) @@ -1220,7 +1218,7 @@ To edit a site profile: 1. In the profile's row select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**. 1. Edit the fields then select **Save profile**. -#### Delete a site profile +### Delete a site profile If a site profile is linked to a security policy, a user cannot delete the profile from this page. See [Scan execution policies](../policies/scan-execution-policies.md) @@ -1234,13 +1232,13 @@ To delete a site profile: 1. In the profile's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. 1. Select **Delete** to confirm the deletion. -#### Validate a site profile +### Validate a site profile Validating a site is required to run an active scan. To validate a site profile: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. 1. Select the **Site Profiles** tab. @@ -1266,7 +1264,7 @@ To validate a site profile: The site is validated and an active scan can run against it. A site profile's validation status is revoked only when it's revoked manually, or its file, header, or meta tag is edited. -#### Retry a failed validation +### Retry a failed validation > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322609) in GitLab 14.3. > - [Deployed behind the `dast_failed_site_validations` flag](../../../administration/feature_flags.md), enabled by default. @@ -1277,13 +1275,13 @@ page. To retry a site profile's failed validation: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. 1. Select the **Site Profiles** tab. 1. In the profile's row, select **Retry validation**. -#### Revoke a site profile's validation status +### Revoke a site profile's validation status WARNING: When a site profile's validation status is revoked, all site profiles that share the same URL also @@ -1297,12 +1295,12 @@ To revoke a site profile's validation status: The site profile's validation status is revoked. -#### Validated site profile headers +### Validated site profile headers The following are code samples of how you can provide the required site profile header in your application. -##### Ruby on Rails example for on-demand scan +#### Ruby on Rails example for on-demand scan Here's how you can add a custom header in a Ruby on Rails application: @@ -1315,7 +1313,7 @@ class DastWebsiteTargetController < ActionController::Base end ``` -##### Django example for on-demand scan +#### Django example for on-demand scan Here's how you can add a [custom header in Django](https://docs.djangoproject.com/en/2.2/ref/request-response/#setting-header-fields): @@ -1329,7 +1327,7 @@ class DastWebsiteTargetView(View): return response ``` -##### Node (with Express) example for on-demand scan +#### Node (with Express) example for on-demand scan Here's how you can add a [custom header in Node (with Express)](https://expressjs.com/en/5x/api.html#res.append): @@ -1341,22 +1339,26 @@ app.get('/dast-website-target', function(req, res) { }) ``` -### Scanner profile +## Scanner profile > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222767) in GitLab 13.4. > - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/225804) in GitLab 13.5: scan mode, AJAX spider, debug messages. -A scanner profile defines the scanner settings used to run an on-demand scan: +A scanner profile defines the configuration details of a security scanner. A scanner profile can be +referenced in `.gitlab-ci.yml` and on-demand scans. -- **Profile name:** A name you give the scanner profile. For example, "Spider_15". +A scanner profile contains: + +- **Profile name:** A name you give the scanner profile. For example, "Spider_15". While a scanner + profile is referenced in either `.gitlab-ci.yml` or an on-demand scan, it **cannot** be renamed. - **Scan mode:** A passive scan monitors all HTTP messages (requests and responses) sent to the target. An active scan attacks the target to find potential vulnerabilities. - **Spider timeout:** The maximum number of minutes allowed for the spider to traverse the site. - **Target timeout:** The maximum number of seconds DAST waits for the site to be available before starting the scan. -- **AJAX spider:** Run the AJAX spider, in addition to the traditional spider, to crawl the target site. +- **AJAX spider:** Run the AJAX spider, in addition to the traditional spider, to crawl the target site. - **Debug messages:** Include debug messages in the DAST console output. -#### Create a scanner profile +### Create a scanner profile To create a scanner profile: @@ -1366,7 +1368,7 @@ To create a scanner profile: 1. Complete the form. For details of each field, see [Scanner profile](#scanner-profile). 1. Select **Save profile**. -#### Edit a scanner profile +### Edit a scanner profile If a scanner profile is linked to a security policy, a user cannot edit the profile from this page. See [Scan execution policies](../policies/scan-execution-policies.md) @@ -1381,7 +1383,7 @@ To edit a scanner profile: 1. Edit the form. 1. Select **Save profile**. -#### Delete a scanner profile +### Delete a scanner profile If a scanner profile is linked to a security policy, a user cannot delete the profile from this page. See [Scan execution policies](../policies/scan-execution-policies.md) diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md index 1f86f2ffa49..f15dce37123 100644 --- a/doc/user/application_security/dast_api/index.md +++ b/doc/user/application_security/dast_api/index.md @@ -55,6 +55,7 @@ The following projects demonstrate DAST API scanning: You can specify the API you want to scan by using: - [OpenAPI v2 or v3 Specification](#openapi-specification) +- [GraphQL Schema](#graphql-schema) - [HTTP Archive (HAR)](#http-archive-har) - [Postman Collection v2.0 or v2.1](#postman-collection) @@ -199,7 +200,119 @@ variables: DAST_API_TARGET_URL: http://test-deployment/ ``` -This is a minimal configuration for DAST API. From here you can: +This example is a minimal configuration for DAST API. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +### GraphQL Schema + +> Support for GraphQL Schema was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352780) in GitLab 15.4. + +GraphQL is a query language for your API and an alternative to REST APIs. +DAST API supports testing GraphQL endpoints multiple ways: + +- Test using the GraphQL Schema. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352780) in GitLab 15.4. +- Test using a recording (HAR) of GraphQL queries. +- Test using a Postman Collection containing GraphQL queries. + +This section documents how to test using a GraphQL schema. The GraphQL schema support in +DAST API is able to query the schema from endpoints that support introspection. +Introspection is enabled by default to allow tools like GraphiQL to work. + +#### DAST API scanning with a GraphQL endpoint URL + +The GraphQL support in DAST API is able to query a GraphQL endpoint for the schema. + +NOTE: +The GraphQL endpoint must support introspection queries for this method to work correctly. + +To configure DAST API to use a GraphQL endpoint URL that provides information about the target API to test: + +1. [Include](../../../ci/yaml/index.md#includetemplate) + the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) in your `.gitlab-ci.yml` file. + +1. Provide the path to the GraphQL endpoint, for example `/api/graphql`. Specify the location by adding the `DAST_API_GRAPHQL` variable. + +1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. See the [dynamic environment solutions](#dynamic-environment-solutions) section of our documentation for more information. + +Complete example configuration of using a GraphQL endpoint path: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +dast_api: + variables: + DAST_API_GRAPHQL: /api/graphql + DAST_API_TARGET_URL: http://test-deployment/ +``` + +This example is a minimal configuration for DAST API. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +#### DAST API scanning with a GraphQL Schema file + +To configure DAST API to use a GraphQL schema file that provides information about the target API to test: + +1. [Include](../../../ci/yaml/index.md#includetemplate) + the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) in your `.gitlab-ci.yml` file. + +1. Provide the GraphQL endpoint path, for example `/api/graphql`. Specify the path by adding the `DAST_API_GRAPHQL` variable. + +1. Provide the location of the GraphQL schema file. You can provide the location as a file path + or URL. Specify the location by adding the `DAST_API_GRAPHQL_SCHEMA` variable. + +1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. See the [dynamic environment solutions](#dynamic-environment-solutions) section of our documentation for more information. + +Complete example configuration of using an GraphQL schema file: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +dast_api: + variables: + DAST_API_GRAPHQL: /api/graphql + DAST_API_GRAPHQL_SCHEMA: test-api-graphql.schema + DAST_API_TARGET_URL: http://test-deployment/ +``` + +Complete example configuration of using an GraphQL schema file URL: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +dast_api: + variables: + DAST_API_GRAPHQL: /api/graphql + DAST_API_GRAPHQL_SCHEMA: http://file-store/files/test-api-graphql.schema + DAST_API_TARGET_URL: http://test-deployment/ +``` + +This example is a minimal configuration for DAST API. From here you can: - [Run your first scan](#running-your-first-scan). - [Add authentication](#authentication). @@ -267,35 +380,266 @@ This is a minimal configuration for DAST API. From here you can: - [Add authentication](#authentication). - Learn how to [handle false positives](#handling-false-positives). -##### Postman variables +#### Postman variables + +> - Support for Postman Environment file format was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356312) in GitLab 15.1. +> - Support for multiple variable files was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356312) in GitLab 15.1. +> - Support for Postman variable scopes: Global and Environment was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356312) in GitLab 15.1. + +##### Variables in Postman Client Postman allows the developer to define placeholders that can be used in different parts of the -requests. These placeholders are called variables, as explained in [Using variables](https://learning.postman.com/docs/sending-requests/variables/). +requests. These placeholders are called variables, as explained in [using variables](https://learning.postman.com/docs/sending-requests/variables/). You can use variables to store and reuse values in your requests and scripts. For example, you can edit the collection to add variables to the document:  +Or alternatively, you can add variables in an environment: + + + You can then use the variables in sections such as URL, headers, and others:  -Variables can be defined at different [scopes](https://learning.postman.com/docs/sending-requests/variables/#variable-scopes) -(for example, Global, Collection, Environment, Local, and Data). In this example, they're defined at -the Environment scope: +Postman has grown from a basic client tool with a nice UX experience to a more complex ecosystem that allows testing APIs with scripts, creating complex collections that trigger secondary requests, and setting variables along the way. Not every feature in the Postman ecosystem is supported. For example, scripts are not supported. The main focus of the Postman support is to ingest Postman Collection definitions that are used by the Postman Client and their related variables defined in the workspace, environments, and the collections themselves. - +Postman allows creating variables in different scopes. Each scope has a different level of visibility in the Postman tools. For example, you can create a variable in a _global environment_ scope that is seen by every operation definition and workspace. You can also create a variable in a specific _environment_ scope that is only visible and used when that specific environment is selected for use. Some scopes are not always available, for example in the Postman ecosystem you can create requests in the Postman Client, these requests do not have a _local_ scope, but test scripts do. -When you export a Postman collection, only Postman collection variables are exported into the -Postman file. For example, Postman does not export environment-scoped variables into the Postman -file. +Variable scopes in Postman can be a daunting topic and not everyone is familiar with it. We strongly recommend that you read [Variable Scopes](https://learning.postman.com/docs/sending-requests/variables/#variable-scopes) from Postman documentation before moving forward. -By default, the DAST API scanner uses the Postman file to resolve Postman variable values. If a JSON file -is set in a GitLab CI/CD environment variable `DAST_API_POSTMAN_COLLECTION_VARIABLES`, then the JSON -file takes precedence to get Postman variable values. +As mentioned above, there are different variable scopes, and each of them has a purpose and can be used to provide more flexibility to your Postman document. There is an important note on how values for variables are computed, as per Postman documentation: -WARNING: -Although Postman can export environment variables into a JSON file, the format is not compatible with the JSON expected by `DAST_API_POSTMAN_COLLECTION_VARIABLES`. +> If a variable with the same name is declared in two different scopes, the value stored in the variable with narrowest scope is used. For example, if there is a global variable named `username` and a local variable named `username`, the local value is used when the request runs. + +The following is a summary of the variable scopes supported by the Postman Client and DAST API: + +- **Global Environment (Global) scope** is a special pre-defined environment that is available throughout a workspace. We can also refer to the _global environment_ scope as the _global_ scope. The Postman Client allows exporting the global environment into a JSON file, which can be used with DAST API. +- **Environment scope** is a named group of variables created by a user in the Postman Client. +The Postman Client supports a single active environment along with the global environment. The variables defined in an active user-created environment take precedence over variables defined in the global environment. The Postman Client allows exporting your environment into a JSON file, which can be used with DAST API. +- **Collection scope** is a group of variables declared in a given collection. The collection variables are available to the collection where they have been declared and the nested requests or collections. Variables defined in the collection scope take precedence over the _global environment_ scope and also the _environment_ scope. +The Postman Client can export one or more collections into a JSON file, this JSON file contains selected collections, requests, and collection variables. +- **DAST API Scope** is a new scope added by DAST API to allow users to provide extra variables, or override variables defined in other supported scopes. This scope is not supported by Postman. The _DAST API Scope_ variables are provided using a [custom JSON file format](#dast-api-scope-custom-json-file-format). + - Override values defined in the environment or collection + - Defining variables from scripts + - Define a single row of data from the unsupported _data scope_ +- **Data scope** is a group of variables in which their name and values come from JSON or CSV files. A Postman collection runner like [Newman](https://learning.postman.com/docs/running-collections/using-newman-cli/command-line-integration-with-newman/) or [Postman Collection Runner](https://learning.postman.com/docs/running-collections/intro-to-collection-runs/) executes the requests in a collection as many times as entries have the JSON or CSV file. A good use case for these variables is to automate tests using scripts in Postman. +DAST API does **not** support reading data from a CSV or JSON file. +- **Local scope** are variables that are defined in Postman scripts. DAST API does **not** support Postman scripts and by extension, variables defined in scripts. You can still provide values for the script-defined variables by defining them in one of the supported scopes, or our custom JSON format. + +Not all scopes are supported by DAST API and variables defined in scripts are not supported. The following table is sorted by broadest scope to narrowest scope. + +| Scope |Postman | DAST API | Comment | +| ------------------ |:---------:|:------------:| :--------| +| Global Environment | Yes | Yes | Special pre-defined environment | +| Environment | Yes | Yes | Named environments | +| Collection | Yes | Yes | Defined in your postman collection | +| DAST API Scope | No | Yes | Custom scope added by DAST API | +| Data | Yes | No | External files in CSV or JSON format | +| Local | Yes | No | Variables defined in scripts | + +For more details on how to define variables and export variables in different scopes, see: + +- [Defining collection variables](https://learning.postman.com/docs/sending-requests/variables/#defining-collection-variables) +- [Defining environment variables](https://learning.postman.com/docs/sending-requests/variables/#defining-environment-variables) +- [Defining global variables](https://learning.postman.com/docs/sending-requests/variables/#defining-global-variables) + +##### Exporting from Postman Client + +The Postman Client lets you export different file formats, for instance, you can export a Postman collection or a Postman environment. +The exported environment can be the global environment (which is always available) or can be any custom environment you previously have created. When you export a Postman Collection, it may contain only declarations for _collection_ and _local_ scoped variables; _environment_ scoped variables are not included. + +To get the declaration for _environment_ scoped variables, you have to export a given environment at the time. Each exported file only includes variables from the selected environment. + +For more details on exporting variables in different supported scopes, see: + +- [Exporting collections](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-collections) +- [Exporting environments](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) +- [Downloading global environments](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) + +##### DAST API Scope, custom JSON file format + +Our custom JSON file format is a JSON object where each object property represents a variable name and the property value represents the variable value. This file can be created using your favorite text editor, or it can be produced by an earlier job in your pipeline. + +This example defines two variables `base_url` and `token` in the DAST API scope: + +```json +{ + "base_url": "http://127.0.0.1/", + "token": "Token 84816165151" +} +``` + +##### Using scopes with DAST API + +The scopes: _global_, _environment_, _collection_, and _GitLab DAST API_ are supported in [GitLab 15.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/356312). GitLab 15.0 and earlier, supports only the _collection_, and _GitLab DAST API_ scopes. + +The following table provides a quick reference for mapping scope files/URLs to DAST API configuration variables: + +| Scope | How to Provide | +| ------------------ | --------------- | +| Global Environment | DAST_API_POSTMAN_COLLECTION_VARIABLES | +| Environment | DAST_API_POSTMAN_COLLECTION_VARIABLES | +| Collection | DAST_API_POSTMAN_COLLECTION | +| DAST API Scope | DAST_API_POSTMAN_COLLECTION_VARIABLES | +| Data | Not supported | +| Local | Not supported | + +The Postman Collection document automatically includes any _collection_ scoped variables. The Postman Collection is provided with the configuration variable `DAST_API_POSTMAN_COLLECTION`. This variable can be set to a single [exported Postman collection](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-collections). + +Variables from other scopes are provided through the `DAST_API_POSTMAN_COLLECTION_VARIABLES` configuration variable. The configuration variable supports a comma (`,`) delimited file list in [GitLab 15.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/356312). GitLab 15.0 and earlier, supports only one single file. The order of the files provided is not important as the files provide the needed scope information. + +The configuration variable `DAST_API_POSTMAN_COLLECTION_VARIABLES` can be set to: + +- [Exported Global environment](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) +- [Exported environments](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) +- [DAST API Custom JSON format](#dast-api-scope-custom-json-file-format) + +##### Undefined Postman variables + +There is a chance that DAST API engine does not find all variables references that your Postman collection file is using. Some cases can be: + +- You are using _data_ or _local_ scoped variables, and as stated previously these scopes are not supported by DAST API. Thus, assuming the values for these variables have not been provided through [the DAST API scope](#dast-api-scope-custom-json-file-format), then the values of the _data_ and _local_ scoped variables are undefined. +- A variable name was typed incorrectly, and the name does not match the defined variable. +- Postman Client supports a new dynamic variable that is not supported by DAST API. + +When possible, DAST API follows the same behavior as the Postman Client does when dealing with undefined variables. The text of the variable reference remains the same, and there is no text substitution. The same behavior also applies to any unsupported dynamic variables. + +For example, if a request definition in the Postman Collection references the variable `{{full_url}}` and the variable is not found it is left unchanged with the value `{{full_url}}`. + +##### Dynamic Postman variables + +In addition to variables that a user can define at various scope levels, Postman has a set of pre-defined variables called _dynamic_ variables. The [_dynamic_ variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) are already defined and their name is prefixed with a dollar sign (`$`), for instance, `$guid`. _Dynamic_ variables can be used like any other variable, and in the Postman Client, they produce random values during the request/collection run. + +An important difference between DAST API and Postman is that DAST API returns the same value for each usage of the same dynamic variables. This differs from the Postman Client behavior which returns a random value on each use of the same dynamic variable. In other words, DAST API uses static values for dynamic variables while Postman uses random values. + +The supported dynamic variables during the scanning process are: + +| Variable | Value | +| ----------- | ----------- | +| `$guid` | `611c2e81-2ccb-42d8-9ddc-2d0bfa65c1b4` | +| `$isoTimestamp` | `2020-06-09T21:10:36.177Z` | +| `$randomAbbreviation` | `PCI` | +| `$randomAbstractImage` | `http://no-a-valid-host/640/480/abstract` | +| `$randomAdjective` | `auxiliary` | +| `$randomAlphaNumeric` | `a` | +| `$randomAnimalsImage` | `http://no-a-valid-host/640/480/animals` | +| `$randomAvatarImage` | `https://no-a-valid-host/path/to/some/image.jpg` | +| `$randomBankAccount` | `09454073` | +| `$randomBankAccountBic` | `EZIAUGJ1` | +| `$randomBankAccountIban` | `MU20ZPUN3039684000618086155TKZ` | +| `$randomBankAccountName` | `Home Loan Account` | +| `$randomBitcoin` | `3VB8JGT7Y4Z63U68KGGKDXMLLH5` | +| `$randomBoolean` | `true` | +| `$randomBs` | `killer leverage schemas` | +| `$randomBsAdjective` | `viral` | +| `$randomBsBuzz` | `repurpose` | +| `$randomBsNoun` | `markets` | +| `$randomBusinessImage` | `http://no-a-valid-host/640/480/business` | +| `$randomCatchPhrase` | `Future-proofed heuristic open architecture` | +| `$randomCatchPhraseAdjective` | `Business-focused` | +| `$randomCatchPhraseDescriptor` | `bandwidth-monitored` | +| `$randomCatchPhraseNoun` | `superstructure` | +| `$randomCatsImage` | `http://no-a-valid-host/640/480/cats` | +| `$randomCity` | `Spinkahaven` | +| `$randomCityImage` | `http://no-a-valid-host/640/480/city` | +| `$randomColor` | `fuchsia` | +| `$randomCommonFileExt` | `wav` | +| `$randomCommonFileName` | `well_modulated.mpg4` | +| `$randomCommonFileType` | `audio` | +| `$randomCompanyName` | `Grady LLC` | +| `$randomCompanySuffix` | `Inc` | +| `$randomCountry` | `Kazakhstan` | +| `$randomCountryCode` | `MD` | +| `$randomCreditCardMask` | `3622` | +| `$randomCurrencyCode` | `ZMK` | +| `$randomCurrencyName` | `Pound Sterling` | +| `$randomCurrencySymbol` | `£` | +| `$randomDatabaseCollation` | `utf8_general_ci` | +| `$randomDatabaseColumn` | `updatedAt` | +| `$randomDatabaseEngine` | `Memory` | +| `$randomDatabaseType` | `text` | +| `$randomDateFuture` | `Tue Mar 17 2020 13:11:50 GMT+0530 (India Standard Time)` | +| `$randomDatePast` | `Sat Mar 02 2019 09:09:26 GMT+0530 (India Standard Time)` | +| `$randomDateRecent` | `Tue Jul 09 2019 23:12:37 GMT+0530 (India Standard Time)` | +| `$randomDepartment` | `Electronics` | +| `$randomDirectoryPath` | `/usr/local/bin` | +| `$randomDomainName` | `trevor.info` | +| `$randomDomainSuffix` | `org` | +| `$randomDomainWord` | `jaden` | +| `$randomEmail` | `Iva.Kovacek61@no-a-valid-host.com` | +| `$randomExampleEmail` | `non-a-valid-user@example.net` | +| `$randomFashionImage` | `http://no-a-valid-host/640/480/fashion` | +| `$randomFileExt` | `war` | +| `$randomFileName` | `neural_sri_lanka_rupee_gloves.gdoc` | +| `$randomFilePath` | `/home/programming_chicken.cpio` | +| `$randomFileType` | `application` | +| `$randomFirstName` | `Chandler` | +| `$randomFoodImage` | `http://no-a-valid-host/640/480/food` | +| `$randomFullName` | `Connie Runolfsdottir` | +| `$randomHexColor` | `#47594a` | +| `$randomImageDataUri` | `data:image/svg+xml;charset=UTF-8,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20version%3D%221.1%22%20baseProfile%3D%22full%22%20width%3D%22undefined%22%20height%3D%22undefined%22%3E%20%3Crect%20width%3D%22100%25%22%20height%3D%22100%25%22%20fill%3D%22grey%22%2F%3E%20%20%3Ctext%20x%3D%220%22%20y%3D%2220%22%20font-size%3D%2220%22%20text-anchor%3D%22start%22%20fill%3D%22white%22%3Eundefinedxundefined%3C%2Ftext%3E%20%3C%2Fsvg%3E` | +| `$randomImageUrl` | `http://no-a-valid-host/640/480` | +| `$randomIngverb` | `navigating` | +| `$randomInt` | `494` | +| `$randomIP` | `241.102.234.100` | +| `$randomIPV6` | `dbe2:7ae6:119b:c161:1560:6dda:3a9b:90a9` | +| `$randomJobArea` | `Mobility` | +| `$randomJobDescriptor` | `Senior` | +| `$randomJobTitle` | `International Creative Liaison` | +| `$randomJobType` | `Supervisor` | +| `$randomLastName` | `Schneider` | +| `$randomLatitude` | `55.2099` | +| `$randomLocale` | `ny` | +| `$randomLongitude` | `40.6609` | +| `$randomLoremLines` | `Ducimus in ut mollitia.\nA itaque non.\nHarum temporibus nihil voluptas.\nIste in sed et nesciunt in quaerat sed.` | +| `$randomLoremParagraph` | `Ab aliquid odio iste quo voluptas voluptatem dignissimos velit. Recusandae facilis qui commodi ea magnam enim nostrum quia quis. Nihil est suscipit assumenda ut voluptatem sed. Esse ab voluptas odit qui molestiae. Rem est nesciunt est quis ipsam expedita consequuntur.` | +| `$randomLoremParagraphs` | `Voluptatem rem magnam aliquam ab id aut quaerat. Placeat provident possimus voluptatibus dicta velit non aut quasi. Mollitia et aliquam expedita sunt dolores nam consequuntur. Nam dolorum delectus ipsam repudiandae et ipsam ut voluptatum totam. Nobis labore labore recusandae ipsam quo.` | +| `$randomLoremSentence` | `Molestias consequuntur nisi non quod.` | +| `$randomLoremSentences` | `Et sint voluptas similique iure amet perspiciatis vero sequi atque. Ut porro sit et hic. Neque aspernatur vitae fugiat ut dolore et veritatis. Ab iusto ex delectus animi. Voluptates nisi iusto. Impedit quod quae voluptate qui.` | +| `$randomLoremSlug` | `eos-aperiam-accusamus, beatae-id-molestiae, qui-est-repellat` | +| `$randomLoremText` | `Quisquam asperiores exercitationem ut ipsum. Aut eius nesciunt. Et reiciendis aut alias eaque. Nihil amet laboriosam pariatur eligendi. Sunt ullam ut sint natus ducimus. Voluptas harum aspernatur soluta rem nam.` | +| `$randomLoremWord` | `est` | +| `$randomLoremWords` | `vel repellat nobis` | +| `$randomMACAddress` | `33:d4:68:5f:b4:c7` | +| `$randomMimeType` | `audio/vnd.vmx.cvsd` | +| `$randomMonth` | `February` | +| `$randomNamePrefix` | `Dr.` | +| `$randomNameSuffix` | `MD` | +| `$randomNatureImage` | `http://no-a-valid-host/640/480/nature` | +| `$randomNightlifeImage` | `http://no-a-valid-host/640/480/nightlife` | +| `$randomNoun` | `bus` | +| `$randomPassword` | `t9iXe7COoDKv8k3` | +| `$randomPeopleImage` | `http://no-a-valid-host/640/480/people` | +| `$randomPhoneNumber` | `700-008-5275` | +| `$randomPhoneNumberExt` | `27-199-983-3864` | +| `$randomPhrase` | `You can't program the monitor without navigating the mobile XML program!` | +| `$randomPrice` | `531.55` | +| `$randomProduct` | `Pizza` | +| `$randomProductAdjective` | `Unbranded` | +| `$randomProductMaterial` | `Steel` | +| `$randomProductName` | `Handmade Concrete Tuna` | +| `$randomProtocol` | `https` | +| `$randomSemver` | `7.0.5` | +| `$randomSportsImage` | `http://no-a-valid-host/640/480/sports` | +| `$randomStreetAddress` | `5742 Harvey Streets` | +| `$randomStreetName` | `Kuhic Island` | +| `$randomTransactionType` | `payment` | +| `$randomTransportImage` | `http://no-a-valid-host/640/480/transport` | +| `$randomUrl` | `https://no-a-valid-host.net` | +| `$randomUserAgent` | `Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.9.8; rv:15.6) Gecko/20100101 Firefox/15.6.6` | +| `$randomUserName` | `Jarrell.Gutkowski` | +| `$randomUUID` | `6929bb52-3ab2-448a-9796-d6480ecad36b` | +| `$randomVerb` | `navigate` | +| `$randomWeekday` | `Thursday` | +| `$randomWord` | `withdrawal` | +| `$randomWords` | `Samoa Synergistic sticky copying Grocery` | +| `$timestamp` | `1562757107` | + +##### Example: Global Scope + +In this example, [the _global_ scope is exported](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) from the Postman Client as `global-scope.json` and provided to DAST API through the `DAST_API_POSTMAN_COLLECTION_VARIABLES` configuration variable. Here is an example of using `DAST_API_POSTMAN_COLLECTION_VARIABLES`: @@ -308,21 +652,170 @@ include: variables: DAST_API_PROFILE: Quick - DAST_API_POSTMAN_COLLECTION: postman-collection_serviceA.json - DAST_API_POSTMAN_COLLECTION_VARIABLES: variable-collection-dictionary.json + DAST_API_POSTMAN_COLLECTION: postman-collection.json + DAST_API_POSTMAN_COLLECTION_VARIABLES: global-scope.json + DAST_API_TARGET_URL: http://test-deployment/ +``` + +##### Example: Environment Scope + +In this example, [the _environment_ scope is exported](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) from the Postman Client as `environment-scope.json` and provided to DAST API through the `DAST_API_POSTMAN_COLLECTION_VARIABLES` configuration variable. + +Here is an example of using `DAST_API_POSTMAN_COLLECTION_VARIABLES`: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_POSTMAN_COLLECTION: postman-collection.json + DAST_API_POSTMAN_COLLECTION_VARIABLES: environment-scope.json DAST_API_TARGET_URL: http://test-deployment/ ``` -The file `variable-collection-dictionary.json` is a JSON document. This JSON is an object with -key-value pairs for properties. The keys are the variables' names, and the values are the variables' +##### Example: Collection Scope + +The _collection_ scope variables are included in the exported Postman Collection file and provided through the `DAST_API_POSTMAN_COLLECTION` configuration variable. + +Here is an example of using `DAST_API_POSTMAN_COLLECTION`: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_POSTMAN_COLLECTION: postman-collection.json + DAST_API_TARGET_URL: http://test-deployment/ +``` + +##### Example: DAST API Scope + +The DAST API Scope is used for two main purposes, defining _data_ and _local_ scope variables that are not supported by DAST API, and changing the value of an existing variable defined in another scope. The DAST API Scope is provided through the `DAST_API_POSTMAN_COLLECTION_VARIABLES` configuration variable. + +Here is an example of using `DAST_API_POSTMAN_COLLECTION_VARIABLES`: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_POSTMAN_COLLECTION: postman-collection.json + DAST_API_POSTMAN_COLLECTION_VARIABLES: dast-api-scope.json + DAST_API_TARGET_URL: http://test-deployment/ +``` + +The file `dast-api-scope.json` uses our [custom JSON file format](#dast-api-scope-custom-json-file-format). This JSON is an object with key-value pairs for properties. The keys are the variables' names, and the values are the variables' values. For example: - ```json - { - "base_url": "http://127.0.0.1/", - "token": "Token 84816165151" - } - ``` +```json +{ + "base_url": "http://127.0.0.1/", + "token": "Token 84816165151" +} +``` + +##### Example: Multiple Scopes + +In this example, a _global_ scope, _environment_ scope, and _collection_ scope are configured. The first step is to export our various scopes. + +- [Export the _global_ scope](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) as `global-scope.json` +- [Export the _environment_ scope](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) as `environment-scope.json` +- Export the Postman Collection which includes the _collection_ scope as `postman-collection.json` + +The Postman Collection is provided using the `DAST_API_POSTMAN_COLLECTION` variable, while the other scopes are provided using the `DAST_API_POSTMAN_COLLECTION_VARIABLES`. DAST API can identify which scope the provided files match using data provided in each file. + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_POSTMAN_COLLECTION: postman-collection.json + DAST_API_POSTMAN_COLLECTION_VARIABLES: global-scope.json,environment-scope.json + DAST_API_TARGET_URL: http://test-deployment/ +``` + +##### Example: Changing a Variables Value + +When using exported scopes, it's often the case that the value of a variable must be changed for use with DAST API. For example, a _collection_ scoped variable might contain a variable named `api_version` with a value of `v2`, while your test needs a value of `v1`. Instead of modifying the exported collection to change the value, the DAST API scope can be used to change its value. This works because the _DAST API_ scope takes precedence over all other scopes. + +The _collection_ scope variables are included in the exported Postman Collection file and provided through the `DAST_API_POSTMAN_COLLECTION` configuration variable. + +The DAST API Scope is provided through the `DAST_API_POSTMAN_COLLECTION_VARIABLES` configuration variable, but first, we must create the file. +The file `dast-api-scope.json` uses our [custom JSON file format](#dast-api-scope-custom-json-file-format). This JSON is an object with key-value pairs for properties. The keys are the variables' names, and the values are the variables' +values. For example: + +```json +{ + "api_version": "v1" +} +``` + +Our CI definition: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_POSTMAN_COLLECTION: postman-collection.json + DAST_API_POSTMAN_COLLECTION_VARIABLES: dast-api-scope.json + DAST_API_TARGET_URL: http://test-deployment/ +``` + +##### Example: Changing a Variables Value with Multiple Scopes + +When using exported scopes, it's often the case that the value of a variable must be changed for use with DAST API. For example, an _environment_ scope might contain a variable named `api_version` with a value of `v2`, while your test needs a value of `v1`. Instead of modifying the exported file to change the value, the DAST API scope can be used. This works because the _DAST API_ scope takes precedence over all other scopes. + +In this example, a _global_ scope, _environment_ scope, _collection_ scope, and _DAST API_ scope are configured. The first step is to export and create our various scopes. + +- [Export the _global_ scope](https://learning.postman.com/docs/sending-requests/variables/#downloading-global-environments) as `global-scope.json` +- [Export the _environment_ scope](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-environments) as `environment-scope.json` +- Export the Postman Collection which includes the _collection_ scope as `postman-collection.json` + +The DAST API scope is used by creating a file `dast-api-scope.json` using our [custom JSON file format](#dast-api-scope-custom-json-file-format). This JSON is an object with key-value pairs for properties. The keys are the variables' names, and the values are the variables' +values. For example: + +```json +{ + "api_version": "v1" +} +``` + +The Postman Collection is provided using the `DAST_API_POSTMAN_COLLECTION` variable, while the other scopes are provided using the `DAST_API_POSTMAN_COLLECTION_VARIABLES`. DAST API can identify which scope the provided files match using data provided in each file. + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_POSTMAN_COLLECTION: postman-collection.json + DAST_API_POSTMAN_COLLECTION_VARIABLES: global-scope.json,environment-scope.json,dast-api-scope.json + DAST_API_TARGET_URL: http://test-deployment/ +``` ## Authentication @@ -332,17 +825,19 @@ provide a script that performs an authentication flow or calculates the token. ### HTTP Basic Authentication [HTTP basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) -is an authentication method built in to the HTTP protocol and used in conjunction with +is an authentication method built into the HTTP protocol and used in conjunction with [transport layer security (TLS)](https://en.wikipedia.org/wiki/Transport_Layer_Security). -To use HTTP basic authentication, two CI/CD variables are added to your `.gitlab-ci.yml` file: -- `DAST_API_HTTP_USERNAME`: The username for authentication. -- `DAST_API_HTTP_PASSWORD`: The password for authentication. +We recommended that you [create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables) +for the password (for example, `TEST_API_PASSWORD`), and set it to be masked. You can create CI/CD +variables from the GitLab project's page at **Settings > CI/CD**, in the **Variables** section. +Because of the [limitations on masked variables](../../../ci/variables/index.md#mask-a-cicd-variable), +you should Base64-encode the password before adding it as a variable. -For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables) -(for example, `TEST_API_PASSWORD`) set to the password. You can create CI/CD variables from the -GitLab projects page at **Settings > CI/CD**, in the **Variables** section. Use that variable -as the value for `DAST_API_HTTP_PASSWORD`: +Finally, add two CI/CD variables to your `.gitlab-ci.yml` file: + +- `DAST_API_HTTP_USERNAME`: The username for authentication. +- `DAST_API_HTTP_PASSWORD_BASE64`: The Base64-encoded password for authentication. ```yaml stages: @@ -356,9 +851,13 @@ variables: DAST_API_HAR: test-api-recording.har DAST_API_TARGET_URL: http://test-deployment/ DAST_API_HTTP_USERNAME: testuser - DAST_API_HTTP_PASSWORD: $TEST_API_PASSWORD + DAST_API_HTTP_PASSWORD_BASE64: $TEST_API_PASSWORD ``` +#### Raw password + +If you do not want to Base64-encode the password (or if you are using GitLab 15.3 or earlier) you can provide the raw password `DAST_API_HTTP_PASSWORD`, instead of using `DAST_API_HTTP_PASSWORD_BASE64`. + ### Bearer tokens Bearer tokens are used by several different authentication mechanisms, including OAuth2 and JSON Web @@ -383,7 +882,7 @@ Follow these steps to provide the Bearer token with `DAST_API_OVERRIDES_ENV`: can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the **Variables** section. Due to the format of `TEST_API_BEARERAUTH` it's not possible to mask the variable. - To mask the token's value, you can create a second variable with the token value's, and define + To mask the token's value, you can create a second variable with the token values, and define `TEST_API_BEARERAUTH` with the value `{"headers":{"Authorization":"Bearer $MASKED_VARIABLE"}}`. 1. In your `.gitlab-ci.yml` file, set `DAST_API_OVERRIDES_ENV` to the variable you just created: @@ -402,12 +901,12 @@ Follow these steps to provide the Bearer token with `DAST_API_OVERRIDES_ENV`: DAST_API_OVERRIDES_ENV: $TEST_API_BEARERAUTH ``` -1. To validate that authentication is working, run an DAST API test and review the job logs +1. To validate that authentication is working, run a DAST API test and review the job logs and the test API's application logs. #### Token generated at test runtime -If the Bearer token must be generated and doesn't expire during testing, you can provide DAST API a file that has the token. A prior stage and job, or part of the DAST API job, can +If the Bearer token must be generated and doesn't expire during testing, you can provide DAST API with a file that has the token. A prior stage and job, or part of the DAST API job, can generate this file. DAST API expects to receive a JSON file with the following structure: @@ -439,7 +938,7 @@ variables: DAST_API_OVERRIDES_FILE: dast-api-overrides.json ``` -To validate that authentication is working, run an DAST API test and review the job logs and +To validate that authentication is working, run a DAST API test and review the job logs and the test API's application logs. #### Token has short expiration @@ -552,8 +1051,10 @@ can be added, removed, and modified by creating a custom configuration. |[`DAST_API_OPENAPI_ALL_MEDIA_TYPES`](#openapi-specification) | Use all supported media types instead of one when generating requests. Causes test duration to be longer. Default is disabled. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333304) in GitLab 14.10. | |[`DAST_API_OPENAPI_MEDIA_TYPES`](#openapi-specification) | Colon (`:`) separated media types accepted for testing. Default is disabled. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333304) in GitLab 14.10. | |[`DAST_API_HAR`](#http-archive-har) | HTTP Archive (HAR) file. | +|[`DAST_API_GRAPHQL`](#graphql-schema) | Path to GraphQL endpoint, for example `/api/graphql`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352780) in GitLab 15.4. | +|[`DAST_API_GRAPHQL_SCHEMA`](#graphql-schema) | A URL or filename for a GraphQL schema in JSON format. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352780) in GitLab 15.4. | |[`DAST_API_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. | -|[`DAST_API_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract postman variable values. | +|[`DAST_API_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract Postman variable values. The support for comma-separated (`,`) files was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356312) in GitLab 15.1. | |[`DAST_API_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | |[`DAST_API_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | |[`DAST_API_OVERRIDES_CMD`](#overrides) | Overrides command. | @@ -562,7 +1063,8 @@ can be added, removed, and modified by creating a custom configuration. |`DAST_API_POST_SCRIPT` | Run user command or script after scan session has finished. | |[`DAST_API_OVERRIDES_INTERVAL`](#overrides) | How often to run overrides command in seconds. Defaults to `0` (once). | |[`DAST_API_HTTP_USERNAME`](#http-basic-authentication) | Username for HTTP authentication. | -|[`DAST_API_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. | +|[`DAST_API_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. Consider using `DAST_API_HTTP_PASSWORD_BASE64` instead. | +|[`DAST_API_HTTP_PASSWORD_BASE64`](#http-basic-authentication) | Password for HTTP authentication, base64-encoded. [Introduced](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing-src/-/merge_requests/702) in GitLab 15.4. | |`DAST_API_SERVICE_START_TIMEOUT` | How long to wait for target API to become available in seconds. Default is 300 seconds. | |`DAST_API_TIMEOUT` | How long to wait for API responses in seconds. Default is 30 seconds. | @@ -1010,21 +1512,21 @@ This example excludes the `/auth` resource. This does not exclude child resource ```yaml variables: - DAST_API_EXCLUDE_PATHS=/auth + DAST_API_EXCLUDE_PATHS: /auth ``` To exclude `/auth`, and child resources (`/auth/child`), we use a wildcard. ```yaml variables: - DAST_API_EXCLUDE_PATHS=/auth* + DAST_API_EXCLUDE_PATHS: /auth* ``` To exclude multiple paths we use the `;` character. In this example we exclude `/auth*` and `/v1/*`. ```yaml variables: - DAST_API_EXCLUDE_PATHS=/auth*;/v1/* + DAST_API_EXCLUDE_PATHS: /auth*;/v1/* ``` To exclude one or more nested levels within a path we use `**`. In this example we are testing API endpoints. We are testing `/api/v1/` and `/api/v2/` of a data query requesting `mass`, `brightness` and `coordinates` data for `planet`, `moon`, `star`, and `satellite` objects. Example paths that could be scanned include, but are not limited to: @@ -1037,7 +1539,7 @@ In this example we test the `brightness` endpoint only: ```yaml variables: - DAST_API_EXCLUDE_PATHS=/api/**/mass;/api/**/coordinates + DAST_API_EXCLUDE_PATHS: /api/**/mass;/api/**/coordinates ``` ### Exclude parameters @@ -1297,7 +1799,15 @@ Each value in `DAST_API_EXCLUDE_URLS` is a regular expression. Characters such a The following example excludes the URL `http://target/api/auth` and its child resources. ```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + variables: + DAST_API_TARGET_URL: http://target/ + DAST_API_OPENAPI: test-api-specification.json DAST_API_EXCLUDE_URLS: http://target/api/auth ``` @@ -1306,7 +1816,15 @@ variables: To exclude the URLs `http://target/api/buy` and `http://target/api/sell` but allowing to scan their child resources, for instance: `http://target/api/buy/toy` or `http://target/api/sell/chair`. You could use the value `http://target/api/buy/$,http://target/api/sell/$`. This value is using two regular expressions, each of them separated by a `,` character. Hence, it contains `http://target/api/buy$` and `http://target/api/sell$`. In each regular expression, the trailing `$` character points out where the matching URL should end. ```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + variables: + DAST_API_TARGET_URL: http://target/ + DAST_API_OPENAPI: test-api-specification.json DAST_API_EXCLUDE_URLS: http://target/api/buy/$,http://target/api/sell/$ ``` @@ -1315,7 +1833,15 @@ variables: In order to exclude the URLs: `http://target/api/buy` and `http://target/api/sell`, and their child resources. To provide multiple URLs we use the `,` character as follows: ```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + variables: + DAST_API_TARGET_URL: http://target/ + DAST_API_OPENAPI: test-api-specification.json DAST_API_EXCLUDE_URLS: http://target/api/buy,http://target/api/sell ``` @@ -1324,7 +1850,15 @@ variables: In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. ```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + variables: + DAST_API_TARGET_URL: http://target/ + DAST_API_OPENAPI: test-api-specification.json DAST_API_EXCLUDE_URLS: https://target/api/v.*/user/create$ ``` @@ -1544,7 +2078,7 @@ The DAST API engine outputs an error message when it cannot establish a connecti **Error message** - In [GitLab 13.11 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/323939), `Failed to start scanner session (version header not found).` -- In GitLab 13.10 and earlier, `API Security version header not found. Are you sure that you are connecting to the API Security server?`. +- In GitLab 13.10 and earlier, `API Security version header not found. Are you sure that you are connecting to the DAST API server?`. **Solution** @@ -1568,12 +2102,15 @@ This solution is for pipelines in which the target API URL doesn't change (is st For environments where the target API remains the same, we recommend you specify the target URL by using the `DAST_API_TARGET_URL` environment variable. In your `.gitlab-ci.yml`, add a variable `DAST_API_TARGET_URL`. The variable must be set to the base URL of API testing target. For example: ```yaml +stages: + - dast + include: - - template: DAST-API.gitlab-ci.yml + - template: DAST-API.gitlab-ci.yml - variables: - DAST_API_TARGET_URL: http://test-deployment/ - DAST_API_OPENAPI: test-api-specification.json +variables: + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OPENAPI: test-api-specification.json ``` #### Dynamic environment solutions @@ -1603,7 +2140,7 @@ deploy-test-target: ### Use OpenAPI with an invalid schema -There are cases where the document is autogenerated with an invalid schema or cannot be edited manually in a timely manner. In those scenarios, the API Security is able to perform a relaxed validation by setting the variable `DAST_API_OPENAPI_RELAXED_VALIDATION`. We recommend providing a fully compliant OpenAPI document to prevent unexpected behaviors. +There are cases where the document is autogenerated with an invalid schema or cannot be edited manually in a timely manner. In those scenarios, the DAST API is able to perform a relaxed validation by setting the variable `DAST_API_OPENAPI_RELAXED_VALIDATION`. We recommend providing a fully compliant OpenAPI document to prevent unexpected behaviors. #### Edit a non-compliant OpenAPI file @@ -1620,25 +2157,25 @@ If your OpenAPI document is generated manually, load your document in the editor Relaxed validation is meant for cases when the OpenAPI document cannot meet OpenAPI specifications, but it still has enough content to be consumed by different tools. A validation is performed but less strictly in regards to document schema. -API Security can still try to consume an OpenAPI document that does not fully comply with OpenAPI specifications. To instruct API Security to perform a relaxed validation, set the variable `DAST_API_OPENAPI_RELAXED_VALIDATION` to any value, for example: +DAST API can still try to consume an OpenAPI document that does not fully comply with OpenAPI specifications. To instruct DAST API to perform a relaxed validation, set the variable `DAST_API_OPENAPI_RELAXED_VALIDATION` to any value, for example: ```yaml - stages: - - dast +stages: + - dast - include: - - template: DAST-API.gitlab-ci.yml +include: + - template: DAST-API.gitlab-ci.yml - variables: - DAST_API_PROFILE: Quick - DAST_API_TARGET_URL: http://test-deployment/ - DAST_API_OPENAPI: test-api-specification.json - DAST_API_OPENAPI_RELAXED_VALIDATION: On +variables: + DAST_API_PROFILE: Quick + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OPENAPI: test-api-specification.json + DAST_API_OPENAPI_RELAXED_VALIDATION: 'On' ``` ### `No operation in the OpenAPI document is consuming any supported media type` -API Security uses the specified media types in the OpenAPI document to generate requests. If no request can be created due to the lack of supported media types, then an error will be thrown. +DAST API uses the specified media types in the OpenAPI document to generate requests. If no request can be created due to the lack of supported media types, then an error will be thrown. **Error message** diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index acbc94cba47..a59399f7e8d 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -32,14 +32,6 @@ The Dependency Scanning analyzers' current major version number is 2. Dependency Scanning is pre-configured with a set of **default images** that are maintained by GitLab, but users can also integrate their own **custom images**. -<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> - -The [`bundler-audit`](https://gitlab.com/gitlab-org/gitlab/-/issues/289832) and [`retire.js`](https://gitlab.com/gitlab-org/gitlab/-/issues/350510) analyzers were deprecated -in GitLab 14.8 and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86704) in 15.0. -Use Gemnasium instead. - -<!--- end_remove --> - ## Official default analyzers Any custom change to the official analyzers can be achieved by using a diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 521bb6adbf0..7aabbdd3194 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -50,12 +50,12 @@ possible, we encourage you to use all of our security scanning tools: declared software dependencies (and those installed as a sub-dependency). Dependency Scanning can not detect software dependencies that are pre-bundled into the container's base image. To identify pre-bundled dependencies, enable - [Container Scanning](../container_scanning/) language scanning using the - [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/#report-language-specific-findings). -- [Container Scanning](../container_scanning/) analyzes your containers and tells + [Container Scanning](../container_scanning/index.md) language scanning using the + [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/index.md#report-language-specific-findings). +- [Container Scanning](../container_scanning/index.md) analyzes your containers and tells you about known risks in the operating system's (OS) packages. You can configure it to also report on software and language dependencies, if you enable it and use - the [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/#report-language-specific-findings). + the [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/index.md#report-language-specific-findings). Turning this variable on can result in some duplicate findings, as we do not yet de-duplicate results between Container Scanning and Dependency Scanning. For more details, efforts to de-duplicate these findings can be tracked in @@ -304,7 +304,7 @@ table.supported-languages ul { <li> <a id="notes-regarding-supported-languages-and-package-managers-3"></a> <p> - npm is only supported when `lockfileVersion = 1` or `lockfileVersion = 2`. Work to add support for `lockfileVersion = 3` is being tracked in issue <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/365176">GitLab#365176</a>. + npm is only supported when <code>lockfileVersion = 1</code> or <code>lockfileVersion = 2</code>. Work to add support for <code>lockfileVersion = 3</code> is being tracked in issue <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/365176">GitLab#365176</a>. </p> </li> <li> @@ -452,8 +452,8 @@ We only execute one build in the directory where a build file has been detected. multiple Gradle, Maven, or sbt builds, or any combination of these, `gemnasium-maven` only analyzes dependencies for the first build file that is detected. Build files are searched for in the following order: -1. `build.gradle` or `build.gradle.kts` for single or [multi-project](https://docs.gradle.org/current/userguide/intro_multi_project_builds.html) Gradle builds. 1. `pom.xml` for single or [multi-module](https://maven.apache.org/pom.html#Aggregation) Maven projects. +1. `build.gradle` or `build.gradle.kts` for single or [multi-project](https://docs.gradle.org/current/userguide/intro_multi_project_builds.html) Gradle builds. 1. `build.sbt` for single or [multi-project](https://www.scala-sbt.org/1.x/docs/Multi-Project.html) sbt builds. The search begins with the root directory and then continues with subdirectories if no builds are found in the root directory. Consequently an sbt build file in the root directory would be detected before a Gradle build file in a subdirectory. @@ -521,7 +521,7 @@ always take the latest dependency scanning artifact available. To enable Dependency Scanning in a project, you can create a merge request: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **Dependency Scanning** row, select **Configure with a merge request**. 1. Review and merge the merge request to enable Dependency Scanning. @@ -627,7 +627,7 @@ The following variables are used for configuring specific analyzers (used for a The previous tables are not an exhaustive list of all variables that can be used. They contain all specific GitLab and analyzer variables we support and test. There are many variables, such as environment variables, that you can pass in and they will work. This is a large list, many of which we may be unaware of, and as such is not documented. For example, to pass the non-GitLab environment variable `HTTPS_PROXY` to all Dependency Scanning jobs, -set it as a [custom CI/CD variable in your `.gitlab-ci.yml`](../../../ci/variables/#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file) +set it as a [custom CI/CD variable in your `.gitlab-ci.yml`](../../../ci/variables/index.md#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file) file like this: ```yaml diff --git a/doc/user/application_security/generate_test_vulnerabilities/index.md b/doc/user/application_security/generate_test_vulnerabilities/index.md index aafbebb91cd..4d424acf9c3 100644 --- a/doc/user/application_security/generate_test_vulnerabilities/index.md +++ b/doc/user/application_security/generate_test_vulnerabilities/index.md @@ -1,6 +1,6 @@ --- type: reference, howto -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/user/application_security/get-started-security.md b/doc/user/application_security/get-started-security.md index 4c2b971b5fa..ee7864b5ce9 100644 --- a/doc/user/application_security/get-started-security.md +++ b/doc/user/application_security/get-started-security.md @@ -6,25 +6,36 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Get started with GitLab application security **(ULTIMATE)** -Complete the following steps to get the most from GitLab application security tools. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Adopting GitLab application security](https://www.youtube.com/watch?v=5QlxkiKR04k). -1. Enable [Secret Detection](secret_detection/index.md) scanning for your default branch. -1. Enable [Dependency Scanning](dependency_scanning/index.md) for your default branch so you can start identifying existing +The following steps will help you get the most from GitLab application security tools. These steps are a recommended order of operations. You can choose to implement capabilities in a different order or omit features that do not apply to your specific needs. + +1. Enable [Secret Detection](secret_detection/index.md) and [Dependency Scanning](dependency_scanning/index.md) + to identify any leaked secrets and vulnerable packages in your codebase. + + - For all security scanners, enable them by updating your `[.gitlab-ci.yml](../../ci/yaml/gitlab_ci_yaml.md)` directly on your `default` branch. This creates a baseline scan of your `default` branch, which is necessary for + feature branch scans to be compared against. This allows [merge requests](../project/merge_requests/index.md) + to display only newly-introduced vulnerabilities. Otherwise, merge requests will display every + vulnerability in the branch, regardless of whether it was introduced by a change in the branch. + - If you are after simplicity, enable only Secret Detection first. It only has one analyzer, + no build requirements, and relatively simple findings: is this a secret or not? + - It is good practice to enable Dependency Scanning early so you can start identifying existing vulnerable packages in your codebase. -1. Add security scans to feature branch pipelines. The same scans should be enabled as are running - on your default branch. Subsequent scans will show only new vulnerabilities by comparing the feature branch to the default branch results. 1. Let your team get comfortable with [vulnerability reports](vulnerability_report/index.md) and establish a vulnerability triage workflow. 1. Consider creating [labels](../project/labels.md) and [issue boards](../project/issue_board.md) to help manage issues created from vulnerabilities. Issue boards allow all stakeholders to have a - common view of all issues. + common view of all issues and track remediation progress. +1. Use [scheduled pipelines](../../ci/pipelines/schedules.md#scheduled-pipelines) to regularly scan important branches such as `default` or those used for maintenance releases. + - Running regular dependency and [container scans](container_scanning/index.md) will surface newly-discovered vulnerabilities that already exist in your repository. + - Scheduled scans are most useful for projects or important branches with low development activity where pipeline scans are infrequent. 1. Create a [scan result policy](policies/index.md) to limit new vulnerabilities from being merged - into your default branch. + into your `default` branch. 1. Monitor the [Security Dashboard](security_dashboard/index.md) trends to gauge success in remediating existing vulnerabilities and preventing the introduction of new ones. 1. Enable other scan types such as [SAST](sast/index.md), [DAST](dast/index.md), [Fuzz testing](coverage_fuzzing/index.md), or [Container Scanning](container_scanning/index.md). - Be sure to add the same scan types to both feature pipelines and default branch pipelines. 1. Use [Compliance Pipelines](../../user/project/settings/index.md#compliance-pipeline-configuration) or [Scan Execution Policies](policies/scan-execution-policies.md) to enforce required scan types and ensure separation of duties between security and engineering. diff --git a/doc/user/application_security/iac_scanning/index.md b/doc/user/application_security/iac_scanning/index.md index 16f08de738b..1b9cdb11ea3 100644 --- a/doc/user/application_security/iac_scanning/index.md +++ b/doc/user/application_security/iac_scanning/index.md @@ -116,7 +116,7 @@ that you can download and analyze. To enable IaC Scanning in a project, you can create a merge request: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **Infrastructure as Code (IaC) Scanning** row, select **Configure with a merge request**. 1. Review and merge the merge request to enable IaC Scanning. diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 7c7d5380a24..fbd617351da 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -98,7 +98,7 @@ approach. ## Security scanning with Auto DevOps To enable all GitLab Security scanning tools, with default settings, enable -[Auto DevOps](../../topics/autodevops/): +[Auto DevOps](../../topics/autodevops/index.md): - [Auto SAST](../../topics/autodevops/stages.md#auto-sast) - [Auto Secret Detection](../../topics/autodevops/stages.md#auto-secret-detection) @@ -361,7 +361,7 @@ Learn more on overriding security jobs: - [Overriding SAST jobs](sast/index.md#overriding-sast-jobs). - [Overriding Dependency Scanning jobs](dependency_scanning/index.md#overriding-dependency-scanning-jobs). - [Overriding Container Scanning jobs](container_scanning/index.md#overriding-the-container-scanning-template). -- [Overriding Secret Detection jobs](secret_detection/index.md#customizing-settings). +- [Overriding Secret Detection jobs](secret_detection/index.md#configure-scan-settings). - [Overriding DAST jobs](dast/index.md#customize-dast-settings). - [Overriding License Compliance jobs](../compliance/license_compliance/index.md#overriding-the-template). @@ -397,15 +397,6 @@ Validation depends on the schema version declared in the security report artifac You can always find supported and deprecated schema versions in the [source code](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/parsers/security/validators/schema_validator.rb). -<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> - -### Enable security report validation (removed) - - This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/354928) in GitLab 14.9 - and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85400) in GitLab 15.0. - - <!--- end_remove --> - ## Interact with findings and vulnerabilities You can interact with the results of the security scanning tools in several locations: @@ -414,7 +405,7 @@ You can interact with the results of the security scanning tools in several loca - [Project Security Dashboard](security_dashboard/index.md) - [Security pipeline tab](security_dashboard/index.md) - [Group Security Dashboard](security_dashboard/index.md) -- [Security Center](security_dashboard/#security-center) +- [Security Center](security_dashboard/index.md#security-center) - [Vulnerability Report](vulnerability_report/index.md) - [Vulnerability Pages](vulnerabilities/index.md) - [Dependency List](dependency_list/index.md) @@ -455,7 +446,7 @@ Security and compliance teams must ensure that security scans: GitLab provides two methods of accomplishing this, each with advantages and disadvantages. -- [Compliance framework pipelines](../project/settings/#compliance-pipeline-configuration) +- [Compliance framework pipelines](../project/settings/index.md#compliance-pipeline-configuration) are recommended when: - Scan execution enforcement is required for any scanner that uses a GitLab template, such as SAST IaC, DAST, Dependency Scanning, @@ -497,6 +488,11 @@ Feedback is welcome on our vision for [unifying the user experience for these tw --> ### Secure job failing with exit code 1 +WARNING: +Debug logging can be a serious security risk. The output may contain the content of +environment variables and other secrets available to the job. The output is uploaded +to the GitLab server and visible in job logs. + If a Secure job is failing and it's unclear why, add `SECURE_LOG_LEVEL: "debug"` as a global CI/CD variable for more verbose output that is helpful for troubleshooting. @@ -534,6 +530,11 @@ Select **new pipeline** to run a new pipeline. ### Getting warning messages `… report.json: no matching files` +WARNING: +Debug logging can be a serious security risk. The output may contain the content of +environment variables and other secrets available to the job. The output is uploaded +to the GitLab server and visible in job logs. + This message is often followed by the [error `No files to upload`](../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload), and preceded by other errors or warnings that indicate why the JSON report wasn't generated. Check the entire job log for such messages. If you don't find these messages, retry the failed job after diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 7aeb094093c..7344695886a 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -87,9 +87,9 @@ above. You can find more information at each of the pages below: - [Container scanning offline directions](../container_scanning/index.md#running-container-scanning-in-an-offline-environment) - [SAST offline directions](../sast/index.md#running-sast-in-an-offline-environment) -- [Secret Detection offline directions](../secret_detection/#running-secret-detection-in-an-offline-environment) +- [Secret Detection offline directions](../secret_detection/index.md#running-secret-detection-in-an-offline-environment) - [DAST offline directions](../dast/run_dast_offline.md#run-dast-in-an-offline-environment) -- [API Fuzzing offline directions](../api_fuzzing/#running-api-fuzzing-in-an-offline-environment) +- [API Fuzzing offline directions](../api_fuzzing/index.md#running-api-fuzzing-in-an-offline-environment) - [License Compliance offline directions](../../compliance/license_compliance/index.md#running-license-compliance-in-an-offline-environment) - [Dependency Scanning offline directions](../dependency_scanning/index.md#running-dependency-scanning-in-an-offline-environment) @@ -236,4 +236,4 @@ an offline environment. Note that these steps are specific to GitLab Secure with AutoDevOps. Using other stages with AutoDevOps may require other steps covered in the -[Auto DevOps documentation](../../../topics/autodevops/). +[Auto DevOps documentation](../../../topics/autodevops/index.md). diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 53f9c400259..9b86ef7316a 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -1,6 +1,6 @@ --- -stage: Protect -group: Container Security +stage: Govern +group: Security Policies info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -54,7 +54,7 @@ to select, edit, and unlink a security policy project. As a project owner, take the following steps to create or edit an association between your current project and a project that you would like to designate as the security policy project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Policies**. 1. Select **Edit Policy Project**, and search for and select the project you would like to link from the dropdown menu. @@ -78,7 +78,7 @@ policies for all available environments. You can check a policy's information (for example, description or enforcement status), and create and edit deployed policies: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Policies**.  @@ -89,7 +89,7 @@ status), and create and edit deployed policies: You can use the policy editor to create, edit, and delete policies: -1. On the top bar, select **Menu > Projects** and find your group. +1. On the top bar, select **Main menu > Projects** and find your group. 1. On the left sidebar, select **Security & Compliance > Policies**. - To create a new policy, select **New policy** which is located in the **Policies** page's header. You can then select which type of policy to create. @@ -144,6 +144,6 @@ for more information on the product direction of security policies within GitLab When you create a new security policy or change an existing policy, a new branch is automatically created with the branch name following the pattern `update-policy-<timestamp>`. For example: `update-policy-1659094451`. -If you have group or instance push rules that do not allow branch name patterns that contain the text `update-policy-<timestamp>`, you will get an error that states `Branch name does not follow the pattern 'update-policy-<timestamp>'`. +If you have group or instance push rules that do not allow branch name patterns that contain the text `update-policy-<timestamp>`, you will get an error that states `Branch name does not follow the pattern 'update-policy-<timestamp>'`. The workaround is to amend your group or instance push rules to allow branches following the pattern `update-policy-` followed by an integer timestamp. diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index eb1f9a7c7b8..f2fc52a2de8 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -1,12 +1,13 @@ --- -stage: Protect -group: Container Security +stage: Govern +group: Security Policies info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Scan execution policies **(ULTIMATE)** -> Group-level security policies were [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4425) in GitLab 15.2 [with a flag](../../../administration/feature_flags.md) named `group_level_security_policies`. Enabled by default. +> - Group-level security policies were [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4425) in GitLab 15.2. +> - Group-level security policies were [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/356258) in GitLab 15.4. Group, sub-group, or project owners can use scan execution policies to require that security scans run on a specified schedule or with the project (or multiple projects if the policy is defined at a group or sub-group level) pipeline. Required scans are injected into the CI pipeline as new jobs @@ -14,10 +15,10 @@ with a long, random job name. In the unlikely event of a job name collision, the any pre-existing job in the pipeline. If a policy is created at the group-level, it will apply to every child project or sub-group. A group-level policy cannot be edited from a child project or sub-group. -This feature has some overlap with [compliance framework pipelines](../../project/settings/#compliance-pipeline-configuration), +This feature has some overlap with [compliance framework pipelines](../../project/settings/index.md#compliance-pipeline-configuration), as we have not [unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312). For details on the similarities and differences between these features, see -[Enforce scan execution](../#enforce-scan-execution). +[Enforce scan execution](../index.md#enforce-scan-execution). NOTE: Policy jobs are created in the `test` stage of the pipeline. If you modify the default pipeline @@ -88,8 +89,7 @@ This rule enforces the defined actions and schedules a scan on the provided date |------------|------|-----------------|-------------| | `type` | `string` | `schedule` | The rule's type. | | `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | -| `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. <!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> | -| `clusters` (removed) | `object` | | This field was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in 15.0. The cluster where the given policy enforces running selected scans (only for `container_scanning`/`cluster_image_scanning` scans). The key of the object is the name of the Kubernetes cluster configured for your project in GitLab. In the optionally provided value of the object, you can precisely select Kubernetes resources that are scanned. <!--- end_remove --> | +| `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. | GitLab supports the following types of CRON syntax for the `cadence` field: @@ -98,23 +98,6 @@ GitLab supports the following types of CRON syntax for the `cadence` field: It is possible that other elements of the CRON syntax will work in the cadence field, however, GitLab does not officially test or support them. -<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> - -### `cluster` schema (removed) - -This schema was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in 15.0. - -Use this schema to define `clusters` objects in the [`schedule` rule type](#schedule-rule-type). - -| Field | Type | Possible values | Description | -|--------------|---------------------|--------------------------|-------------| -| `containers` | `array` of `string` | | The container name that is scanned (only the first value is currently supported). | -| `resources` | `array` of `string` | | The resource name that is scanned (only the first value is currently supported). | -| `namespaces` | `array` of `string` | | The namespace that is scanned (only the first value is currently supported). | -| `kinds` | `array` of `string` | `deployment`/`daemonset` | The resource kind that should be scanned (only the first value is currently supported). | - -<!--- end_remove --> - ## `scan` action type This action executes the selected `scan` with additional parameters when conditions for at least one @@ -146,7 +129,7 @@ Note the following: - A container scanning and cluster image scanning scans configured for the `pipeline` rule type ignores the cluster defined in the `clusters` object. They use predefined CI/CD variables defined for your project. Cluster selection with the `clusters` object is supported for the `schedule` rule type. A cluster with a name provided in the `clusters` object must be created and configured for the project. -- The SAST scan uses the default template and runs in a [child pipeline](../../../ci/pipelines/parent_child_pipelines.md). +- The SAST scan uses the default template and runs in a [child pipeline](../../../ci/pipelines/downstream_pipelines.md#parent-child-pipelines). ## Example security policies project diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md index 3eee4957e2f..78a97b36e92 100644 --- a/doc/user/application_security/policies/scan-result-policies.md +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -1,6 +1,6 @@ --- -stage: Protect -group: Container Security +stage: Govern +group: Security Policies info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index cbd64e278c8..ec8e8e6fd93 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) from GitLab Ultimate to GitLab Free in 13.3. Static Application Security Testing (SAST) uses analyzers -to detect vulnerabilities in source code. Each analyzer is a wrapper around a [scanner](../terminology/#scanner), a third-party code analysis tool. +to detect vulnerabilities in source code. Each analyzer is a wrapper around a [scanner](../terminology/index.md#scanner), a third-party code analysis tool. The analyzers are published as Docker images that SAST uses to launch dedicated containers for each analysis. @@ -20,7 +20,7 @@ For each scanner, an analyzer: - Exposes its detection logic. - Handles its execution. -- Converts its output to a [standard format](../terminology/#secure-report-format). +- Converts its output to a [standard format](../terminology/index.md#secure-report-format). ## SAST analyzers @@ -77,7 +77,7 @@ You can choose to disable the other analyzers early and use Semgrep-based scanni - You'll enjoy significantly faster scanning, reduced CI minutes usage, and more customizable scanning rules. - However, vulnerabilities previously reported by language-specific analyzers will be reported again under certain conditions, including if you've dismissed the vulnerabilities before. The system behavior depends on: - whether you've excluded the Semgrep-based analyzer from running in the past. - - which analyzer first discovered the vulnerabilities shown in the project's [Vulnerability Report](../vulnerability_report/). + - which analyzer first discovered the vulnerabilities shown in the project's [Vulnerability Report](../vulnerability_report/index.md). ### Vulnerability translation @@ -103,7 +103,7 @@ You can choose to use Semgrep-based scanning instead of language-specific analyz We recommend taking this approach if any of these cases applies: -- You haven't used SAST before on a project, so you don't already have SAST vulnerabilities in your [Vulnerability Report](../vulnerability_report/). +- You haven't used SAST before on a project, so you don't already have SAST vulnerabilities in your [Vulnerability Report](../vulnerability_report/index.md). - You're having trouble configuring one of the analyzers whose coverage overlaps with Semgrep-based coverage. For example, you might have trouble setting up the SpotBugs-based analyzer to compile your code. - You've already seen and dismissed vulnerabilities created by ESLint, Gosec, or Flawfinder scanning, and you've kept the re-created vulnerabilities created by Semgrep. @@ -120,6 +120,36 @@ To switch to Semgrep-based scanning early, you can: 1. Merge the MR and wait for the default-branch pipeline to run. 1. Use the Vulnerability Report to dismiss the findings that are no longer detected by the language-specific analyzers. +#### Preview Semgrep-based scanning + +You can see how Semgrep-based scanning will work in your projects before the GitLab-managed Stable CI/CD template for SAST is updated. +We recommend that you test this change in a merge request but continue using the Stable template in your default branch pipeline configuration. + +In GitLab 15.3, we [activated a feature flag](https://gitlab.com/gitlab-org/gitlab/-/issues/362179) to migrate security findings on the default branch from other analyzers to Semgrep. +We plan to [plan to remove the deprecated analyzers](https://gitlab.com/gitlab-org/gitlab/-/issues/352554) from the Stable CI/CD template in GitLab 15.4. + +To preview the upcoming changes to the CI/CD configuration: + +1. Open an MR to switch from the Stable CI/CD template, `SAST.gitlab-ci.yaml`, to [the Latest template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.latest.gitlab-ci.yml), `SAST.latest.gitlab-ci.yaml`. + - On GitLab.com, use the latest template directly: + + ```yaml + include: + template: 'SAST.latest.gitlab-ci.yaml' + ``` + + - On a Self-Managed instance, download the template from GitLab.com: + + ```yaml + include: + remote: 'https://gitlab.com/gitlab-org/gitlab/-/raw/2851f4d5/lib/gitlab/ci/templates/Jobs/SAST.latest.gitlab-ci.yml' + ``` + +1. Verify that scanning jobs succeed in the MR. You'll notice findings from the removed analyzers in _Fixed_ and findings from Semgrep in _New_. (Some findings may show different names, descriptions, and severities, since GitLab manages and edits the Semgrep rulesets.) +1. Close the MR. + +To learn more about Stable and Latest templates, see documentation on [CI/CD template versioning](../../../development/cicd/templates.md#versioning). + ## Customize analyzers Use [CI/CD variables](index.md#available-cicd-variables) diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index d4b0d5b972c..c9bfecffecc 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -78,16 +78,17 @@ You can also [view our language roadmap](https://about.gitlab.com/direction/secu |------------------------------------------------|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------| | .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | | .NET Framework<sup>1</sup> | [Security Code Scan](https://security-code-scan.github.io) | 13.0 | +| .NET (all versions, C# only) | [Semgrep](https://semgrep.dev) | 15.4 | | Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 | | C | [Semgrep](https://semgrep.dev) | 14.2 | | C/C++ | [Flawfinder](https://github.com/david-a-wheeler/flawfinder) | 10.7 | | Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.1 | | Go | [Gosec](https://github.com/securego/gosec) | 10.7 | | Go | [Semgrep](https://semgrep.dev) | 14.4 | -| Groovy<sup>2</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Ant, Maven, SBT) | +| Groovy<sup>2</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Maven, SBT) | | Helm Charts | [Kubesec](https://github.com/controlplaneio/kubesec) | 13.1 | | Java (any build system) | [Semgrep](https://semgrep.dev) | 14.10 | -| Java<sup>2</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (Ant, SBT) | +| Java<sup>2</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (SBT) | | Java (Android) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 | | JavaScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8 | | JavaScript | [Semgrep](https://semgrep.dev) | 13.10 | @@ -103,16 +104,16 @@ You can also [view our language roadmap](https://about.gitlab.com/direction/secu | React | [Semgrep](https://semgrep.dev) | 13.10 | | Ruby | [brakeman](https://brakemanscanner.org) | 13.9 | | Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 | -| Scala<sup>2</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven) | +| Scala<sup>2</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Gradle, Maven) | | Swift (iOS) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 | | TypeScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.9, [merged](https://gitlab.com/gitlab-org/gitlab/-/issues/36059) with ESLint in 13.2 | | TypeScript | [Semgrep](https://semgrep.dev) | 13.10 | -1. .NET 4 support is limited. The analyzer runs in a Linux container and does not have access to Windows-specific libraries or features. We currently plan to [migrate C# coverage to Semgrep-based scanning](https://gitlab.com/gitlab-org/gitlab/-/issues/347258) to make it easier to scan C# projects. -1. The SpotBugs-based analyzer supports [Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/), and [SBT](https://www.scala-sbt.org/). It can also be used with variants like the +1. .NET 4 support is limited. The analyzer runs in a Linux container and does not have access to Windows-specific libraries or features. Use the Semgrep-based scanner if you need .NET 4 support. +1. The SpotBugs-based analyzer supports [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/), and [SBT](https://www.scala-sbt.org/). It can also be used with variants like the [Gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html), [Grails](https://grails.org/), -and the [Maven wrapper](https://github.com/takari/maven-wrapper). +and the [Maven wrapper](https://github.com/takari/maven-wrapper). However, SpotBugs has [limitations](https://gitlab.com/gitlab-org/gitlab/-/issues/350801) when used against [Ant](https://ant.apache.org/)-based projects. We recommend using the Semgrep-based analyzer for Ant-based Java projects. ### Multi-project support @@ -242,7 +243,7 @@ successfully, and an error may occur. To enable and configure SAST with default settings: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance** > **Configuration**. 1. In the SAST section, select **Configure with a merge request**. 1. Review and merge the merge request to enable SAST. @@ -262,7 +263,7 @@ successfully, and an error may occur. To enable and configure SAST with customizations: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. If the project does not have a `.gitlab-ci.yml` file, select **Enable SAST** in the Static Application Security Testing (SAST) row, otherwise select **Configure SAST**. @@ -337,7 +338,7 @@ False positive detection is available in a subset of the [supported languages](# > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5144) in GitLab 14.2. Source code is volatile; as developers make changes, source code may move within files or between files. -Security analyzers may have already reported vulnerabilities that are being tracked in the [Vulnerability Report](../vulnerability_report/). +Security analyzers may have already reported vulnerabilities that are being tracked in the [Vulnerability Report](../vulnerability_report/index.md). These vulnerabilities are linked to specific problematic code fragments so that they can be found and fixed. If the code fragments are not tracked reliably as they move, vulnerability management is harder because the same vulnerability could be reported again. @@ -550,8 +551,8 @@ Some analyzers can be customized with CI/CD variables. | `KUBESEC_HELM_CHARTS_PATH` | Kubesec | Optional path to Helm charts that `helm` uses to generate a Kubernetes manifest that `kubesec` scans. If dependencies are defined, `helm dependency build` should be ran in a `before_script` to fetch the necessary dependencies. | | `KUBESEC_HELM_OPTIONS` | Kubesec | Additional arguments for the `helm` executable. | | `COMPILE` | Gosec, SpotBugs | Set to `false` to disable project compilation and dependency fetching. [Introduced for `SpotBugs`](https://gitlab.com/gitlab-org/gitlab/-/issues/195252) analyzer in GitLab 13.1 and [`Gosec`](https://gitlab.com/gitlab-org/gitlab/-/issues/330678) analyzer in GitLab 14.0. | -| `ANT_HOME` | SpotBugs | The `ANT_HOME` variable. | -| `ANT_PATH` | SpotBugs | Path to the `ant` executable. | +| `ANT_HOME` | SpotBugs | The `ANT_HOME` variable. | +| `ANT_PATH` | SpotBugs | Path to the `ant` executable. | | `GRADLE_PATH` | SpotBugs | Path to the `gradle` executable. | | `JAVA_OPTS` | SpotBugs | Additional arguments for the `java` executable. | | `JAVA_PATH` | SpotBugs | Path to the `java` executable. | @@ -565,6 +566,22 @@ Some analyzers can be customized with CI/CD variables. | `PHPCS_SECURITY_AUDIT_PHP_EXTENSIONS` | phpcs-security-audit | Comma separated list of additional PHP Extensions. | | `SAST_DISABLE_BABEL` | NodeJsScan | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64025)** in GitLab 13.5 | | `SAST_SEMGREP_METRICS` | Semgrep | Set to `"false"` to disable sending anonymized scan metrics to [r2c](https://r2c.dev/). Default: `true`. Introduced in GitLab 14.0 from the [confidential issue](../../project/issues/confidential_issues.md) `https://gitlab.com/gitlab-org/gitlab/-/issues/330565`. | +| `SAST_SCANNER_ALLOWED_CLI_OPTS` | Semgrep | CLI options (arguments with value, or flags) that are passed to the underlying security scanner when running scan operation. Only a limited set of [options](#security-scanner-configuration) are accepted. Separate a CLI option and its value using either a blank space or equals (`=`) character. For example: `name1 value1` or `name1=value1`. Multiple options must be separated by blank spaces. For example: `name1 value1 name2 value2`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368565) in GitLab 15.3. | + +#### Security scanner configuration + +SAST analyzers internally use OSS security scanners to perform the analysis. We set the recommended +configuration for the security scanner so that you need not to worry about tuning them. However, +there can be some rare cases where our default scanner configuration does not suit your +requirements. + +To allow some customization of scanner behavior, you can add a limited set of flags to the +underlying scanner. Specify the flags in the `SAST_SCANNER_ALLOWED_CLI_OPTS` CI/CD variable. These +flags are added to the scanner's CLI options. + +| Analyzer | CLI option | Description | +|------------------------------------------------------------------------------|--------------------|------------------------------------------------------------------------------| +| [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) | `--max-memory` | Sets the maximum system memory to use when running a rule on a single file. Measured in MB. | #### Custom CI/CD variables @@ -729,6 +746,31 @@ variables: SECURE_LOG_LEVEL: "debug" ``` +### Pipeline errors related to changes in the GitLab-managed CI/CD template + +The [GitLab-managed SAST CI/CD template](#configure-sast-manually) controls which [analyzer](analyzers.md) jobs run and how they're configured. While using the template, you might experience a job failure or other pipeline error. For example, you might: + +- See an error message like `'<your job>' needs 'spotbugs-sast' job, but 'spotbugs-sast' is not in any previous stage` when you view an affected pipeline. +- Experience another type of unexpected issue with your CI/CD pipeline configuration. + +If you're experiencing a job failure or seeing a SAST-related `yaml invalid` pipeline status, you can temporarily revert to an older version of the template so your pipelines keep working while you investigate the issue. To use an older version of the template, change the existing `include` statement in your CI/CD YAML file to refer to a specific template version, such as `v15.3.3-ee`: + +```yaml +include: + remote: 'https://gitlab.com/gitlab-org/gitlab/-/raw/v15.3.3-ee/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml' +``` + +If your GitLab instance has limited network connectivity, you can also download the file and host it elsewhere. + +We recommend that you only use this solution temporarily and that you return to [the standard template](#configure-sast-manually) as soon as possible. + +### Errors in a specific analyzer job + +GitLab SAST [analyzers](analyzers.md) are released as container images. +If you're seeing a new error that doesn't appear to be related to [the GitLab-managed SAST CI/CD template](#configure-sast-manually) or changes in your own project, you can try [pinning the affected analyzer to a specific older version](#pinning-to-minor-image-version). + +Each [analyzer project](analyzers.md#sast-analyzers) has a `CHANGELOG.md` file listing the changes made in each available version. + ### `Error response from daemon: error processing tar file: docker-tar: relocation error` This error occurs when the Docker version that runs the SAST job is `19.03.0`. diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 93c32f998fa..fe029b26ce5 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -6,26 +6,42 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Secret Detection **(FREE)** -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/222788) from GitLab Ultimate to GitLab Free in 13.3. - -A recurring problem when developing applications is that people may accidentally commit secrets to -their remote Git repositories. Secrets include keys, passwords, API tokens, and other sensitive +> - In GitLab 13.1, Secret Detection was split from the [SAST configuration](../sast/index.md#configuration) +> into its own CI/CD template. If you're using GitLab 13.0 or earlier and SAST is enabled, then +> Secret Detection is already enabled. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/222788) from GitLab Ultimate to GitLab +> Free in 13.3. +> - [In GitLab 14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/297269), Secret Detection jobs +> `secret_detection_default_branch` and `secret_detection` were consolidated into one job, +> `secret_detection`. + +People may accidentally commit secrets to +remote Git repositories. Secrets include keys, passwords, API tokens, and other sensitive information. Anyone with access to the repository could use the secrets for malicious purposes. -Secrets exposed in this way must be treated as compromised, and be replaced, which can be costly. -It's important to prevent secrets from being committed to a Git repository. +Exposed secrets are compromised and must be replaced, which can be costly. + +To help prevent secrets from being committed to a Git repository, you can use Secret Detection +to scan your repository for secrets. Scanning is language +and framework agnostic, but does not support scanning binary files. + +Secret Detection uses a specific analyzer containing the +[Gitleaks](https://github.com/zricethezav/gitleaks) tool to scan the repository for secrets, in a +`secret-detection` job. The results are saved as a +[Secret Detection report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportssecret_detection) +that you can later download and analyze. Due to implementation limitations, we always take the +latest Secret Detection artifact available. + +GitLab SaaS supports post-processing hooks, so you can take action when a secret is found. For +more information, see [Post-processing and revocation](post_processing.md). -Secret Detection uses the [Gitleaks](https://github.com/zricethezav/gitleaks) tool to scan the -repository for secrets. All identified secrets are reported in the: +All identified secrets are reported in the: - Merge request widget - Pipelines' **Security** tab -- [Security Dashboard](../security_dashboard/) +- [Security Dashboard](../security_dashboard/index.md)  -WARNING: -Secret Detection does not support scanning binary files. - ## Detected secrets Secret Detection uses a [default ruleset](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/blob/master/gitleaks.toml) @@ -34,105 +50,57 @@ patterns using [custom rulesets](#custom-rulesets). If you want to contribute ru "well-identifiable" secrets, follow the steps detailed in the [community contributions guidelines](https://gitlab.com/gitlab-org/gitlab/-/issues/345453). -## Requirements - -To run Secret Detection jobs, by default, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. -If you're using the shared runners on GitLab.com, this is enabled by default. - -WARNING: -Our Secret Detection jobs expect a Linux/amd64 container type. Windows containers are not supported. - -WARNING: -If you use your own runners, make sure the Docker version installed -is **not** `19.03.0`. See [troubleshooting information](../sast#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. - -### Making Secret Detection available to all GitLab tiers +## Features per tier -To make Secret Detection available to as many customers as possible, we have enabled it for all GitLab tiers. -However not all features are available on every tier. See the breakdown below for more details. +Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/). -#### Summary of features per tier +| Capability | In Free & Premium | In Ultimate | +|:---------------------------------------------------------------- |:-----------------------|:-----------------------| +| [Configure Secret Detection scanner](#enable-secret-detection) | **{check-circle}** Yes | **{check-circle}** Yes | +| [Customize Secret Detection settings](#configure-scan-settings) | **{check-circle}** Yes | **{check-circle}** Yes | +| Download [JSON Report](../sast/index.md#reports-json-format) | **{check-circle}** Yes | **{check-circle}** Yes | +| See new findings in the merge request widget | **{dotted-circle}** No | **{check-circle}** Yes | +| View identified secrets in the pipelines' **Security** tab | **{dotted-circle}** No | **{check-circle}** Yes | +| [Manage vulnerabilities](../vulnerabilities/index.md) | **{dotted-circle}** No | **{check-circle}** Yes | +| [Access the Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** No | **{check-circle}** Yes | +| [Customize Secret Detection rulesets](#custom-rulesets) | **{dotted-circle}** No | **{check-circle}** Yes | -Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), -as shown in the following table: +## Enable Secret Detection -| Capability | In Free & Premium | In Ultimate | -|:----------------------------------------------------------------|:--------------------|:-------------------| -| [Configure Secret Detection scanner](#configuration) | **{check-circle}** | **{check-circle}** | -| [Customize Secret Detection settings](#customizing-settings) | **{check-circle}** | **{check-circle}** | -| Download [JSON Report](../sast/index.md#reports-json-format) | **{check-circle}** | **{check-circle}** | -| See new findings in the merge request widget | **{dotted-circle}** | **{check-circle}** | -| View identified secrets in the pipelines' **Security** tab | **{dotted-circle}** | **{check-circle}** | -| [Manage vulnerabilities](../vulnerabilities/index.md) | **{dotted-circle}** | **{check-circle}** | -| [Access the Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | -| [Customize Secret Detection rulesets](#custom-rulesets) | **{dotted-circle}** | **{check-circle}** | +Prerequisites: -## Configuration +- GitLab Runner with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or +[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. If you're using the +shared runners on GitLab.com, this is enabled by default. +- If you use your own runners, make sure the Docker version installed is **not** `19.03.0`. See + [troubleshooting information](../sast#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) + for details. +- Linux/amd64 container type. Windows containers are not supported. +- GitLab CI/CD configuration (`.gitlab-ci.yml`) must include the `test` stage. -> - In GitLab 13.1, Secret Detection was split from the [SAST configuration](../sast#configuration) into its own CI/CD template. If you're using GitLab 13.0 or earlier and SAST is enabled, then Secret Detection is already enabled. -> - [In GitLab 14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/297269), Secret Detection jobs `secret_detection_default_branch` and `secret_detection` were consolidated into one job, `secret_detection`. +To enable Secret Detection, either: -Secret Detection is performed by a [specific analyzer](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Secret-Detection.gitlab-ci.yml) -during the `secret-detection` job. It runs regardless of your app's programming language. +- Enable [Auto DevOps](../../../topics/autodevops/index.md), which includes [Auto Secret Detection](../../../topics/autodevops/stages.md#auto-secret-detection). -The Secret Detection analyzer includes [Gitleaks](https://github.com/zricethezav/gitleaks) checks. +- [Enable Secret Detection by including the template](#enable-secret-detection-by-including-the-template). -Note that the Secret Detection analyzer ignores Password-in-URL vulnerabilities if the password -begins with a dollar sign (`$`), as this likely indicates the password is an environment variable. -For example, `https://username:$password@example.com/path/to/repo` isn't detected, while -`https://username:password@example.com/path/to/repo` is. +- [Enable Secret Detection using a merge request](#enable-secret-detection-using-a-merge-request). -NOTE: -You don't have to configure Secret Detection manually as shown in this section if you're using -[Auto Secret Detection](../../../topics/autodevops/stages.md#auto-secret-detection), -provided by [Auto DevOps](../../../topics/autodevops/index.md). +### Enable Secret Detection by including the template -To enable Secret Detection for GitLab 13.1 and later, you must include the -`Secret-Detection.gitlab-ci.yml` template that's provided as a part of your GitLab installation. For -GitLab versions earlier than 11.9, you can copy and use the job as defined in that template. +We recommend this method if you have an existing GitLab CI/CD configuration file. -Ensure your `.gitlab-ci.yml` file has a `stage` called `test`, and add the following to your `.gitlab-ci.yml` file: +Add the following extract to your `.gitlab-ci.yml` file: ```yaml include: - template: Security/Secret-Detection.gitlab-ci.yml ``` -The included template creates Secret Detection jobs in your CI/CD pipeline and scans -your project's source code for secrets. - -The results are saved as a -[Secret Detection report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportssecret_detection) -that you can later download and analyze. Due to implementation limitations, we -always take the latest Secret Detection artifact available. - -### Supported distributions - -The default scanner images are build off a base Alpine image for size and maintainability. - -#### FIPS-enabled images - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6479) in GitLab 14.10. - -GitLab offers [Red Hat UBI](https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image) -versions of the images that are FIPS-enabled. To use the FIPS-enabled images, you can either: - -- Set the `SAST_IMAGE_SUFFIX` to `-fips`. -- Add the `-fips` extension to the default image name. - -For example: - -```yaml -variables: - SECRET_DETECTION_IMAGE_SUFFIX: '-fips' - -include: - - template: Security/Secret-Detection.gitlab-ci.yml -``` +Pipelines now include a Secret Detection job, and the results are included in the merge request +widget. -### Enable Secret Detection via an automatic merge request +### Enable Secret Detection using a merge request > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4496) in GitLab 13.11, deployed behind a feature flag, enabled by default. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/329886) in GitLab 14.1. @@ -142,45 +110,36 @@ This method works best with no existing `.gitlab-ci.yml` file, or with a minimal file. If you have a complex GitLab configuration file it may not be parsed successfully, and an error may occur. -To enable Secret Detection in a project, you can create a merge request: +To enable Secret Detection using a merge request: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. In the **Secret Detection** row, select **Configure with a merge request**. -1. Review and merge the merge request to enable Secret Detection. +1. Review and merge the merge request. -Pipelines now include a Secret Detection job. +Pipelines now include a Secret Detection job, and the results are included in the merge request +widget. -### Customizing settings +## Configure scan settings The Secret Detection scan settings can be changed through [CI/CD variables](#available-cicd-variables) -by using the -[`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. +by using the [`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. WARNING: -All customization of GitLab security scanning tools should be tested in a merge request before +All configuration of GitLab security scanning tools should be tested in a merge request before merging these changes to the default branch. Failure to do so can give unexpected results, including a large number of false positives. To override a job definition, (for example, change properties like `variables` or `dependencies`), -declare a job with the same name as the secret detection job to override. Place this new job after the template -inclusion and specify any additional keys under it. - -WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except) -is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/index.md#rules) instead. +declare a job with the same name as the secret detection job to override. Place this new job after +the template inclusion and specify any additional keys under it. -#### `GIT_DEPTH` variable +In the following example _extract_ of a `.gitlab-ci.yml` file: -The [`GIT_DEPTH` CI/CD variable](../../../ci/runners/configure_runners.md#shallow-cloning) affects Secret Detection. -The Secret Detection analyzer relies on generating patches between commits to scan content for -secrets. If you override the default, ensure the value is greater than 1. If the number of commits -in an MR is greater than the `GIT_DEPTH` value, Secret Detection will [fail to detect secrets](#error-couldnt-run-the-gitleaks-command-exit-status-2). - -#### Custom settings example - -In the following example, we include the Secret Detection template and at the same time we -override the `secret_detection` job with the `SECRET_DETECTION_HISTORIC_SCAN` CI/CD variable to `true`: +- The Secret Detection template is [included](../../../ci/yaml/index.md#include). +- In the `secret_detection` job, the CI/CD variable `SECRET_DETECTION_HISTORIC_SCAN` is set to + `true`. Because the template is evaluated before the pipeline configuration, the last mention of + the variable takes precedence. ```yaml include: @@ -191,10 +150,7 @@ secret_detection: SECRET_DETECTION_HISTORIC_SCAN: "true" ``` -Because the template is [evaluated before](../../../ci/yaml/index.md#include) -the pipeline configuration, the last mention of the variable takes precedence. - -#### Available CI/CD variables +### Available CI/CD variables Secret Detection can be customized by defining available CI/CD variables: @@ -202,7 +158,7 @@ Secret Detection can be customized by defining available CI/CD variables: |-----------------------------------|---------------|-------------| | `SECRET_DETECTION_EXCLUDED_PATHS` | "" | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs (see [`doublestar.Match`](https://pkg.go.dev/github.com/bmatcuk/doublestar/v4@v4.0.2#Match) for supported patterns), or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225273) in GitLab 13.3. | | `SECRET_DETECTION_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | -| `SECRET_DETECTION_IMAGE_SUFFIX` | "" | Suffix added to the image name. If set to `-fips`, `FIPS-enabled` images are used for scan. See [FIPS-enabled images](#fips-enabled-images) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355519) in GitLab 14.10. | +| `SECRET_DETECTION_IMAGE_SUFFIX` | "" | Suffix added to the image name. If set to `-fips`, `FIPS-enabled` images are used for scan. See [Use FIPS-enabled images](#use-fips-enabled-images) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355519) in GitLab 14.10. | | `SECRET_DETECTION_LOG_OPTIONS` | "" | [`git log`](https://git-scm.com/docs/git-log) options used to define commit ranges. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350660) in GitLab 15.1.| In previous GitLab versions, the following variables were also available: @@ -211,45 +167,79 @@ In previous GitLab versions, the following variables were also available: |-----------------------------------|---------------|-------------| | `SECRET_DETECTION_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | | `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | -| `SECRET_DETECTION_COMMITS` | - | The list of commits that Gitleaks should scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/352565) in GitLab 15.0. | +| `SECRET_DETECTION_COMMITS` | - | The list of commits that Gitleaks should scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/352565) in GitLab 15.0. | -### Custom rulesets **(ULTIMATE)** +#### Use FIPS-enabled images -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211387) in GitLab 13.5. -> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/339614) support for -> passthrough chains. Expanded to include additional passthrough types of `file`, `git`, and `url` in GitLab 14.6. -> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/235359) support for overriding rules in GitLab 14.8. +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6479) in GitLab 14.10. -You can customize the default secret detection rules provided with GitLab. -Ruleset customization supports the following capabilities that can be used -simultaneously: +The default scanner images are built off a base Alpine image for size and maintainability. GitLab +offers [Red Hat UBI](https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image) +versions of the images that are FIPS-enabled. -- [Disabling predefined rules](#disable-predefined-analyzer-rules). -- [Overriding predefined rules](#override-predefined-analyzer-rules). -- Modifying the default behavior of the Secret Detection analyzer by [synthesizing and passing a custom configuration](#synthesize-a-custom-configuration). +To use the FIPS-enabled images, either: -Customization allows replacing the default secret detection rules with rules that you define. +- Set the `SECRET_DETECTION_IMAGE_SUFFIX` CI/CD variable to `-fips`. +- Add the `-fips` extension to the default image name. -To create a custom ruleset: +For example: -1. Create a `.gitlab` directory at the root of your project, if one doesn't already exist. -1. Create a custom ruleset file named `secret-detection-ruleset.toml` in the `.gitlab` directory. +```yaml +variables: + SECRET_DETECTION_IMAGE_SUFFIX: '-fips' -#### Disable predefined analyzer rules +include: + - template: Security/Secret-Detection.gitlab-ci.yml +``` -To disable analyzer rules: +## Full history Secret Detection -1. Set the `disabled` flag to `true` in the context of a `ruleset` section. +By default, Secret Detection scans only the current state of the Git repository. Any secrets +contained in the repository's history are not detected. To address this, Secret Detection can +scan the Git repository's full history. + +We recommend you do a full history scan only once, after enabling Secret Detection. A full history +can take a long time, especially for larger repositories with lengthy Git histories. After +completing an initial full history scan, use only standard Secret Detection as part of your +pipeline. + +### Enable full history Secret Detection + +To enable full history Secret Detection, set the variable `SECRET_DETECTION_HISTORIC_SCAN` to `true` in your `.gitlab-ci.yml` file. + +## Custom rulesets **(ULTIMATE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211387) in GitLab 13.5. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/339614) support for passthrough chains. +> Expanded to include additional passthrough types of `file`, `git`, and `url` in GitLab 14.6. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/235359) support for overriding rules in +> GitLab 14.8. + +You can customize the default Secret Detection rules provided with GitLab. -1. In one or more `ruleset.identifier` subsections, list the rules that you want disabled. Every `ruleset.identifier` section has: +The following customization options can be used separately, or in combination: - - a `type` field, to name the predefined rule identifier. - - a `value` field, to name the rule to be disabled. +- [Disable predefined rules](#disable-predefined-analyzer-rules). +- [Override predefined rules](#override-predefined-analyzer-rules). +- [Synthesize a custom configuration](#synthesize-a-custom-configuration). -##### Example: Disable predefined rules of Secret Detection analyzer +### Disable predefined analyzer rules -In the following example, the disabled rules is assigned to `secrets` -by matching the `type` and `value` of identifiers: +If there are specific Secret Detection rules that you don't want active, you can disable them. + +To disable analyzer rules: + +1. Create a `.gitlab` directory at the root of your project, if one doesn't already exist. +1. Create a custom ruleset file named `secret-detection-ruleset.toml` in the `.gitlab` directory, if + one doesn't already exist. +1. Set the `disabled` flag to `true` in the context of a `ruleset` section. +1. In one or more `ruleset.identifier` subsections, list the rules to disable. Every + `ruleset.identifier` section has: + - A `type` field for the predefined rule identifier. + - A `value` field for the rule name. + +In the following example `secret-detection-ruleset.toml` file, the disabled rules are assigned to +`secrets` by matching the `type` and `value` of identifiers: ```toml [secrets] @@ -260,29 +250,30 @@ by matching the `type` and `value` of identifiers: value = "RSA private key" ``` -#### Override predefined analyzer rules +### Override predefined analyzer rules -To override rules: - -1. In one or more `ruleset.identifier` subsections, list the rules that you want to override. Every `ruleset.identifier` section has: +If there are specific Secret Detection rules you want to customize, you can override them. For +example, you might increase the severity of specific secrets. - - a `type` field, to name the predefined rule identifier that the Secret Detection analyzer uses. - - a `value` field, to name the rule to be overridden. +To override rules: +1. Create a `.gitlab` directory at the root of your project, if one doesn't already exist. +1. Create a custom ruleset file named `secret-detection-ruleset.toml` in the `.gitlab` directory, if + one doesn't already exist. +1. In one or more `ruleset.identifier` subsections, list the rules to override. Every + `ruleset.identifier` section has: + - A `type` field for the predefined rule identifier. + - A `value` field for the rule name. 1. In the `ruleset.override` context of a `ruleset` section, provide the keys to override. Any combination of keys can be overridden. Valid keys are: - - description - message - name - severity (valid options are: Critical, High, Medium, Low, Unknown, Info) -##### Example: Override predefined rules of Secret Detection analyzer - -In the following example, rules -are matched by the `type` and `value` of identifiers and -then overridden: +In the following example `secret-detection-ruleset.toml` file, rules are matched by the `type` and +`value` of identifiers and then overridden: ```toml [secrets] @@ -297,166 +288,157 @@ then overridden: severity = "Info" ``` -#### Synthesize a custom configuration +### Synthesize a custom configuration -To create a custom configuration, you can use passthrough chains. +To create a custom configuration, you can use passthrough chains. Passthroughs can also be chained +to build more complex configurations. For more details, see +[SAST Customize ruleset](../sast/customize_rulesets.md). -1. In the `secret-detection-ruleset.toml` file, do one of the following: +In the `secret-detection-ruleset.toml` file, do one of the following: - - Define a custom ruleset: +- Define a custom ruleset, for example: - ```toml - [secrets] - description = 'secrets custom rules configuration' + ```toml + [secrets] + description = 'secrets custom rules configuration' - [[secrets.passthrough]] - type = "raw" - target = "gitleaks.toml" - value = """\ - title = "gitleaks config" - # add regexes to the regex table - [[rules]] - description = "Test for Raw Custom Rulesets" - regex = '''Custom Raw Ruleset T[est]{3}''' - """ - ``` + [[secrets.passthrough]] + type = "raw" + target = "gitleaks.toml" + value = """\ + title = "gitleaks config" + # add regexes to the regex table + [[rules]] + description = "Test for Raw Custom Rulesets" + regex = '''Custom Raw Ruleset T[est]{3}''' + """ + ``` - - Provide the name of the file containing a custom ruleset: +- Provide the name of the file containing a custom ruleset, for example: - ```toml - [secrets] - description = 'secrets custom rules configuration' + ```toml + [secrets] + description = 'secrets custom rules configuration' - [[secrets.passthrough]] - type = "file" - target = "gitleaks.toml" - value = "config/gitleaks.toml" - ``` + [[secrets.passthrough]] + type = "file" + target = "gitleaks.toml" + value = "config/gitleaks.toml" + ``` -Passthroughs can also be chained to build more complex configurations. -For more details, see [SAST Customize ruleset section](../sast/customize_rulesets.md). +## Running Secret Detection in an offline environment **(PREMIUM SELF)** -### Logging level - -To control the verbosity of logs set the `SECURE_LOG_LEVEL` CI/CD variable. Messages of this logging level or higher are output. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. - -From highest to lowest severity, the logging levels are: - -- `fatal` -- `error` -- `warn` -- `info` (default) -- `debug` - -## Post-processing and revocation +For self-managed GitLab instances in an environment with limited, restricted, or intermittent access +to external resources through the internet, some configuration changes are required for the Secret +Detection job to run successfully. The instructions in this section must be completed together with +the instructions detailed in [offline environments](../offline_deployments/index.md). -Upon detection of a secret, GitLab SaaS supports post-processing hooks. -For more information, see [Post-processing and revocation](post_processing.md). +### Configure GitLab Runner -## Full History Secret Detection +By default, a runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. We recommend using this default setting, to ensure Docker images remain current. +However, if no network connectivity is available, you must change the default GitLab Runner +`pull_policy` variable. -GitLab 12.11 introduced support for scanning the full history of a repository. This new functionality -is particularly useful when you are enabling Secret Detection in a repository for the first time and you -want to perform a full secret detection scan. Running a secret detection scan on the full history can take a long time, -especially for larger repositories with lengthy Git histories. We recommend not setting this CI/CD variable -as part of your normal job definition. +Configure the GitLab Runner CI/CD variable `pull_policy` to +[`if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy). -A new configuration variable ([`SECRET_DETECTION_HISTORIC_SCAN`](#available-cicd-variables)) -can be set to change the behavior of the GitLab Secret Detection scan to run on the entire Git history of a repository. +### Use local Secret Detection analyzer image -## Running Secret Detection in an offline environment +Use a local Secret Detection analyzer image if you want to obtain the image from a local Docker +registry instead of the GitLab container registry. -For self-managed GitLab instances in an environment with limited, restricted, or intermittent access -to external resources through the internet, some adjustments are required for the Secret Detection job to -run successfully. For more information, see [Offline environments](../offline_deployments/index.md). +Prerequisites: -### Requirements for offline Secret Detection +- Importing Docker images into a local offline Docker registry depends on your + network security policy. Consult your IT staff to find an accepted and approved process + to import or temporarily access external resources. -To use Secret Detection in an offline environment, you need: +1. Import the default Secret Detection analyzer image from `registry.gitlab.com` into your + [local Docker container registry](../../packages/container_registry/index.md): -- GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). -- A Docker Container Registry with locally available copy of Secret Detection [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. -- Configure certificate checking of packages (optional). + ```plaintext + registry.gitlab.com/security-products/secret-detection:3 + ``` -GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner tries to pull Docker images from the GitLab container registry even if a local -copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) -in an offline environment if you prefer using only locally available Docker images. However, we -recommend keeping the pull policy setting to `always` if not in an offline environment, as this -enables the use of updated scanners in your CI/CD pipelines. + The Secret Detection analyzer's image is [periodically updated](../index.md#vulnerability-scanner-maintenance) + so you may need to periodically update the local copy. -### Make GitLab Secret Detection analyzer image available inside your Docker registry +1. Set the CI/CD variable `SECURE_ANALYZERS_PREFIX` to the local Docker container registry. -Import the following default Secret Detection analyzer images from `registry.gitlab.com` into your -[local Docker container registry](../../packages/container_registry/index.md): + ```yaml + include: + - template: Security/Secret-Detection.gitlab-ci.yml -```plaintext -registry.gitlab.com/security-products/secret-detection:3 -``` + variables: + SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers" + ``` -The process for importing Docker images into a local offline Docker registry depends on -**your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. These scanners are [periodically updated](../index.md#vulnerability-scanner-maintenance) -with new definitions, and you may be able to make occasional updates on your own. +The Secret Detection job should now use the local copy of the Secret Detection analyzer Docker +image, without requiring internet access. -For details on saving and transporting Docker images as a file, see Docker's documentation on -[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), -[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). +### Configure a custom Certificate Authority -### Set Secret Detection CI/CD variables to use the local Secret Detection analyzer container image +To trust a custom Certificate Authority, set the `ADDITIONAL_CA_CERT_BUNDLE` variable to the bundle +of CA certificates that you trust. Do this either in the `.gitlab-ci.yml` file, in a file +variable, or as a CI/CD variable. -Add the following configuration to your `.gitlab-ci.yml` file. You must replace -`SECURE_ANALYZERS_PREFIX` to refer to your local Docker container registry: +- In the `.gitlab-ci.yml` file, the `ADDITIONAL_CA_CERT_BUNDLE` value must contain the + [text representation of the X.509 PEM public-key certificate](https://tools.ietf.org/html/rfc7468#section-5.1). -```yaml -include: - - template: Security/Secret-Detection.gitlab-ci.yml + For example: -variables: - SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers" -``` + ```yaml + variables: + ADDITIONAL_CA_CERT_BUNDLE: | + -----BEGIN CERTIFICATE----- + MIIGqTCCBJGgAwIBAgIQI7AVxxVwg2kch4d56XNdDjANBgkqhkiG9w0BAQsFADCB + ... + jWgmPqF3vUbZE0EyScetPJquRFRKIesyJuBFMAs= + -----END CERTIFICATE----- + ``` -The Secret Detection job should now use the local copy of the Secret Detection analyzer Docker image to scan your code and generate -security reports without requiring internet access. +- If using a file variable, set the value of `ADDITIONAL_CA_CERT_BUNDLE` to the path to the + certificate. -#### If support for Custom Certificate Authorities are needed +- If using a variable, set the value of `ADDITIONAL_CA_CERT_BUNDLE` to the text + representation of the certificate. -Support for custom certificate authorities was introduced in the following versions. +## Troubleshooting -| Analyzer | Version | -| -------- | ------- | -| secrets | [v3.0.0](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/releases/v3.0.0) | +### Set the logging level -To trust a custom Certificate Authority, set the `ADDITIONAL_CA_CERT_BUNDLE` variable to the bundle -of CA certs that you want to trust in the SAST environment. The `ADDITIONAL_CA_CERT_BUNDLE` value should contain the [text representation of the X.509 PEM public-key certificate](https://tools.ietf.org/html/rfc7468#section-5.1). For example, to configure this value in the `.gitlab-ci.yml` file, use the following: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. -```yaml -variables: - ADDITIONAL_CA_CERT_BUNDLE: | - -----BEGIN CERTIFICATE----- - MIIGqTCCBJGgAwIBAgIQI7AVxxVwg2kch4d56XNdDjANBgkqhkiG9w0BAQsFADCB - ... - jWgmPqF3vUbZE0EyScetPJquRFRKIesyJuBFMAs= - -----END CERTIFICATE----- -``` +Set the logging level to `debug` when you need diagnostic information in a Secret Detection job log. -The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. +WARNING: +Debug logging can be a serious security risk. The output may contain the content of environment +variables and other secrets available to the job. The output is uploaded to the GitLab server and +visible in job logs. -## Troubleshooting +1. In the `.gitlab-ci.yml` file, set the `SECURE_LOG_LEVEL` CI/CD variable to `debug`. +1. Run the Secret Detection job. +1. Analyze the content of the Secret Detection job. +1. In the `.gitlab-ci.yml` file, set the `SECURE_LOG_LEVEL` CI/CD variable to `info` (default). -### Getting warning message `gl-secret-detection-report.json: no matching files` +### Warning: `gl-secret-detection-report.json: no matching files` For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). ### Error: `Couldn't run the gitleaks command: exit status 2` -If a pipeline is triggered from a merge request containing 60 commits while the `GIT_DEPTH` variable's -value is less than that, the Secret Detection job fails as the clone is not deep enough to contain all of the -relevant commits. For information on the current default value, see the -[pipeline configuration documentation](../../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone). +The Secret Detection analyzer relies on generating patches between commits to scan content for +secrets. If the number of commits in a merge request is greater than the value of the +[`GIT_DEPTH` CI/CD variable](../../../ci/runners/configure_runners.md#shallow-cloning), Secret +Detection [fails to detect secrets](#error-couldnt-run-the-gitleaks-command-exit-status-2). + +For example, if a pipeline is triggered from a merge request containing 60 commits and the +`GIT_DEPTH` variable's value is less than 60, the Secret Detection job fails as the clone is not +deep enough to contain all of the relevant commits. To veridy the current value, see +[pipeline configuration](../../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone). -To confirm this as the cause of the error, set the -[logging level](../../application_security/secret_detection/index.md#logging-level) to `debug`, then +To confirm this as the cause of the error, set the [logging level](#set-the-logging-level) to `debug`, then rerun the pipeline. The logs should look similar to the following example. The text "object not found" is a symptom of this error. @@ -476,11 +458,12 @@ secret_detection: GIT_DEPTH: 100 ``` -### `secret-detection` job fails with `ERR fatal: ambiguous argument` message +### Error: `ERR fatal: ambiguous argument` -Your `secret-detection` job can fail with `ERR fatal: ambiguous argument` error if your -repository's default branch is unrelated to the branch the job was triggered for. -See issue [!352014](https://gitlab.com/gitlab-org/gitlab/-/issues/352014) for more details. +Secret Detection can fail with the message `ERR fatal: ambiguous argument` error if your +repository's default branch is unrelated to the branch the job was triggered for. See issue +[!352014](https://gitlab.com/gitlab-org/gitlab/-/issues/352014) for more details. -To resolve the issue, make sure to correctly [set your default branch](../../project/repository/branches/default.md#change-the-default-branch-name-for-a-project) on your repository. You should set it to a branch -that has related history with the branch you run the `secret-detection` job on. +To resolve the issue, make sure to correctly [set your default branch](../../project/repository/branches/default.md#change-the-default-branch-name-for-a-project) +on your repository. You should set it to a branch that has related history with the branch you run +the `secret-detection` job on. diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index f3c834e06c7..967e8da58a9 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -1,6 +1,6 @@ --- type: reference, howto -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -49,12 +49,15 @@ To reduce false negatives in [dependency scans](../../../user/application_securi > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285477) in GitLab 13.11, date range slider to visualize data between given dates. The project Security Dashboard shows the total number of vulnerabilities -over time, with up to 365 days of historical data. Data refreshes daily at 01:15 UTC. -It shows statistics for all vulnerabilities. +over time, with up to 365 days of historical data. Data refresh begins daily at 01:15 UTC via a scheduled job. +Each refresh captures a snapshot of open vulnerabilities. Data is not backported to prior days +so vulnerabilities opened after the job has already run for the day will not be reflected in the +counts until the following day's refresh job. +Project Security Dashboards show statistics for all vulnerabilities with a current status of `Needs triage` or `Confirmed` . To view total number of vulnerabilities over time: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Security Dashboard**. 1. Filter and search for what you need. - To filter the chart by severity, select the legend name. @@ -67,7 +70,7 @@ To view total number of vulnerabilities over time: To download an SVG image of the vulnerabilities chart: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Security dashboard**. 1. Select **Save chart as an image** (**{download}**). @@ -78,7 +81,7 @@ branches of projects in a group and its subgroups. To view vulnerabilities over time for a group: -1. On the top bar, select **Menu > Groups** and select a group. +1. On the top bar, select **Main menu > Groups** and select a group. 1. Select **Security > Security Dashboard**. 1. Hover over the chart to get more details about vulnerabilities. - You can display the vulnerability trends over a 30, 60, or 90-day time frame (the default is 90 days). @@ -88,15 +91,16 @@ To view vulnerabilities over time for a group: ## View project security status for a group -Use the group Security Dashboard to view the security status of projects. The security status is based -on the number of detected vulnerabilities. +Use the group Security Dashboard to view the security status of projects. To view project security status for a group: -1. On the top bar, select **Menu > Groups** and select a group. +1. On the top bar, select **Main menu > Groups** and select a group. 1. Select **Security > Security Dashboard**. -Projects are [graded](#project-vulnerability-grades) by vulnerability severity. Dismissed vulnerabilities are excluded. +Each project is assigned a letter [grade](#project-vulnerability-grades) according to the highest-severity open vulnerability. +Dismissed or resolved vulnerabilities are excluded. Each project can receive only one letter grade and will appear only once +in the Project security status report. To view vulnerabilities, go to the group's [vulnerability report](../vulnerability_report/index.md). @@ -127,13 +131,13 @@ The Security Center includes: ### View the Security Center -To view the Security Center, on the top bar, select **Menu > Security**. +To view the Security Center, on the top bar, select **Main menu > Security**. ### Add projects to the Security Center To add projects to the Security Center: -1. On the top bar, select **Menu > Security**. +1. On the top bar, select **Main menu > Security**. 1. On the left sidebar, select **Settings**, or select **Add projects**. 1. Use the **Search your projects** text box to search for and select projects. 1. Select **Add projects**. diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index ad397c3fe04..91793272cce 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -1,5 +1,5 @@ --- -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -46,14 +46,14 @@ are reintroduced and detected by subsequent scans have a _new_ vulnerability rec existing vulnerability is no longer detected in a project's `default` branch, you should change its status to **Resolved**. This ensures that if it is accidentally reintroduced in a future merge, it is reported again as a new record. You can use the Vulnerability Report's -[Activity filter](../vulnerability_report/#activity-filter) to select all vulnerabilities that are +[Activity filter](../vulnerability_report/index.md#activity-filter) to select all vulnerabilities that are no longer detected, and change their status. ## Change status of a vulnerability To change a vulnerability's status from its Vulnerability Page: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. From the **Status** dropdown list select a status, then select **Change status**. @@ -77,7 +77,7 @@ that when Jira integration is enabled, the GitLab issue feature is not available To create a GitLab issue for a vulnerability: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. Select **Create issue**. @@ -99,7 +99,7 @@ Prerequisites: To create a Jira issue for a vulnerability: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. Select **Create Jira issue**. @@ -132,7 +132,7 @@ Be aware of the following conditions between a vulnerability and a linked issue: To link a vulnerability to existing issues: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. In the **Linked issues** section, select the plus icon (**{plus}**). @@ -167,7 +167,7 @@ To resolve a vulnerability, you can either: To resolve the vulnerability with a merge request: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. From the **Resolve with merge request** dropdown list, select **Resolve with merge request**. @@ -179,7 +179,7 @@ Process the merge request according to your standard workflow. To manually apply the patch that GitLab generated for a vulnerability: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. From the **Resolve with merge request** dropdown list, select **Download patch to resolve**. @@ -197,7 +197,7 @@ Security training helps your developers learn how to fix vulnerabilities. Develo To enable security training for vulnerabilities in your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. 1. On the tab bar, select **Vulnerability Management**. 1. To enable a security training provider, turn on the toggle. @@ -215,7 +215,7 @@ Vulnerabilities with a CWE are most likely to return a training result. To view the security training for a vulnerability: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Vulnerability report**. 1. Select the vulnerability for which you want to view security training. 1. Select **View training**. diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md index 987dac677e7..aed86cd93aa 100644 --- a/doc/user/application_security/vulnerabilities/severities.md +++ b/doc/user/application_security/vulnerabilities/severities.md @@ -1,6 +1,6 @@ --- type: reference -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index 15a287356f8..ba448d410ea 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -1,6 +1,6 @@ --- type: reference, howto -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -44,7 +44,7 @@ At the project level, the Vulnerability Report also contains: To view the project-level vulnerability report: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Vulnerability report**. ## Vulnerability Report actions @@ -57,6 +57,7 @@ From the Vulnerability Report you can: - [View an issue raised for a vulnerability](#view-issues-raised-for-a-vulnerability). - [Change the status of vulnerabilities](#change-status-of-vulnerabilities). - [Export details of vulnerabilities](#export-vulnerability-details). +- [Sort vulnerabilities by date](#sort-vulnerabilities-by-date-detected). - [Manually add a vulnerability finding](#manually-add-a-vulnerability-finding). ## Vulnerability Report filters @@ -186,6 +187,12 @@ Vulnerability records cannot be deleted, so a permanent record always remains. If a vulnerability is dismissed in error, reverse the dismissal by changing its status. +## Sort vulnerabilities by date detected + +By default, vulnerabilities are sorted by severity level, with the highest-severity vulnerabilities listed at the top. + +To sort vulnerabilities by the date each vulnerability was detected, click the "Detected" column header. + ## Export vulnerability details > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213014) in the Security Center (previously known as the Instance Security Dashboard) and project-level Vulnerability Report (previously known as the Project Security Dashboard) in GitLab 13.0. @@ -247,7 +254,7 @@ To undo this action, select a different status from the same menu. To add a new vulnerability finding from your project level Vulnerability Report page: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Vulnerability Report**. 1. Select **Submit Vulnerability**. 1. Complete the fields and submit the form. diff --git a/doc/user/application_security/vulnerability_report/pipeline.md b/doc/user/application_security/vulnerability_report/pipeline.md index 32916f4c9c7..7faf273515c 100644 --- a/doc/user/application_security/vulnerability_report/pipeline.md +++ b/doc/user/application_security/vulnerability_report/pipeline.md @@ -1,6 +1,6 @@ --- type: reference, howto -stage: Secure +stage: Govern group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w To view vulnerabilities in a pipeline: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. From the list, select the pipeline you want to check for vulnerabilities. 1. Select the **Security** tab. @@ -26,7 +26,7 @@ For example, if a pipeline contains DAST and SAST jobs, but the DAST job fails b The pipeline vulnerability report only shows results contained in the security report artifacts. This report differs from the [Vulnerability Report](index.md), which contains cumulative results of all successful jobs, and from the merge request -[security widget](../#view-security-scan-information-in-merge-requests), which combines the branch results with +[security widget](../index.md#view-security-scan-information-in-merge-requests), which combines the branch results with cumulative results. Before GitLab displays results, the vulnerability findings in all pipeline reports are [deduplicated](#deduplication-process). @@ -35,7 +35,7 @@ Before GitLab displays results, the vulnerability findings in all pipeline repor **Scan details** shows a summary of vulnerability findings in the pipeline and the source reports. -GitLab displays one row of information for each [scan type](../terminology/#scan-type-report-type) artifact present in +GitLab displays one row of information for each [scan type](../terminology/index.md#scan-type-report-type) artifact present in the pipeline. Note that each scan type's total number of vulnerabilities includes dismissed findings. If the number of findings @@ -83,11 +83,11 @@ incorporated once the pipeline finishes. When a pipeline contains jobs that produce multiple security reports of the same type, it is possible that the same vulnerability finding is present in multiple reports. This duplication is common when different scanners are used to -increase coverage. The deduplication process allows you to maximize the vulnerability scanning coverage while reducing +increase coverage, but can also exist within a single report. The deduplication process allows you to maximize the vulnerability scanning coverage while reducing the number of findings you need to manage. -A finding is considered a duplicate of another finding when their [scan type](../terminology/#scan-type-report-type), -[location](../terminology/#location-fingerprint) and +A finding is considered a duplicate of another finding when their [scan type](../terminology/index.md#scan-type-report-type), +[location](../terminology/index.md#location-fingerprint), and one or more of its [identifiers](../../../development/integrations/secure.md#identifiers) are the same. The scan type must match because each can have its own definition for the location of a vulnerability. For example, @@ -95,8 +95,9 @@ static analyzers are able to locate a file path and line number, whereas a conta name instead. When comparing identifiers, GitLab does not compare `CWE` and `WASC` during deduplication because they are -"type identifiers" and are used to classify groups of vulnerabilities. Including these identifiers results in -many findings being incorrectly considered duplicates. +"type identifiers" and are used to classify groups of vulnerabilities. Including these identifiers would result in +many findings being incorrectly considered duplicates. Two findings are considered unique if none of their +identifiers match. In a set of duplicated findings, the first occurrence of a finding is kept and the remaining are skipped. Security reports are processed in alphabetical file path order, and findings are processed sequentially in the order they @@ -124,16 +125,17 @@ appear in a report. - Location fingerprint: `adc83b19e793491b1c6ea0fd8b46cd9f32e592fc` - Identifiers: CWE-798 - Deduplication result: duplicates because `CWE` identifiers are ignored. -- Example 3: matching scan type, location and identifiers. +- Example 3: matching scan type, location and an identifier. - Finding - Scan type: `container_scanning` - Location fingerprint: `adc83b19e793491b1c6ea0fd8b46cd9f32e592fc` - - Identifiers: CVE-2022-25510, CWE-259 + - Identifiers: CVE-2019-12345, CVE-2022-25510, CWE-259 - Other Finding - Scan type: `container_scanning` - Location fingerprint: `adc83b19e793491b1c6ea0fd8b46cd9f32e592fc` - Identifiers: CVE-2022-25510, CWE-798 - Deduplication result: duplicates because all criteria match, and type identifiers are ignored. + Only one identifier needs to match, in this case CVE-2022-25510. The examples above don't include the raw location values. Each scan type defines its own `fingerprint_data`, which is used to generate a `SHA1` hash that is used as the `location_fingerprint`. diff --git a/doc/user/clusters/agent/ci_cd_workflow.md b/doc/user/clusters/agent/ci_cd_workflow.md index 16b92eb92a3..7a6c6dc8cd6 100644 --- a/doc/user/clusters/agent/ci_cd_workflow.md +++ b/doc/user/clusters/agent/ci_cd_workflow.md @@ -62,7 +62,7 @@ Authorization configuration can take one or two minutes to propagate. To authorize the agent to access the GitLab project where you keep Kubernetes manifests: -1. On the top bar, select **Menu > Projects** and find the project that contains the [agent configuration file](install/index.md#create-an-agent-configuration-file) (`config.yaml`). +1. On the top bar, select **Main menu > Projects** and find the project that contains the [agent configuration file](install/index.md#create-an-agent-configuration-file) (`config.yaml`). 1. Edit the `config.yaml` file. Under the `ci_access` keyword, add the `projects` attribute. 1. For the `id`, add the path: @@ -85,7 +85,7 @@ Choose the context to run `kubectl` commands from your CI/CD scripts. To authorize the agent to access all of the GitLab projects in a group or subgroup: -1. On the top bar, select **Menu > Projects** and find the project that contains the [agent configuration file](install/index.md#create-an-agent-configuration-file) (`config.yaml`). +1. On the top bar, select **Main menu > Projects** and find the project that contains the [agent configuration file](install/index.md#create-an-agent-configuration-file) (`config.yaml`). 1. Edit the `config.yaml` file. Under the `ci_access` keyword, add the `groups` attribute. 1. For the `id`, add the path: @@ -127,7 +127,7 @@ Run `kubectl config get-contexts`. ### Environments with both certificate-based and agent-based connections -When you deploy to an environment that has both a +When you deploy to an environment that has both a [certificate-based cluster](../../infrastructure/clusters/index.md) (deprecated) and an agent connection: - The certificate-based cluster's context is called `gitlab-deploy`. This context diff --git a/doc/user/clusters/agent/gitops.md b/doc/user/clusters/agent/gitops.md index 4978b56917b..67439788ef7 100644 --- a/doc/user/clusters/agent/gitops.md +++ b/doc/user/clusters/agent/gitops.md @@ -49,7 +49,7 @@ For details, view the [architecture documentation](https://gitlab.com/gitlab-org To update a Kubernetes cluster by using GitOps, complete the following steps. -1. Ensure you have a working Kubernetes cluster, and that the manifests are in a GitLab project. +1. Ensure you have a working Kubernetes cluster, and that the manifests or [Helm charts](gitops/helm.md) are in a GitLab project. 1. In the same project, [register and install the GitLab agent](install/index.md). 1. Configure the agent configuration file so that the agent monitors the project for changes to the Kubernetes manifests. Use the [GitOps configuration reference](#gitops-configuration-reference) for guidance. @@ -112,12 +112,12 @@ a Kubernetes SIG project. You can read more about the available annotations in t ## Automatic drift remediation -Drift happens when the current configuration of an infrastructure resource differs from its expected configuration. -Typically, this is caused by manually editing resources directly through the service that created the resource. Minimizing the -risk of drift helps to ensure configuration consistency and successful operations. +Drift happens when the current configuration of an infrastructure resource differs from its desired configuration. +Typically, this is caused by manually editing resources directly rather than via the used infrastructure-as-code +mechanism. Minimizing the risk of drift helps to ensure configuration consistency and successful operations. -In GitLab, the agent for Kubernetes regularly compares the expected state from the `git` repository with -the known state from the `cluster`. Deviations from the `git` state are fixed at every check. These checks +In GitLab, the agent for Kubernetes regularly compares the desired state from the `git` repository with +the actual state from the Kubernetes cluster. Deviations from the `git` state are fixed at every check. These checks happen automatically every 5 minutes. They are not configurable. The agent uses [server-side applies](https://kubernetes.io/docs/reference/using-api/server-side-apply/). @@ -127,6 +127,7 @@ are checked for drift. This facilitates the use of in-cluster controllers to mod ## Related topics +- [Deploying Helm charts with the GitOps workflow](gitops/helm.md) - [GitOps working examples for training and demos](https://gitlab.com/groups/guided-explorations/gl-k8s-agent/gitops/-/wikis/home) - [Self-paced classroom workshop](https://gitlab-for-eks.awsworkshop.io) (Uses AWS EKS, but you can use for other Kubernetes clusters) - [Managing Kubernetes secrets in a GitOps workflow](gitops/secrets_management.md) diff --git a/doc/user/clusters/agent/gitops/helm.md b/doc/user/clusters/agent/gitops/helm.md new file mode 100644 index 00000000000..bdc2664e7ba --- /dev/null +++ b/doc/user/clusters/agent/gitops/helm.md @@ -0,0 +1,112 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Using Helm charts to update a Kubernetes cluster (Alpha) **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371019) in GitLab 15.4. + +You can deploy Helm charts to your Kubernetes cluster and keep the resources in your cluster in sync +with your charts and values. To do this, you use the pull-based GitOps features of the agent for +Kubernetes. + +This feature is in Alpha and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/7938) +to track future work. Please tell us about your use cases by leaving comments in the epic. + +NOTE: +This feature is Alpha. In future releases, to accommodate new features, the configuration format might change without notice. + +## GitOps workflow steps + +To update a Kubernetes cluster by using GitOps with charts, complete the following steps. + +1. Ensure you have a working Kubernetes cluster, and that the chart is in a GitLab project. +1. In the same project, [register and install the GitLab agent](../install/index.md). +1. Configure the agent configuration file so that the agent monitors the project for changes to the chart. + Use the [GitOps configuration reference](#helm-configuration-reference) for guidance. + +## Helm chart with GitOps workflow + +To update a Kubernetes cluster by using Helm charts: + +1. Ensure you have a working Kubernetes cluster. +1. In a GitLab project: + - Store your Helm charts. + - [Register and install the GitLab agent](../install/index.md). +1. Update the agent configuration file so that the agent monitors the project for changes to the chart. + Use the [configuration reference](#helm-configuration-reference) for guidance. + +Any time you commit updates to your chart repository, the agent applies the chart in the cluster. + +## Helm configuration reference + +The following snippet shows an example of the possible keys and values for the GitOps section of an [agent configuration file](../install/index.md#create-an-agent-configuration-file) (`config.yaml`). + +```yaml +gitops: + charts: + - release_name: my-application-release + source: + project: + id: my-group/my-project-with-chart + path: dir-in-project/with/charts + namespace: my-ns + max_history: 1 +``` + +| Keyword | Description | +|--|--| +| `charts` | List of charts you want to be applied in your cluster. Charts are applied concurrently. All charts must be in the same directory. | +| `release_name` | Required. Name of the release to use when applying the chart. | +| `id` | Required. ID of the project where Helm chart is committed. No authentication mechanisms are currently supported. | +| `path` | Optional. Path of the chart in the project repository. Root of the repository is used by default. This is the directory with the `Chart.yaml` file. | +| `namespace` | Optional. Namespace to use when applying the chart. Defaults to `default`. | +| `max_history` | Optional. Maximum number of release [revisions to store in the cluster](https://helm.sh/docs/helm/helm_history/). | + +## Automatic drift remediation + +Drift happens when the current configuration of an infrastructure resource differs from its desired configuration. +Typically, drift is caused by manually editing resources directly, rather than by editing the code that describes the desired state. Minimizing the risk of drift helps to ensure configuration consistency and successful operations. +mechanism. Minimizing the risk of drift helps to ensure configuration consistency and successful operations. + +In GitLab, the agent for Kubernetes regularly compares the desired state from the chart source with +the actual state from the Kubernetes cluster. Deviations from the desired state are fixed at every check. These checks +happen automatically every 5 minutes. They are not configurable. + +## Example repository layout + +```plaintext +/my-chart + ├── templates + | └── ... + ├── charts + | └── ... + ├── Chart.yaml + ├── Chart.lock + ├── values.yaml + ├── values.schema.json + └── some-file-used-in-chart.txt +``` + +## Known issues + +The following are known issues: + +- Your chart must be in a GitLab project. The project must be an agent configuration project or a public + project. This known issue also exists for manifest-based GitOps and is tracked in + [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7704). +- Values for the chart must be in a `values.yaml` file. This file must be with the chart, + in the same project and path. +- Because of drift detection and remediation, release history, stored in the cluster, is not useful. + A new release is created every five minutes and the oldest release is discarded. + Eventually history consists only of the same information. + View [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/372023) for details. + +## Troubleshooting + +### Agent cannot find values for the chart + +Make sure values are in `values.yaml` and in the same directory as the `Chart.yaml` file. +The filename must be lowercase, with `.yaml` extension (not `.yml`). diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index 0d2b68e154d..eb62a733d36 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -29,14 +29,40 @@ For more details about the agent's purpose and architecture, see the [architectu ## Workflows -You can choose from two primary workflows. +You can choose from two primary workflows. The GitOps workflow is recommended. -In a [**GitOps** workflow](gitops.md), you keep your Kubernetes manifests in GitLab. You install a GitLab agent in your cluster, and -any time you update your manifests, the agent updates the cluster. This workflow is fully driven with Git and is considered pull-based, +### GitOps workflow + +In a [**GitOps** workflow](gitops.md): + +- You keep your Kubernetes manifests in GitLab. +- You install a GitLab agent in your cluster. +- Any time you update your manifests, the agent updates the cluster. +- The cluster automatically cleans up unexpected changes. It uses + [server-side applies](https://kubernetes.io/docs/reference/using-api/server-side-apply/) + to fix any configuration inconsistencies that third parties introduce. + +This workflow is fully driven with Git and is considered **pull-based**, because the cluster is pulling updates from your GitLab repository. -In a [**CI/CD** workflow](ci_cd_workflow.md), you use GitLab CI/CD to query and update your cluster by using the Kubernetes API. -This workflow is considered push-based, because GitLab is pushing requests from GitLab CI/CD to your cluster. +GitLab recommends this workflow. We are actively investing in this workflow +so we can provide a first-class experience. + +### GitLab CI/CD workflow + +In a [**CI/CD** workflow](ci_cd_workflow.md): + +- You configure GitLab CI/CD to use the Kubernetes API to query and update your cluster. + +This workflow is considered **push-based**, because GitLab is pushing requests +from GitLab CI/CD to your cluster. + +Use this workflow: + +- When you have a heavily pipeline-oriented processes. +- When you need to migrate to the agent but the GitOps workflow cannot support the use case you need. + +This workflow has a weaker security model and is not recommended for production deployments. ## Supported cluster versions diff --git a/doc/user/clusters/agent/install/index.md b/doc/user/clusters/agent/install/index.md index 4b0d8b77493..0240fbb45f0 100644 --- a/doc/user/clusters/agent/install/index.md +++ b/doc/user/clusters/agent/install/index.md @@ -40,7 +40,7 @@ To install the agent in your cluster: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7, the agent configuration file can be added to multiple directories (or subdirectories) of the repository. > - Group authorization was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. -The agent uses a YAML file for configuration settings. You must create this file if: +For configuration settings, the agent uses a YAML file in the GitLab project. You must create this file if: - You use [a GitOps workflow](../gitops.md#gitops-workflow-steps). - You use [a GitLab CI/CD workflow](../ci_cd_workflow.md#gitlab-cicd-workflow-steps) and want to authorize a different project to use the agent. @@ -56,7 +56,7 @@ To create an agent configuration file: - Start with an alphanumeric character. - End with an alphanumeric character. -1. In the repository, create a directory in this location: +1. In the repository, in the default branch, create this directory at the root: ```plaintext .gitlab/agents/<agent-name> @@ -81,7 +81,7 @@ Prerequisites: You must register an agent before you can install the agent in your cluster. To register an agent: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. If you have an [agent configuration file](#create-an-agent-configuration-file), it must be in this project. Your cluster manifest files should also be in this project. 1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. diff --git a/doc/user/clusters/agent/vulnerabilities.md b/doc/user/clusters/agent/vulnerabilities.md index 5afe3ccec2b..2d20675b68b 100644 --- a/doc/user/clusters/agent/vulnerabilities.md +++ b/doc/user/clusters/agent/vulnerabilities.md @@ -6,7 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Operational Container Scanning **(ULTIMATE)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6346) in GitLab 14.8. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6346) in GitLab 14.8. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/368828) the starboard directive in GitLab 15.4. The starboard directive will be removed in GitLab 16.0. To view cluster vulnerabilities, you can view the [vulnerability report](../../application_security/vulnerabilities/index.md). You can also configure your agent so the vulnerabilities are displayed with other agent information in GitLab. @@ -19,12 +20,12 @@ to scan container images in your cluster for security vulnerabilities. NOTE: In GitLab 15.0 and later, you do not need to install Starboard operator in the Kubernetes cluster. -To begin scanning all resources in your cluster, add a `starboard` +To begin scanning all resources in your cluster, add a `container_scanning` configuration block to your agent configuration with a `cadence` field containing a CRON expression for when the scans will be run. ```yaml -starboard: +container_scanning: cadence: '0 0 * * *' # Daily at 00:00 (Kubernetes cluster time) ``` @@ -42,7 +43,7 @@ if you would like to scan only the `development`, `staging`, and `production` namespaces, you can use this configuration: ```yaml -starboard: +container_scanning: cadence: '0 0 * * *' vulnerability_report: namespaces: @@ -59,7 +60,7 @@ Prerequisite: To view vulnerability information in GitLab: -1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file. +1. On the top bar, select **Main menu > Projects** and find the project that contains the agent configuration file. 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. 1. Select the **Agent** tab. 1. Select an agent to view the cluster vulnerabilities. diff --git a/doc/user/clusters/agent/work_with_agent.md b/doc/user/clusters/agent/work_with_agent.md index 058243ec218..b28f7546379 100644 --- a/doc/user/clusters/agent/work_with_agent.md +++ b/doc/user/clusters/agent/work_with_agent.md @@ -18,7 +18,7 @@ Prerequisite: To view the list of agents: -1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. +1. On the top bar, select **Main menu > Projects** and find the project that contains your agent configuration file. 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. 1. Select **Agent** tab to view clusters connected to GitLab through the agent. @@ -37,7 +37,7 @@ The activity logs help you to identify problems and get the information you need for troubleshooting. You can see events from a week before the current date. To view an agent's activity: -1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. +1. On the top bar, select **Main menu > Projects** and find the project that contains your agent configuration file. 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. 1. Select the agent you want to see activity for. @@ -59,7 +59,6 @@ To debug the cluster-side component (`agentk`) of the agent, set the log level according to the available options: - `error` -- `warning` - `info` - `debug` @@ -95,7 +94,7 @@ For more information about debugging, see [troubleshooting documentation](troubl To reset the agent token without downtime: 1. Create a new token: - 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. 1. Select the agent you want to create a token for. 1. On the **Access tokens** tab, select **Create token**. @@ -117,7 +116,7 @@ clean up those resources manually. To remove an agent from the UI: -1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file. +1. On the top bar, select **Main menu > Projects** and find the project that contains the agent configuration file. 1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. 1. In the table, in the row for your agent, in the **Options** column, select the vertical ellipsis (**{ellipsis_v}**). 1. Select **Delete agent**. diff --git a/doc/user/clusters/environments.md b/doc/user/clusters/environments.md index cf71729b517..96f41531576 100644 --- a/doc/user/clusters/environments.md +++ b/doc/user/clusters/environments.md @@ -33,7 +33,7 @@ With cluster environments, you can gain insight into:  -Access to cluster environments is restricted to +Access to cluster environments is restricted to [group maintainers and owners](../permissions.md#group-members-permissions) ## Usage diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index 361276194b0..62f70faa630 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -61,7 +61,7 @@ To associate a cluster management project with your cluster: **Infrastructure > Kubernetes clusters** page. - [Group-level cluster](../group/clusters/index.md), go to your group's **Kubernetes** page. - - [Instance-level cluster](../instance/clusters/index.md), on the top bar, select **Menu > Admin > Kubernetes**. + - [Instance-level cluster](../instance/clusters/index.md), on the top bar, select **Main menu > Admin > Kubernetes**. 1. Expand **Advanced settings**. 1. From the **Cluster management project** dropdown, select the cluster management project you created in the previous step. diff --git a/doc/user/clusters/management_project_template.md b/doc/user/clusters/management_project_template.md index 4b00784a7ae..cd71be321cc 100644 --- a/doc/user/clusters/management_project_template.md +++ b/doc/user/clusters/management_project_template.md @@ -45,7 +45,8 @@ If you have already configured the agent and connected a cluster with GitLab: To create a project from the cluster management project template: -1. On the top bar, select **Menu > Projects > Create new project**. +1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Create from template**. 1. From the list of templates, next to **GitLab Cluster Management**, select **Use template**. 1. Enter the project details. @@ -80,7 +81,7 @@ If you remove everything below this comment, the pipeline will succeed. ### The main `helmfile.yml` file -The template contains a [Helmfile](https://github.com/roboll/helmfile) you can use to manage +The template contains a [Helmfile](https://github.com/helmfile/helmfile) you can use to manage cluster applications with [Helm v3](https://helm.sh/). This file has a list of paths to other Helm files for each app. They're all commented out by default, so you must uncomment @@ -89,7 +90,7 @@ the paths for the apps that you would like to use in your cluster. By default, each `helmfile.yaml` in these sub-paths has the attribute `installed: true`. This means that every time the pipeline runs, Helmfile tries to either install or update your apps according to the current state of your cluster and Helm releases. If you change this attribute to `installed: false`, Helmfile tries try to uninstall this app -from your cluster. [Read more](https://github.com/roboll/helmfile) about how Helmfile works. +from your cluster. [Read more](https://helmfile.readthedocs.io/en/latest/) about how Helmfile works. ### Built-in applications diff --git a/doc/user/clusters/migrating_from_gma_to_project_template.md b/doc/user/clusters/migrating_from_gma_to_project_template.md index 9a59d135fa0..ce39e13d928 100644 --- a/doc/user/clusters/migrating_from_gma_to_project_template.md +++ b/doc/user/clusters/migrating_from_gma_to_project_template.md @@ -4,7 +4,7 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Migrate from GitLab Managed Apps to Cluster Management Projects **(FREE)** +# Migrate from GitLab Managed Apps to Cluster Management Projects (DEPRECATED) **(FREE)** The GitLab Managed Apps were deprecated in GitLab 14.0 in favor of user-controlled Cluster Management projects. diff --git a/doc/user/compliance/compliance_report/index.md b/doc/user/compliance/compliance_report/index.md index 4cd2705e917..96cb3f3ef38 100644 --- a/doc/user/compliance/compliance_report/index.md +++ b/doc/user/compliance/compliance_report/index.md @@ -1,6 +1,6 @@ --- type: reference, howto -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -31,7 +31,7 @@ Prerequisites: To view the compliance report: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Compliance report**. ### Severity levels scale @@ -105,7 +105,7 @@ Depending on the merge strategy, the merge commit SHA can be a merge commit, squ To generate the Chain of Custody report: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Compliance report**. 1. Select **List of all merge commits**. @@ -122,7 +122,7 @@ Authenticated group owners can generate a commit-specific Chain of Custody repor - Using the GitLab UI: - 1. On the top bar, select **Menu > Groups** and find your group. + 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Compliance report**. 1. At the top of the compliance report, to the right of **List of all merge commits**, select the down arrow (**{chevron-lg-down}**). 1. Enter the merge commit SHA, and then select **Export commit custody report**. diff --git a/doc/user/compliance/index.md b/doc/user/compliance/index.md index 7b46886e236..c6c4834228b 100644 --- a/doc/user/compliance/index.md +++ b/doc/user/compliance/index.md @@ -1,6 +1,6 @@ --- type: reference -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/user/compliance/license_compliance/img/denied_licenses_v13_3.png b/doc/user/compliance/license_compliance/img/denied_licenses_v13_3.png Binary files differdeleted file mode 100644 index aa3deb0c154..00000000000 --- a/doc/user/compliance/license_compliance/img/denied_licenses_v13_3.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/denied_licenses_v15_3.png b/doc/user/compliance/license_compliance/img/denied_licenses_v15_3.png Binary files differnew file mode 100644 index 00000000000..4ed84047133 --- /dev/null +++ b/doc/user/compliance/license_compliance/img/denied_licenses_v15_3.png diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index 1c9f9e85ab7..19b01e4d854 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -736,7 +736,9 @@ Note, the merge request is not able to be merged until the `denied` license is r You may add a [`License-Check` approval rule](#enabling-license-approvals-within-a-project), which enables a designated approver that can approve and then merge a merge request with `denied` license. - +These policies can be configured by using the [Managed Licenses API](../../../api/managed_licenses.md). + + The **Policies** tab in the project's license compliance section displays your project's license policies. Project maintainers can specify policies in this section. @@ -763,7 +765,7 @@ license. You can enable `License-Check` one of two ways: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Merge request approvals**. 1. Select **Enable** or **Edit**. @@ -793,6 +795,18 @@ We recommend that you use the most recent version of all containers, and the mos ## Troubleshooting +### The License Compliance widget is stuck in a loading state + +A loading spinner is displayed in the following scenarios: + +- While the pipeline is in progress. +- If the pipeline is complete, but still parsing the results in the background. +- If the license scanning job is complete, but the pipeline is still running. + +The License Compliance widget polls every few seconds for updated results. When the pipeline is complete, the first poll after pipeline completion triggers the parsing of the results. This can take a few seconds depending on the size of the generated report. + +The final state is when a successful pipeline run has been completed, parsed, and the licenses displayed in the widget. + ### ASDF_PYTHON_VERSION does not automatically install the version Defining a non-latest Python version in ASDF_PYTHON_VERSION [doesn't have it automatically installed](https://gitlab.com/gitlab-org/gitlab/-/issues/325604). If your project requires a non-latest version of Python: @@ -888,3 +902,17 @@ root@6abb70e9f193:~# NOTE: Selecting a custom version of [Mono](https://www.mono-project.com/) or [.NET Core](https://dotnet.microsoft.com/download/dotnet) is currently not supported. + +### LicenseFinder::Maven: is not installed error + +If your project contains a `mvnw` or `mvnw.cmd` file, then the license scanning job may fail with the `LicenseFinder::Maven: is not installed error` error. To resolve this, modify the license scanning job to remove the files in the `before_script` section. Example: + +```yaml +include: + - template: License-Scanning.gitlab-ci.yml + +license_scanning: + before_script: + - rm mvnw + - rm mvnw.cmd +``` diff --git a/doc/user/crm/index.md b/doc/user/crm/index.md index e71a983ccfd..79f18e3abf1 100644 --- a/doc/user/crm/index.md +++ b/doc/user/crm/index.md @@ -1,6 +1,6 @@ --- stage: Plan -group: Product Planning +group: Certify info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -36,7 +36,7 @@ you must enable CRM features for the subgroup. To enable customer relations management in a group or subgroup: -1. On the top bar, select **Menu > Groups** and find your group or subgroup. +1. On the top bar, select **Main menu > Groups** and find your group or subgroup. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Permissions and group features** section. 1. Select **Customer relations is enabled**. @@ -48,7 +48,7 @@ To enable customer relations management in a group or subgroup: To view a group's contacts: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Customer relations > Contacts**.  @@ -57,7 +57,7 @@ To view a group's contacts: To create a contact: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Customer relations > Contacts**. 1. Select **New contact**. 1. Complete all required fields. @@ -70,7 +70,7 @@ contacts using the GraphQL API. To edit an existing contact: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Customer relations > Contacts**. 1. Next to the contact you wish to edit, select **Edit** (**{pencil}**). 1. Edit the required fields. @@ -85,7 +85,7 @@ contacts using the GraphQL API. To view a group's organizations: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Customer relations > Organizations**.  @@ -94,7 +94,7 @@ To view a group's organizations: To create an organization: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Customer relations > Organizations**. 1. Select **New organization**. 1. Complete all required fields. @@ -107,7 +107,7 @@ organizations using the GraphQL API. To edit an existing organization: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Customer relations > Organizations**. 1. Next to the organization you wish to edit, select **Edit** (**{pencil}**). 1. Edit the required fields. @@ -125,7 +125,7 @@ issues are linked to contacts matching the email addresses in the sender and CC To view a contact's issues, select a contact from the issue sidebar, or: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Customer relations > Contacts**. 1. Next to the contact whose issues you wish to view, select **View issues** (**{issues}**). @@ -133,7 +133,7 @@ To view a contact's issues, select a contact from the issue sidebar, or: To view an organization's issues: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Customer relations > Organizations**. 1. Next to the organization whose issues you wish to view, select **View issues** (**{issues}**). diff --git a/doc/user/discussions/img/create-new-issue_v15.png b/doc/user/discussions/img/create-new-issue_v15.png Binary files differdeleted file mode 100644 index 779196b6ba4..00000000000 --- a/doc/user/discussions/img/create-new-issue_v15.png +++ /dev/null diff --git a/doc/user/discussions/img/create_new_issue_v15_4.png b/doc/user/discussions/img/create_new_issue_v15_4.png Binary files differnew file mode 100644 index 00000000000..3720b601cc5 --- /dev/null +++ b/doc/user/discussions/img/create_new_issue_v15_4.png diff --git a/doc/user/discussions/img/unresolved_threads_v15.png b/doc/user/discussions/img/unresolved_threads_v15.png Binary files differdeleted file mode 100644 index 113af20effc..00000000000 --- a/doc/user/discussions/img/unresolved_threads_v15.png +++ /dev/null diff --git a/doc/user/discussions/img/unresolved_threads_v15_4.png b/doc/user/discussions/img/unresolved_threads_v15_4.png Binary files differnew file mode 100644 index 00000000000..1d1669de0f1 --- /dev/null +++ b/doc/user/discussions/img/unresolved_threads_v15_4.png diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 3fb0be6480c..ed0bcec9f49 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -91,7 +91,7 @@ For example, `28719b171a056960dfdc0012b625d0b47b123196` becomes You can add comments and threads to a particular commit. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Commits**. 1. Below the commits, in the **Comment** field, enter a comment. 1. Select **Comment** or select the down arrow (**{chevron-down}**) to select **Start thread**. @@ -309,15 +309,15 @@ To resolve a thread: At the top of the page, the number of unresolved threads is updated: - + ### Move all unresolved threads in a merge request to an issue If you have multiple unresolved threads in a merge request, you can create an issue to resolve them separately. In the merge request, at the top of the page, -select **Create issue to resolve all threads** (**{issue-new}**): +click the ellipsis icon button (**{ellipsis_v}**) in the threads control and then select **Create issue to resolve all threads**: - + All threads are marked as resolved, and a link is added from the merge request to the newly created issue. @@ -339,10 +339,9 @@ You can prevent merge requests from being merged until all threads are resolved. When this setting is enabled, the **Unresolved threads** counter in a merge request is shown in orange when at least one thread remains unresolved. -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. -1. Under **Merge checks**, select the **All threads must be resolved** checkbox. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge checks** section, select the **All threads must be resolved** checkbox. 1. Select **Save changes**. ### Automatically resolve threads in a merge request when they become outdated @@ -350,11 +349,10 @@ is shown in orange when at least one thread remains unresolved. You can set merge requests to automatically resolve threads when lines are modified with a new push. -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. -1. Under **Merge options**, select the - **Automatically resolve merge request diff threads when they become outdated** checkbox. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge options** section, select + **Automatically resolve merge request diff threads when they become outdated**. 1. Select **Save changes**. Threads are now resolved if a push makes a diff section outdated. diff --git a/doc/user/free_user_limit.md b/doc/user/free_user_limit.md index a62b9c9e363..60091449256 100644 --- a/doc/user/free_user_limit.md +++ b/doc/user/free_user_limit.md @@ -8,10 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w From October 19, 2022, namespaces in GitLab.com on the Free tier will be limited to five (5) members per [namespace](namespace/index.md). -This limit applies to top-level groups and personal namespaces. - -In a personal namespace, the limit applies across all projects in your personal -namespace. +This limit applies to top-level private groups. On the transition date, if your namespace has six or more unique members: @@ -40,17 +37,11 @@ Prerequisite: - You must have the Owner role for the group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups > View all groups** and find your group. 1. On the left sidebar, select **Settings > Usage Quotas**. 1. To view all members, select the **Seats** tab. 1. To remove a member, select **Remove user**. -NOTE: -The **Usage Quotas** page is not available for personal namespaces. You can -view and [remove members](project/members/index.md#remove-a-member-from-a-project) -in each project instead. The five user limit includes all -unique members across all projects in your personal namespace. - If you need more time to manage your members, or to try GitLab features with a team of more than five members, you can [start a trial](https://about.gitlab.com/free-trial/). A trial lasts for 30 days and includes an unlimited number of members. diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 53e459f7a09..62b685ea4f4 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -208,9 +208,9 @@ from those IPs and allow them. GitLab.com is fronted by Cloudflare. For incoming connections to GitLab.com, you might need to allow CIDR blocks of Cloudflare ([IPv4](https://www.cloudflare.com/ips-v4/) and [IPv6](https://www.cloudflare.com/ips-v6/)). -For outgoing connections from CI/CD runners, we are not providing static IP -addresses. All GitLab.com shared runners are deployed into Google Cloud Platform (GCP). Any -IP-based firewall can be configured by looking up all +For outgoing connections from CI/CD runners, we are not providing static IP addresses. +All GitLab.com shared runners are deployed into Google Cloud Platform (GCP) in `us-east1`. +Any IP-based firewall can be configured by looking up [IP address ranges or CIDR blocks for GCP](https://cloud.google.com/compute/docs/faq#find_ip_range). ## Hostname list @@ -328,6 +328,12 @@ for `shared_buffers` is quite high, and we are GitLab.com uses the default of 60 seconds for [Puma request timeouts](../../administration/operations/puma.md#change-the-worker-timeout). +## Merge request reviewer maximum + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91406) in GitLab 15.3. + +A maximum of 100 reviewers can be assigned to a merge request. + ## GitLab.com-specific rate limits NOTE: @@ -336,7 +342,7 @@ documentation. When a request is rate limited, GitLab responds with a `429` status code. The client should wait before attempting the request again. There -are also informational headers with this response detailed in +are also informational headers with this response detailed in [rate limiting responses](#rate-limiting-responses). The following table describes the rate limits for GitLab.com, both before and @@ -358,8 +364,8 @@ after the limits change in January, 2021: | **Pipeline creation** requests (for a given **project, user, and commit**) | | **25** requests per minute | | **Alert integration endpoint** requests (for a given **project**) | | **3600** requests per hour | -More details are available on the rate limits for -[protected paths](#protected-paths-throttle) and +More details are available on the rate limits for +[protected paths](#protected-paths-throttle) and [raw endpoints](../../user/admin_area/settings/rate_limits_on_raw_endpoints.md). GitLab can rate-limit requests at several layers. The rate limits listed here diff --git a/doc/user/group/access_and_permissions.md b/doc/user/group/access_and_permissions.md index c469d6c2f6d..bdef13af3f9 100644 --- a/doc/user/group/access_and_permissions.md +++ b/doc/user/group/access_and_permissions.md @@ -12,13 +12,21 @@ Configure your groups to control group permissions and access. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34370) in GitLab 12.8. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/224129) in GitLab 13.4. +> - [Moved to Settings/Repository](https://gitlab.com/gitlab-org/gitlab/-/issues/220365) in GitLab 15.4. Group push rules allow group maintainers to set [push rules](../project/repository/push_rules.md) for newly created projects in the specific group. -To configure push rules for a group: +In GitLab 15.4 and later, to configure push rules for a group: -1. Go to the groups's **Push Rules** page. +1. On the left sidebar, select **Push rules**. +1. Select the settings you want. +1. Select **Save Push Rules**. + +In GitLab 15.3 and earlier, to configure push rules for a group: + +1. On the left sidebar, select **Settings > Repository** page. +1. Expand the **Pre-defined push rules** section. 1. Select the settings you want. 1. Select **Save Push Rules**. @@ -27,6 +35,27 @@ The group's new subgroups have push rules set for them based on either: - The closest parent group with push rules defined. - Push rules set at the instance level, if no parent groups have push rules defined. +## Restrict Git access protocols + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/365601) in GitLab 15.1 [with a flag](../../administration/feature_flags.md) named `group_level_git_protocol_control`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to +[enable the feature flag](../../administration/feature_flags.md) named `group_level_git_protocol_control`. On GitLab.com, +this feature is available. + +You can set the permitted protocols used to access a group's repositories to either SSH, HTTPS, or both. This setting +is disabled when the [instance setting](../admin_area/settings/visibility_and_access_controls.md#configure-enabled-git-access-protocols) is +configured by an administrator. + +To change the permitted Git access protocols for a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Expand the **Permissions and group features** section. +1. Choose the permitted protocols from **Enabled Git access protocols**. +1. Select **Save changes**. + ## Restrict group access by IP address **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1985) in GitLab 12.0. @@ -43,8 +72,6 @@ applies to: You should consider some security implications before configuring IP address restrictions. -- Restricting HTTP traffic on GitLab.com with IP address restrictions causes SSH requests (including Git operations over - SSH) to fail. For more information, see [the relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/271673). - Administrators and group owners can access group settings from any IP address, regardless of IP restriction. However: - Groups owners cannot access projects belonging to the group when accessing from a disallowed IP address. - Administrators can access projects belonging to the group when accessing from a disallowed IP address. @@ -57,14 +84,17 @@ You should consider some security implications before configuring IP address res restricted IP address, the IP restriction prevents code from being cloned. - Users may still see some events from the IP restricted groups and projects on their dashboard. Activity may include push, merge, issue, or comment events. +- IP access restrictions for Git operations via SSH are supported only on GitLab SaaS. + IP access restrictions applied to self-managed instances block SSH completely. ### Restrict group access by IP address To restrict group access by IP address: -1. Go to the group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. 1. Expand the **Permissions and group features** section. -1. In the **Allow access to the following IP addresses** field, enter IPv4 or IPv6 address ranges in CIDR notation. +1. In the **Restrict access by IP address** field, enter IPv4 or IPv6 address ranges in CIDR notation. 1. Select **Save changes**. In self-managed installations of GitLab 15.1 and later, you can also configure @@ -81,7 +111,8 @@ You can prevent users with email addresses in specific domains from being added To restrict group access by domain: -1. Go to the group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. 1. Expand the **Permissions and group features** section. 1. In the **Restrict membership by email** field, enter the domain names. 1. Select **Save changes**. @@ -124,23 +155,24 @@ If you prevent group sharing outside the hierarchy for the **Animals** group: To prevent sharing outside of the group's hierarchy: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. -1. Select **Prevent members from sending invitations to groups outside of `<group_name>` and its subgroups**. +1. Select **Members cannot invite groups outside of `<group_name>` and its subgroups**. 1. Select **Save changes**. ## Prevent a project from being shared with groups -Prevent projects in a group from -[sharing a project with another group](../project/members/share_project_with_groups.md) +Prevent projects in a group from +[sharing a project with another group](../project/members/share_project_with_groups.md) to enable tighter control over project access. To prevent a project from being shared with other groups: -1. Go to the group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. 1. Expand the **Permissions and group features** section. -1. Select **Prevent sharing a project in `<group_name>` with other groups**. +1. Select **Projects in `<group_name>` cannot be shared with other groups**. 1. Select **Save changes**. This setting applies to all subgroups unless overridden by a group owner. Groups already @@ -151,7 +183,7 @@ added to a project lose access when the setting is enabled. As a group owner, you can prevent non-members from requesting access to your group. -1. On the top bar, select **Menu > Groups**. +1. On the top bar, **Main menu > Groups** and find your group. 1. Select **Your Groups**. 1. Find the group and select it. 1. From the left menu, select **Settings > General**. @@ -173,7 +205,8 @@ If even one is set to `true`, then the group does not allow outside forks. To prevent projects from being forked outside the group: -1. Go to the top-level group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. 1. Expand the **Permissions and group features** section. 1. Check **Prevent project forking outside current group**. 1. Select **Save changes**. @@ -194,9 +227,9 @@ The setting does not cascade. Projects in subgroups observe the subgroup configu To prevent members from being added to projects in a group: -1. Go to the group's **Settings > General** page. -1. Expand the **Permissions and group features** section. -1. Under **Membership**, select **Prevent adding new members to projects within this group**. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Under **Membership**, select **Users cannot be added to projects in this group**. 1. Select **Save changes**. All users who previously had permissions can no longer add members to a group. @@ -241,7 +274,8 @@ To create group links via filter: LDAP user permissions can be manually overridden by an administrator. To override a user's permissions: -1. Go to your group's **Group information > Members** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Group information > Members**. 1. In the row for the user you are editing, select the pencil (**{pencil}**) icon. 1. Select **Edit permissions** in the modal. diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index 0da7eaa4d55..17a5551adbf 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -21,7 +21,7 @@ your group, enabling you to use the same cluster across multiple projects. To view your group-level Kubernetes clusters: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Kubernetes**. ## Cluster management project @@ -89,7 +89,7 @@ your cluster, which can cause deployment jobs to fail. To clear the cache: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Kubernetes**. 1. Select your cluster. 1. Expand **Advanced settings**. diff --git a/doc/user/group/contribution_analytics/index.md b/doc/user/group/contribution_analytics/index.md index ba805d6e1bb..7750782a623 100644 --- a/doc/user/group/contribution_analytics/index.md +++ b/doc/user/group/contribution_analytics/index.md @@ -19,7 +19,7 @@ group. To view Contribution Analytics: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Contribution**. ## Using Contribution Analytics diff --git a/doc/user/group/devops_adoption/index.md b/doc/user/group/devops_adoption/index.md index 333bef84dcc..bb18c69f7ae 100644 --- a/doc/user/group/devops_adoption/index.md +++ b/doc/user/group/devops_adoption/index.md @@ -36,7 +36,7 @@ Prerequisite: To view DevOps Adoption: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > DevOps adoption** ## DevOps Adoption categories @@ -80,7 +80,7 @@ twelve months. The chart only shows data from when you enabled DevOps Adoption f To view feature adoption over time: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > DevOps adoption**. 1. Select the **Overview** tab. diff --git a/doc/user/group/epics/epic_boards.md b/doc/user/group/epics/epic_boards.md index f03a56d8a00..a8bbb0575b3 100644 --- a/doc/user/group/epics/epic_boards.md +++ b/doc/user/group/epics/epic_boards.md @@ -15,7 +15,7 @@ labels. To view an epic board: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Epics > Boards**.  @@ -28,7 +28,7 @@ Prerequisites: To create a new epic board: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Epics > Boards**. 1. In the upper left corner, select the dropdown with the current board name. 1. Select **Create new board**. @@ -77,7 +77,7 @@ Prerequisites: To create a new list: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Epics > Boards**. 1. In the upper-right corner, select **Create list**. 1. In the **New list** column expand the **Select a label** dropdown and select the label to use as @@ -155,6 +155,42 @@ into another list. Learn about possible effects in [Dragging epics between lists To move a list, select its top bar, and drag it horizontally. You can't move the **Open** and **Closed** lists, but you can hide them when editing an epic board. +#### Move an epic to the start of the list + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367473) in GitLab 15.4. + +When you have many epics, it's inconvenient to manually drag an epic from the bottom of a board list all +the way to the top. You can move epics to the top of the list with a menu shortcut. + +Your epic is moved to the top of the list even if other epics are hidden by a filter. + +Prerequisites: + +- You must at least have the Reporter role for a group. + +To move an epic to the start of the list: + +1. In an epic board, hover over the card of the epic you want to move. +1. Select the vertical ellipsis (**{ellipsis_v}**), then **Move to start of list**. + +#### Move an epic to the end of the list + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367473) in GitLab 15.4. + +When you have many epics, it's inconvenient to manually drag an epic from the top of a board list all +the way to the bottom. You can move epics to the bottom of the list with a menu shortcut. + +Your epic is moved to the bottom of the list even if other epics are hidden by a filter. + +Prerequisites: + +- You must at least have the Reporter role for a group. + +To move an epic to the end of the list: + +1. In an epic board, hover over the card of the epic you want to move. +1. Select the vertical ellipsis (**{ellipsis_v}**), then **Move to end of list**. + #### Dragging epics between lists When you drag epics between lists, the result is different depending on the source list diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index 71d7b7fbb0c..26826ec5832 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -210,7 +210,7 @@ Prerequisites: To view epics in a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Epics**. ### Cached epic count @@ -244,7 +244,7 @@ You can filter the list of epics by: To filter: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Epics**. 1. Select the field **Search or filter results**. 1. From the dropdown menu, select the scope or enter plain text to search by epic title or description. diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index edf4d7677df..21b389581ae 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -88,6 +88,7 @@ migrated: - Board Lists - Boards - Epics ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250281) in 13.7) + - Epic resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - Finisher - Group Labels ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292429) in 13.9) - Iterations ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292428) in 13.10) @@ -116,10 +117,17 @@ On self-managed GitLab, migrating project resources are not available by default - CI Pipelines ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339407) in GitLab 14.6) - Designs ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339421) in GitLab 15.1) - Issues ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267946) in GitLab 14.4) + - Issue resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - Labels ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339419) in GitLab 14.4) - LFS Objects ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339405) in GitLab 14.8) - Members ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/341886) in GitLab 14.8) - Merge Requests ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339403) in GitLab 14.5) + - Multiple merge request assignees ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request reviewers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request approvers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Merge request resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - Migrate Push Rules ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339403) in GitLab 14.6) - Pull Requests (including external pull requests) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339409) in GitLab 14.5) - Pipeline History ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339412) in GitLab 14.6) @@ -167,3 +175,17 @@ import = BulkImports::Entity.where(namespace_id: Group.id).map(&:bulk_import) import.status #=> 3 means that the import timed out. ``` + +### Error: `404 Group Not Found` + +If you attempt to import a group that has a path comprised of only numbers (for example, `5000`), GitLab attempts to find the group by ID instead of the +path. This causes a `404 Group Not Found` error. To solve this, the source group path must be changed to include a non-numerical character using either: + +- The GitLab UI: + + 1. On the top bar, select **Main menu > Groups** and find your group. + 1. On the left sidebar, select **Settings > General**. + 1. Expand **Advanced**. + 1. Under **Change group URL**, change the group URL to include non-numeric characters. + +- The [Groups API](../../../api/groups.md#update-group). diff --git a/doc/user/group/insights/index.md b/doc/user/group/insights/index.md index b3fdeb0ab41..0cde0f1f133 100644 --- a/doc/user/group/insights/index.md +++ b/doc/user/group/insights/index.md @@ -1,51 +1,48 @@ --- -type: reference, howto stage: Manage group: Optimize info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Insights **(ULTIMATE)** +# Insights for groups **(ULTIMATE)** > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/725) in GitLab 12.0. -Configure the Insights that matter for your groups. Explore data such as -triage hygiene, issues created or closed for a given period, average time for merge -requests to be merged, and much more. - - +Configure Insights to explore data about you group's activity, such as +triage hygiene, issues created or closed in a given period, and average time for merge +requests to be merged. ## View your group's Insights +Prerequisites: + +- You must have [permission](../../permissions.md#group-members-permissions) to view the group. +- You must have access to a project to view information about its merge requests and issues, + and permission to view them if they are confidential. + To access your group's Insights: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Insights**. + + ## Configure your Insights -GitLab reads Insights from the [default configuration file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/fixtures/insights/default.yml). -If you want to customize it: +GitLab reads Insights from the +[default configuration file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/fixtures/insights/default.yml). +You can also create custom Insights charts that are more relevant for your group. -1. Create a new file [`.gitlab/insights.yml`](../../project/insights/index.md) +To customize your Insights: + +1. Create a new file [`.gitlab/insights.yml`](../../project/insights/index.md#writing-your-gitlabinsightsyml) in a project that belongs to your group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Insights**. 1. Select the project that contains your `.gitlab/insights.yml` configuration file. 1. Select **Save changes**. -## Permissions - -If you have access to view a group, then you have access to view its Insights. - -NOTE: -Issues or merge requests that you don't have access to (because you don't have -access to the project they belong to, or because they are confidential) are -filtered out of the Insights charts. - -You may also consult the [group permissions table](../../permissions.md#group-members-permissions). - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/group/issues_analytics/index.md b/doc/user/group/issues_analytics/index.md index 62337dabcc0..26d77edd89b 100644 --- a/doc/user/group/issues_analytics/index.md +++ b/doc/user/group/issues_analytics/index.md @@ -15,7 +15,7 @@ prior. To access the chart: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Issue Analytics**. Hover over each bar to see the total number of issues. diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md index 530635802a6..4cc879ef32d 100644 --- a/doc/user/group/iterations/index.md +++ b/doc/user/group/iterations/index.md @@ -12,11 +12,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Moved to GitLab Premium in 13.9. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/221047) in GitLab 14.6. [Feature flag `group_iterations`](https://gitlab.com/gitlab-org/gitlab/-/issues/221047) removed. -WARNING: -After [Iteration Cadences](#iteration-cadences) becomes generally available, -manual iteration scheduling will be [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/356069) in GitLab 15.6. -To enhance the role of iterations as time boundaries, we will also deprecate the title field. - Iterations are a way to track issues over a period of time. This allows teams to track velocity and volatility metrics. Iterations can be used with [milestones](../../project/milestones/index.md) for tracking over different time periods. @@ -37,7 +32,7 @@ In GitLab, iterations are similar to milestones, with a few differences: To view the iterations list: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Select **Issues > Iterations**. To view all the iterations in a cadence, ordered by descending date, select that iteration cadence. @@ -66,7 +61,7 @@ Prerequisites: To create an iteration: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Issues > Iterations**. 1. Select **New iteration**. 1. Enter the title, a description (optional), a start date, and a due date. @@ -174,7 +169,7 @@ and get a more accurate understanding of scope attributable to each label. To group issues by label: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Issues > Iterations**. 1. In the **Group by** dropdown, select **Label**. 1. Select the **Filter by label** dropdown. @@ -187,7 +182,7 @@ To group issues by label: > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5077) in GitLab 14.1 [with a flag](../../../administration/feature_flags.md), named `iteration_cadences`. Disabled by default. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/354977) in GitLab 15.0: All scheduled iterations must start on the same day of the week as the cadence start day. Start date of cadence cannot be edited after the first iteration starts. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/354878) in GitLab 15.0. -> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/367493) in GitLab 15.3: A new automation start date can be selected for cadence. Upcoming iterations will be scheduled to start on the same day of the week as the changed start date. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/367493) in GitLab 15.4: A new automation start date can be selected for cadence. Upcoming iterations will be scheduled to start on the same day of the week as the changed start date. Iteration cadences can be manually managed by turning off the automatic scheduling feature. Iteration cadences automate iteration scheduling. You can use them to automate creating iterations every 1, 2, 3, or 4 weeks. You can also @@ -203,11 +198,12 @@ Prerequisites: To create an iteration cadence: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Issues > Iterations**. 1. Select **New iteration cadence**. -1. Complete the fields. - - Enter the title and description of the iteration cadence. +1. Enter the title and description of the iteration cadence. +1. To manually manage the iteration cadence, clear the **Enable automatic scheduling** checkbox and skip the next step. +1. Complete the required fields to use automatic scheduling. - Select the automation start date of the iteration cadence. Iterations will be scheduled to begin on the same day of the week as the day of the week of the start date. - From the **Duration** dropdown list, select how many weeks each iteration should last. @@ -224,11 +220,11 @@ Prerequisites: To edit an iteration cadence: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Issues > Iterations**. 1. Select **Edit iteration cadence**. -When you edit the **Automation start date** field, +When you are using automatic scheduling and edit the **Automation start date** field, you must set a new start date that doesn't overlap with the existing current or past iterations. @@ -236,52 +232,23 @@ Editing **Upcoming iterations** is a non-destructive action. If ten upcoming iterations already exist, changing the number under **Upcoming iterations** to `2` doesn't delete the eight existing upcoming iterations. -### Delete an iteration cadence - -> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. - -Prerequisites: - -- You must have at least the Reporter role for a group. - -Deleting an iteration cadence also deletes all iterations within that cadence. - -To delete an iteration cadence: - -1. On the top bar, select **Menu > Groups** and find your group. -1. On the left sidebar, select **Issues > Iterations**. -1. Select the three-dot menu (**{ellipsis_v}**) > **Delete cadence** for the cadence you want to delete. -1. Select **Delete cadence** in the confirmation modal. - -### Manual iteration cadences - -When you **enable** the iteration cadences feature, all previously -created iterations are added to a default iteration cadence. -You can continue to add, edit, and remove iterations in -this default cadence. - -#### Convert a manual cadence to use automatic scheduling - -WARNING: -The upgrade is irreversible. After it's done, a new manual iteration cadence cannot be created. - -Prerequisites: +#### Turn on automatic scheduling for manual iterations cadence -- You must have created [iterations](#iterations) without cadences before enabling iteration cadences for your group. -To upgrade the iteration cadence to use the automation features: - -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Issues > Iterations**. -1. Select the three-dot menu (**{ellipsis_v}**) > **Edit cadence** for the cadence you want to upgrade. +1. Select the three-dot menu (**{ellipsis_v}**) > **Edit cadence** for the cadence for which you want to enable automatic scheduling. +1. Check the **Enable automatic scheduling** checkbox. 1. Complete the required fields **Duration**, **Upcoming iterations**, and **Automation start date**. For **Automation start date**, you can select any date that doesn't overlap with the existing open iterations. If you have upcoming iterations, the automatic scheduling adjusts them appropriately to fit -your chosen duration. +your chosen duration. 1. Select **Save changes**. -#### Converted cadences example +When you want to manage your iterations cadence manually again, edit your cadence and uncheck the **Enable automatic scheduling** checkbox. + +#### Example of turning on automatic scheduling for manual iterations cadence -For example, suppose it's Friday, April 15, and you have three iterations in a manual cadence: +Suppose it's Friday, April 15, and you have three iteration in a manual iterations cadence: - Monday, April 4 - Friday, April 8 (closed) - Tuesday, April 12 - Friday, April 15 (ongoing) @@ -300,8 +267,25 @@ after the conversion you have the following iterations: - Monday, April 18 - Sunday, April 24 (upcoming) - Monday, April 25 - Sunday, May 1 (upcoming) -Your existing upcoming iteration "Tuesday, April 12 - Friday, April 15" +Your existing upcoming iteration "Tuesday, April 12 - Friday, April 15" is changed to "April 18 - Sunday, April 24". An additional upcoming iteration "April 25 - Sunday, May 1" is scheduled to satisfy the requirement that there are at least two upcoming iterations scheduled. + +### Delete an iteration cadence + +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. + +Prerequisites: + +- You must have at least the Reporter role for a group. + +Deleting an iteration cadence also deletes all iterations within that cadence. + +To delete an iteration cadence: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Issues > Iterations**. +1. Select the three-dot menu (**{ellipsis_v}**) > **Delete cadence** for the cadence you want to delete. +1. Select **Delete cadence**. diff --git a/doc/user/group/manage.md b/doc/user/group/manage.md index a9cc6cc8432..ca9ff0d7b3e 100644 --- a/doc/user/group/manage.md +++ b/doc/user/group/manage.md @@ -10,8 +10,7 @@ Use groups to manage one or more related projects at the same time. ## View groups -1. On the top bar, select **Menu > Groups**. -1. Select **Explore groups**. +To view groups, on the top bar, select **Main menu > Groups > View all groups**. The **Groups** page shows a list of groups, sorted by last updated date. @@ -25,7 +24,7 @@ The **Groups** page shows a list of groups, sorted by last updated date. To create a group: 1. On the top bar, either: - - Select **Menu > Groups**, and on the right, select **Create group**. + - Select **Main menu > Groups > View all groups**, and on the right, select **New group**. - To the left of the search box, select the plus sign and then **New group**. 1. Select **Create group**. 1. Enter a name for the group in **Group name**. For a list of words that cannot be used as group names, see @@ -45,7 +44,7 @@ For details about groups, watch [GitLab Namespaces (users, groups and subgroups) To remove a group and its contents: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Advanced** section. 1. In the **Remove group** section, select **Remove group**. @@ -54,7 +53,7 @@ To remove a group and its contents: A group can also be removed from the groups dashboard: -1. On the top bar, select **Menu > Groups**. +1. On the top bar, select **Main menu > Groups > View all groups**. 1. Select **Your Groups**. 1. Select (**{ellipsis_v}**) for the group you want to delete. 1. Select **Delete**. @@ -83,7 +82,7 @@ Prerequisites: To immediately remove a group marked for deletion: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. In the "Permanently remove group" section, select **Remove group**. @@ -98,7 +97,8 @@ are deleted. To restore a group that is marked for deletion: -1. Go to your group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. Select **Settings > General**. 1. Expand the **Path, transfer, remove** section. 1. In the Restore group section, select **Restore group**. @@ -106,7 +106,7 @@ To restore a group that is marked for deletion: As a user, you can request to be a member of a group, if an administrator allows it. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. Under the group name, select **Request Access**. As many as ten of the most-recently-active group owners receive an email with your request. @@ -127,7 +127,7 @@ To find members in a group, you can sort, filter, or search. Filter a group to find members. By default, all members in the group and subgroups are displayed. -1. Go to the group and select **Group information > Members**. +1. On the top bar, select **Main menu > Groups** and find your group. 1. Above the list of members, in the **Filter members** box, enter filter criteria. - To view members in the group only, select **Membership = Direct**. - To view members of the group and its subgroups, select **Membership = Inherited**. @@ -138,7 +138,8 @@ Filter a group to find members. By default, all members in the group and subgrou You can search for members by name, username, or email. -1. Go to the group and select **Group information > Members**. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Group information > Members**. 1. Above the list of members, in the **Filter members** box, enter search criteria. 1. To the right of the **Filter members** box, select the magnifying glass (**{search}**). @@ -146,7 +147,8 @@ You can search for members by name, username, or email. You can sort members by **Account**, **Access granted**, **Max role**, or **Last sign-in**. -1. Go to the group and select **Group information > Members**. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Group information > Members**. 1. Above the list of members, on the top right, from the **Account** list, select the criteria to filter by. 1. To switch the sort between ascending and descending, to the right of the **Account** list, select the @@ -156,7 +158,7 @@ You can sort members by **Account**, **Access granted**, **Max role**, or **Last You can give a user access to all projects in a group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Group information > Members**. 1. Select **Invite members**. 1. Fill in the fields. @@ -182,11 +184,12 @@ Prerequisites: To remove a member from a group: -1. Go to the group. -1. From the left menu, select **Group information > Members**. -1. Next to the member you want to remove, select **Delete**. -1. Optional. On the **Remove member** confirmation box, select the - **Also unassign this user from linked issues and merge requests** checkbox. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Group information > Members**. +1. Next to the member you want to remove, select **Remove member**. +1. Optional. On the **Remove member** confirmation box: + - To remove direct user membership from subgroups and projects, select the **Also remove direct user membership from subgroups and projects** checkbox. + - To unassign the user from linked issues and merge requests, select the **Also unassign this user from linked issues and merge requests** checkbox. 1. Select **Remove member**. ## Add projects to a group @@ -207,12 +210,12 @@ By default, users with at least the Developer role can create projects under a g To change this setting for a specific group: -1. On the top bar, select **Menu > Groups**. +1. On the top bar, select **Main menu > Groups > View all groups**. 1. Select **Your Groups**. 1. Find the group and select it. 1. From the left menu, select **Settings > General**. 1. Expand the **Permissions and group features** section. -1. Select the desired option in the **Allowed to create projects** dropdown list. +1. Select the desired option in the **Roles allowed to create projects** dropdown list. 1. Select **Save changes**. To change this setting globally, see [Default project creation protection](../admin_area/settings/visibility_and_access_controls.md#define-which-roles-can-create-projects). @@ -223,11 +226,13 @@ You can change the owner of a group. Each group must always have at least one member with the Owner role. - As an administrator: - 1. Go to the group and from the left menu, select **Group information > Members**. + 1. On the top bar, select **Main menu > Groups** and find your group. + 1. On the left sidebar, select **Group information > Members**. 1. Give a different member the **Owner** role. 1. Refresh the page. You can now remove the **Owner** role from the original owner. - As the current group's owner: - 1. Go to the group and from the left menu, select **Group information > Members**. + 1. On the top bar, select **Main menu > Groups** and find your group. + 1. On the left sidebar, select **Group information > Members**. 1. Give a different member the **Owner** role. 1. Have the new owner sign in and remove the **Owner** role from you. @@ -246,7 +251,8 @@ create a new group and transfer projects to it instead. To change your group path (group URL): -1. Go to your group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General** page. 1. Expand the **Advanced** section. 1. Under **Change group URL**, enter a new name. 1. Select **Change group URL**. @@ -326,7 +332,7 @@ When transferring groups, note: To transfer a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Advanced** section. 1. In the **Remove group** section, select **Transfer group**. @@ -342,7 +348,7 @@ To transfer a group: > - [User interface changed](https://gitlab.com/gitlab-org/gitlab/-/issues/352961) in GitLab 15.1. [Delayed project deletion](../project/settings/index.md#delayed-project-deletion) is locked and disabled unless the instance-level settings for -[deletion protection](../admin_area/settings/visibility_and_access_controls.md#deletion-protection) is enabled for either groups only or groups and projects. +[deletion protection](../admin_area/settings/visibility_and_access_controls.md#deletion-protection) are enabled for either groups only or groups and projects. When enabled on groups, projects in the group are deleted after a period of delay. During this period, projects are in a read-only state and can be restored. The default period is seven days but [is configurable at the instance level](../admin_area/settings/visibility_and_access_controls.md#retention-period). @@ -356,7 +362,8 @@ the default setting. To enable delayed deletion of projects in a group: -1. Go to the group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. 1. Expand the **Permissions and group features** section. 1. Scroll to: - (GitLab 15.1 and later) **Deletion protection** and select **Keep deleted projects**. @@ -377,9 +384,10 @@ You can disable all email notifications related to the group, which includes its To disable email notifications: -1. Go to the group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. 1. Expand the **Permissions and group features** section. -1. Select **Disable email notifications**. +1. Select **Email notifications are disabled**. 1. Select **Save changes**. ## Disable group mentions @@ -396,9 +404,10 @@ This is particularly helpful for groups with a large number of users. To disable group mentions: -1. Go to the group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. 1. Expand the **Permissions and group features** section. -1. Select **Disable group mentions**. +1. Select **Group mentions are disabled**. 1. Select **Save changes**. ## Export members as CSV **(PREMIUM)** @@ -408,7 +417,8 @@ To disable group mentions: You can export a list of members in a group or subgroup as a CSV. -1. Go to your group or subgroup and select either **Group information > Members** or **Subgroup information > Members**. +1. On the top bar, select **Main menu > Groups** and find your group or subgroup. +1. On the left sidebar, select either **Group information > Members** or **Subgroup information > Members**. 1. Select **Export as CSV**. 1. After the CSV file has been generated, it is emailed as an attachment to the user that requested it. @@ -434,7 +444,7 @@ Prerequisite: To specify a user cap: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. You can set a cap on the top-level group only. 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. @@ -456,7 +466,7 @@ Prerequisite: To remove the user cap: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. 1. In the **User cap** box, delete the value. @@ -477,7 +487,7 @@ Prerequisite: To approve members that are pending because they've exceeded the user cap: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Usage Quotas**. 1. On the **Seats** tab, under the alert, select **View pending approvals**. 1. For each member you want to approve, select **Approve**. @@ -508,7 +518,8 @@ Define project templates at a group level by setting a group as the template sou To enable group file templates: -1. Go to the group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. 1. Expand the **Templates** section. 1. Choose a project to act as the template repository. 1. Select **Save changes**. @@ -525,7 +536,8 @@ that belong to the group. To view the merge request approval settings for a group: -1. Go to the top-level group's **Settings > General** page. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. 1. Expand the **Merge request approvals** section. 1. Select the settings you want. 1. Select **Save changes**. @@ -548,7 +560,7 @@ Changes to [group wikis](../project/wiki/group.md) do not appear in group activi You can view the most recent actions taken in a group, either in your browser or in an RSS feed: -1. On the top bar, select **Menu > Groups**. +1. On the top bar, select **Main menu > Groups > View all groups**. 1. Select **Your Groups**. 1. Find the group and select it. 1. On the left sidebar, select **Group information > Activity**. @@ -568,3 +580,13 @@ the following checks when creating or updating namespaces or groups: In the unlikely event that you see these errors in your GitLab installation, [contact Support](https://about.gitlab.com/support/) so that we can improve this validation. + +### Find groups using an SQL query + +To find and store an array of groups based on an SQL query in the [rails console](../../administration/operations/rails_console.md): + +```ruby +# Finds groups and subgroups that end with '%oup' +Group.find_by_sql("SELECT * FROM namespaces WHERE name LIKE '%oup'") +=> [#<Group id:3 @test-group>, #<Group id:4 @template-group/template-subgroup>] +``` diff --git a/doc/user/group/repositories_analytics/index.md b/doc/user/group/repositories_analytics/index.md index 6e3ea7d6c0f..f130ecb61dc 100644 --- a/doc/user/group/repositories_analytics/index.md +++ b/doc/user/group/repositories_analytics/index.md @@ -37,7 +37,7 @@ The **Analytics > Repositories** group page displays the average test coverage o To see the latest code coverage for each project in your group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Repositories**. 1. In the **Latest test coverage results** section, from the **Select projects** dropdown list, choose the projects you want to check. @@ -52,7 +52,7 @@ You can get a CSV of the code coverage data for all of the projects in your grou To get the report: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Repositories**. 1. Select **Download historic test coverage data (.csv)**. 1. Select the projects and date range you want to include in the report. diff --git a/doc/user/group/saml_sso/example_saml_config.md b/doc/user/group/saml_sso/example_saml_config.md index 97e8f9c54a3..d09a8524247 100644 --- a/doc/user/group/saml_sso/example_saml_config.md +++ b/doc/user/group/saml_sso/example_saml_config.md @@ -31,20 +31,27 @@ If you are currently having an issue with GitLab, you may want to check your [su ## Azure Active Directory -Basic SAML app configuration: +This section has screenshots for the elements of Azure Active Directory configuration. + +### Basic SAML app configuration  -User claims and attributes: +### User claims and attributes  -SCIM mapping: +### SCIM mapping + +Provisioning:  + +Attribute mapping: +  -Group Sync: +### Group Sync  diff --git a/doc/user/group/saml_sso/group_sync.md b/doc/user/group/saml_sso/group_sync.md index 8bc316f9396..322b417d466 100644 --- a/doc/user/group/saml_sso/group_sync.md +++ b/doc/user/group/saml_sso/group_sync.md @@ -70,9 +70,9 @@ role. Users granted: - A higher role with Group Sync are displayed as having - [direct membership](../../project/members/#display-direct-members) of the group. + [direct membership](../../project/members/index.md#display-direct-members) of the group. - A lower or the same role with Group Sync are displayed as having - [inherited membership](../../project/members/#display-inherited-members) of the group. + [inherited membership](../../project/members/index.md#display-inherited-members) of the group. ### Automatic member removal @@ -167,3 +167,18 @@ graph TB > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/290367) in GitLab 15.3. You can use the GitLab API to [list, add, and delete](../../../api/groups.md#saml-group-links) SAML group links. + +## Troubleshooting + +This section contains possible solutions for problems you might encounter. + +### User that belongs to many SAML groups automatically removed from GitLab group + +When using Azure AD as the SAML identity provider, users that belong to many SAML groups can be automatically removed from your GitLab group. Users are removed from GitLab +groups if the group claim is missing from the user's SAML assertion. + +Because of a [known issue with Azure AD](https://support.esri.com/en/technical-article/000022190), if a user belongs to more than 150 SAML groups, the group claim is not sent +in the user's SAML assertion. + +To work around this issue, allow more than 150 group IDs to be sent in SAML token using configuration steps in the +[Azure AD documentation](https://support.esri.com/en/technical-article/000022190). diff --git a/doc/user/group/saml_sso/img/scim_token_v13_3.png b/doc/user/group/saml_sso/img/scim_token_v13_3.png Binary files differdeleted file mode 100644 index e98f755718a..00000000000 --- a/doc/user/group/saml_sso/img/scim_token_v13_3.png +++ /dev/null diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 25060f8e749..d33cde0a6b1 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -24,7 +24,7 @@ If required, you can find [a glossary of common terms](../../../integration/saml ## Configure your identity provider 1. Find the information in GitLab required for configuration: - 1. On the top bar, select **Menu > Groups** and find your group. + 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > SAML SSO**. 1. Note the **Assertion consumer service URL**, **Identifier**, and **GitLab single sign-on URL**. 1. Configure your SAML identity provider app using the noted details. @@ -79,7 +79,7 @@ You can configure the following attributes with GitLab.com Group SAML: GitLab provides metadata XML that can be used to configure your identity provider. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > SAML SSO**. 1. Copy the provided **GitLab metadata URL**. 1. Follow your identity provider's documentation and paste the metadata URL when it's requested. @@ -88,7 +88,7 @@ GitLab provides metadata XML that can be used to configure your identity provide After you set up your identity provider to work with GitLab, you must configure GitLab to use it for authentication: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > SAML SSO**. 1. Find the SSO URL from your identity provider and enter it the **Identity provider single sign-on URL** field. 1. Find and enter the fingerprint for the SAML token signing certificate in the **Certificate** field. @@ -101,6 +101,19 @@ After you set up your identity provider to work with GitLab, you must configure NOTE: The certificate [fingerprint algorithm](../../../integration/saml.md#notes-on-configuring-your-identity-provider) must be in SHA1. When configuring the identity provider (such as [Google Workspace](#google-workspace-setup-notes)), use a secure signature algorithm. +### Additional configuration information + +Many SAML terms can vary between providers. It is possible that the information you are looking for is listed under another name. + +For more information, start with your identity provider's documentation. Look for their options and examples to see how they configure SAML. This can provide hints on what you need to configure GitLab to work with these providers. + +It can also help to look at our [more detailed docs for self-managed GitLab](../../../integration/saml.md). +SAML configuration for GitLab.com is mostly the same as for self-managed instances. +However, self-managed GitLab instances use a configuration file that supports more options as described in the external [OmniAuth SAML documentation](https://github.com/omniauth/omniauth-saml/). +Internally that uses the [`ruby-saml` library](https://github.com/onelogin/ruby-saml), so we sometimes check there to verify low level details of less commonly used options. + +It can also help to compare the XML response from your provider with our [example XML used for internal testing](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/fixtures/saml/response.xml). + ### SSO enforcement > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5291) in GitLab 11.8. @@ -122,7 +135,7 @@ prompts the user to sign in again through SSO. An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/297389) to add a similar SSO requirement for API activity. -SSO has the following effects when enabled: +SSO enforcement has the following effects when enabled: - For groups, users can't share a project in the group outside the top-level group, even if the project is forked. @@ -157,8 +170,8 @@ If you have any questions on configuring the SAML app, please contact your provi Follow the Azure documentation on [configuring single sign-on to applications](https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/view-applications-portal) with the notes below for consideration. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For a demo of the Azure SAML setup including SCIM, see [SCIM Provisioning on Azure Using SAML SSO for Groups Demo](https://youtu.be/24-ZxmTeEBU). The video is outdated in regard to -objectID mapping and the [SCIM documentation should be followed](scim_setup.md#azure-configuration-steps). +For a demo of the Azure SAML setup including SCIM, see [SCIM Provisioning on Azure Using SAML SSO for Groups Demo](https://youtu.be/24-ZxmTeEBU). +The video is outdated in regard to objectID mapping and you should follow the [SCIM documentation](scim_setup.md#configure-azure-active-directory). | GitLab Setting | Azure Field | | ------------------------------------ | ------------------------------------------ | @@ -168,7 +181,7 @@ objectID mapping and the [SCIM documentation should be followed](scim_setup.md#a | Identity provider single sign-on URL | Login URL | | Certificate fingerprint | Thumbprint | -The recommended attributes and claims settings are: +The recommended attributes are: - **Unique User Identifier (Name identifier)** set to `user.objectID`. - **nameid-format** set to persistent. @@ -311,6 +324,13 @@ On subsequent visits, you should be able to go [sign in to GitLab.com with SAML] 1. From the list of apps, select the "GitLab.com" app. (The name is set by the administrator of the identity provider.) 1. You are then signed in to GitLab.com and redirected to the group. +### Change NameID for one or more users + +If the NameID changes for one or more users, they need to reconnect their SAML account. + +1. Ask relevant users to [unlink their account from the group](#unlinking-accounts). +1. Ask relevant users to [link their account to the new SAML app](#linking-saml-to-your-existing-gitlabcom-account). + ### Configure user settings from SAML response > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263661) in GitLab 13.7. @@ -357,6 +377,18 @@ convert the information to XML. An example SAML response is shown here. </saml2:AttributeStatement> ``` +### Bypass user email confirmation with verified domains + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/238461) in GitLab 15.4. + +By default, users provisioned with SAML or SCIM are sent a verification email to verify their identity. Instead, you can +[configure GitLab with a custom domain](../../project/pages/custom_domains_ssl_tls_certification/index.md) and GitLab +will automatically confirm user accounts. Users will still receive an enterprise user welcome email. +Confirmation is bypassed for users: + +- That are provisioned with SAML or SCIM. +- That have an email address that belongs to the verified domain. + ### Role Starting from [GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/issues/214523), group owners can set a @@ -412,160 +444,4 @@ The [Generated passwords for users created through integrated authentication](.. ## Troubleshooting -This section contains possible solutions for problems you might encounter. - -### SAML debugging tools - -SAML responses are base64 encoded, so we recommend the following browser plugins to decode them on the fly: - -- [SAML-tracer](https://addons.mozilla.org/en-US/firefox/addon/saml-tracer/) for Firefox. -- [SAML Message Decoder](https://chrome.google.com/webstore/detail/saml-message-decoder/mpabchoaimgbdbbjjieoaeiibojelbhm?hl=en) for Chrome. - -Specific attention should be paid to: - -- The [NameID](#nameid), which we use to identify which user is signing in. If the user has previously signed in, this [must match the value we have stored](#verifying-nameid). -- The presence of a `X509Certificate`, which we require to verify the response signature. -- The `SubjectConfirmation` and `Conditions`, which can cause errors if misconfigured. - -#### Generate a SAML Response - -SAML Responses can be used to preview the attribute names and values sent in the assertions list while attempting to sign in using an IdP. - -To generate a SAML Response: - -1. Install either: - - [SAML Chrome Panel](https://chrome.google.com/webstore/detail/saml-chrome-panel/paijfdbeoenhembfhkhllainmocckace) for Chrome. - - [SAML-tracer](https://addons.mozilla.org/en-US/firefox/addon/saml-tracer/) for Firefox. -1. Open a new browser tab. -1. Open the SAML tracer console: - - Chrome: Right-click on the page, select **Inspect**, then select the **SAML** tab in the opened developer console. - - Firefox: Select the SAML-tracer icon located on the browser toolbar. -1. Go to the GitLab single sign-on URL for the group in the same browser tab with the SAML tracer open. -1. Select **Authorize** or attempt to log in. A SAML response is displayed in the tracer console that resembles this - [example SAML response](#example-saml-response). -1. Within the SAML tracer, select the **Export** icon to save the response in JSON format. - -### Verifying configuration - -For convenience, we've included some [example resources](../../../user/group/saml_sso/example_saml_config.md) used by our Support Team. While they may help you verify the SAML app configuration, they are not guaranteed to reflect the current state of third-party products. - -### Verifying NameID - -In troubleshooting the Group SAML setup, any authenticated user can use the API to verify the NameID GitLab already has linked to the user by visiting [`https://gitlab.com/api/v4/user`](https://gitlab.com/api/v4/user) and checking the `extern_uid` under identities. - -Similarly, group members of a role with the appropriate permissions can make use of the [members API](../../../api/members.md) to view group SAML identity information for members of the group. - -This can then be compared to the [NameID](#nameid) being sent by the identity provider by decoding the message with a [SAML debugging tool](#saml-debugging-tools). We require that these match in order to identify users. - -### Users receive a 404 - -Because SAML SSO for groups is a paid feature, your subscription expiring can result in a `404` error when you're signing in using SAML SSO on GitLab.com. -If all users are receiving a `404` when attempting to log in using SAML, confirm -[there is an active subscription](../../../subscriptions/gitlab_com/index.md#view-your-gitlab-saas-subscription) being used in this SAML SSO namespace. - -If you receive a `404` during setup when using "verify configuration", make sure you have used the correct -[SHA-1 generated fingerprint](../../../integration/saml.md#notes-on-configuring-your-identity-provider). - -If a user is trying to sign in for the first time and the GitLab single sign-on URL has not [been configured](#configure-your-identity-provider), they may see a 404. -As outlined in the [user access section](#linking-saml-to-your-existing-gitlabcom-account), a group Owner needs to provide the URL to users. - -### Message: "SAML authentication failed: Extern UID has already been taken" - -This error suggests you are signed in as a GitLab user but have already linked your SAML identity to a different GitLab user. Sign out and then try to sign in again using the SSO SAML link, which should log you into GitLab with the linked user account. - -If you do not wish to use that GitLab user with the SAML login, you can [unlink the GitLab account from the group's SAML](#unlinking-accounts). - -### Message: "SAML authentication failed: User has already been taken" - -The user that you're signed in with already has SAML linked to a different identity, or the NameID value has changed. -Here are possible causes and solutions: - -| Cause | Solution | -| ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| You've tried to link multiple SAML identities to the same user, for a given identity provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](#unlinking-accounts) from this GitLab account before attempting to sign in again. | -| The NameID changes every time the user requests SSO identification | Check the NameID is not set with `Transient` format, or the NameID is not changing on subsequent requests.| - -### Message: "SAML authentication failed: Email has already been taken" - -| Cause | Solution | -| ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | -| When a user account with the email address already exists in GitLab, but the user does not have the SAML identity tied to their account. | The user needs to [link their account](#user-access-and-management). | - -User accounts are created in one of the following ways: - -- User registration -- Sign in through OAuth -- Sign in through SAML -- SCIM provisioning - -### Message: "SAML authentication failed: Extern UID has already been taken, User has already been taken" - -Getting both of these errors at the same time suggests the NameID capitalization provided by the identity provider didn't exactly match the previous value for that user. - -This can be prevented by configuring the [NameID](#nameid) to return a consistent value. Fixing this for an individual user involves [unlinking SAML in the GitLab account](#unlinking-accounts), although this causes group membership and to-do items to be lost. - -### Message: "Request to link SAML account must be authorized" - -Ensure that the user who is trying to link their GitLab account has been added as a user within the identity provider's SAML app. - -Alternatively, the SAML response may be missing the `InResponseTo` attribute in the -`samlp:Response` tag, which is [expected by the SAML gem](https://github.com/onelogin/ruby-saml/blob/9f710c5028b069bfab4b9e2b66891e0549765af5/lib/onelogin/ruby-saml/response.rb#L307-L316). -The identity provider administrator should ensure that the login is -initiated by the service provider and not the identity provider. - -### Message: "Sign in to GitLab to connect your organization's account" - -A user can see this message when they are trying to [manually link SAML to their existing GitLab.com account](#linking-saml-to-your-existing-gitlabcom-account). - -To resolve this problem, the user should check they are using the correct GitLab password to log in. They first need to -[reset their password](https://gitlab.com/users/password/new) if both: - -- The account was provisioned by SCIM. -- This is the first time the user has logged in the username and password. - -### Stuck in a login "loop" - -Ensure that the **GitLab single sign-on URL** has been configured as "Login URL" (or similarly named field) in the identity provider's SAML app. - -Alternatively, when users need to [link SAML to their existing GitLab.com account](#linking-saml-to-your-existing-gitlabcom-account), provide the **GitLab single sign-on URL** and instruct users not to use the SAML app on first sign in. - -### The NameID has changed - -| Cause | Solution | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| As mentioned in the [NameID](#nameid) section, if the NameID changes for any user, the user can be locked out. This is a common problem when an email address is used as the identifier. | Follow the steps outlined in the ["SAML authentication failed: User has already been taken"](#message-saml-authentication-failed-user-has-already-been-taken) section. | - -### I need additional information to configure my identity provider - -Many SAML terms can vary between providers. It is possible that the information you are looking for is listed under another name. - -For more information, start with your identity provider's documentation. Look for their options and examples to see how they configure SAML. This can provide hints on what you need to configure GitLab to work with these providers. - -It can also help to look at our [more detailed docs for self-managed GitLab](../../../integration/saml.md). -SAML configuration for GitLab.com is mostly the same as for self-managed instances. -However, self-managed GitLab instances use a configuration file that supports more options as described in the external [OmniAuth SAML documentation](https://github.com/omniauth/omniauth-saml/). -Internally that uses the [`ruby-saml` library](https://github.com/onelogin/ruby-saml), so we sometimes check there to verify low level details of less commonly used options. - -It can also help to compare the XML response from your provider with our [example XML used for internal testing](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/fixtures/saml/response.xml). - -### Searching Rails log - -With access to the rails log or `production_json.log` (available only to GitLab team members for GitLab.com), -you should be able to find the base64 encoded SAML response by searching with the following filters: - -- `json.meta.caller_id`: `Groups::OmniauthCallbacksController#group_saml` -- `json.meta.user` or `json.username`: `username` -- `json.method`: `POST` -- `json.path`: `/groups/GROUP-PATH/-/saml/callback` - -In a relevant log entry, the `json.params` should provide a valid response with: - -- `"key": "SAMLResponse"` and the `"value": (full SAML response)`, -- `"key": "RelayState"` with `"value": "/group-path"`, and -- `"key": "group_id"` with `"value": "group-path"`. - -In some cases, if the SAML response is lengthy, you may receive a `"key": "truncated"` with `"value":"..."`. -In these cases, please ask a group owner for a copy of the SAML response from when they select -the "Verify SAML Configuration" button on the group SSO Settings page. - -Use a base64 decoder to see a human-readable version of the SAML response. +See our [troubleshooting SAML guide](troubleshooting.md). diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index 7962f171166..a67b2fbe02e 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -5,107 +5,129 @@ group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# SCIM provisioning using SAML SSO for GitLab.com groups **(PREMIUM SAAS)** +# Configure SCIM for GitLab.com groups **(PREMIUM SAAS)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/9388) in GitLab 11.10. -System for Cross-domain Identity Management (SCIM), is an open standard that enables the -automation of user provisioning. When SCIM is provisioned for a GitLab group, membership of -that group is synchronized between GitLab and the identity provider. +You can use the open standard System for Cross-domain Identity Management (SCIM) to automatically: -The GitLab [SCIM API](../../../api/scim.md) implements part of [the RFC7644 protocol](https://tools.ietf.org/html/rfc7644). - -## Features +- Create users. +- Remove users (deactivate SCIM identity). -The following actions are available: +GitLab SAML SSO SCIM doesn't support updating users. -- Create users -- Remove users (deactivate SCIM identity) +When SCIM is enabled for a GitLab group, membership of that group is synchronized between GitLab and an identity provider. -The following identity providers are supported: - -- Azure -- Okta +The GitLab [SCIM API](../../../api/scim.md) implements part of [the RFC7644 protocol](https://tools.ietf.org/html/rfc7644). -## Requirements +## Configure GitLab -- [Group Single Sign-On](index.md) must be configured. +Prerequisites: -## GitLab configuration +- [Group single sign-on](index.md) must be configured. -Once [Group Single Sign-On](index.md) has been configured, we can: +To configure GitLab SAML SSO SCIM: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > SAML SSO**. 1. Select **Generate a SCIM token**. -1. Save the token and URL for use in the next step. +1. For configuration of your identity provider, save the: + - Token from the **Your SCIM token** field. + - URL from the **SCIM API endpoint URL** field. - +## Configure an identity provider -## Identity Provider configuration +You can configure one of the following as an identity provider: -- [Azure](#azure-configuration-steps) -- [Okta](#okta-configuration-steps) +- [Azure Active Directory](#configure-azure-active-directory). +- [Okta](#configure-okta). +- [OneLogin](#configure-onelogin). -### Azure configuration steps +NOTE: +Other providers can work with GitLab but they have not been tested and are not supported. -The SAML application that was created during [Single sign-on](index.md) setup for [Azure](https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/view-applications-portal) now needs to be set up for SCIM. You can refer to [Azure SCIM setup documentation](https://docs.microsoft.com/en-us/azure/active-directory/app-provisioning/use-scim-to-provision-users-and-groups#getting-started). +### Configure Azure Active Directory -1. In your app, go to the Provisioning tab, and set the **Provisioning Mode** to **Automatic**. - Then fill in the **Admin Credentials**, and save. The **Tenant URL** and **secret token** are the items - retrieved in the [previous step](#gitlab-configuration). +The SAML application created during [single sign-on](index.md) set up for +[Azure Active Directory](https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/view-applications-portal) +must be set up for SCIM. For an example, see [example configuration](example_saml_config.md#scim-mapping). -1. After saving, two more tabs appear: +To configure Azure Active Directory for SCIM: - - **Settings**: We recommend setting a notification email and selecting the **Send an email notification when a failure occurs** checkbox. - You also control what is actually synced by selecting the **Scope**. For example, **Sync only assigned users and groups** only syncs the users and groups assigned to the application. Otherwise, it syncs the whole Active Directory. +1. In your app, go to the **Provisioning** tab and select **Get started**. +1. Set the **Provisioning Mode** to **Automatic**. +1. Complete the **Admin Credentials** using the value of: + - **SCIM API endpoint URL** in GitLab for the **Tenant URL** field. + - **Your SCIM token** in GitLab for the **Secret Token** field. +1. Select **Test Connection**. If the test is successful, save your configuration before continuing, or see the + [troubleshooting](#troubleshooting) information. +1. Select **Save**. - - **Mappings**: We recommend keeping **Provision Azure Active Directory Users** enabled, and disable **Provision Azure Active Directory Groups**. - Leaving **Provision Azure Active Directory Groups** enabled does not break the SCIM user provisioning, but it causes errors in Azure AD that may be confusing and misleading. +After saving, **Settings** and **Mappings** sections appear. + +1. Under **Settings**, if required, set a notification email and select the + **Send an email notification when a failure occurs** checkbox. +1. Under **Mappings**, we recommend you: + 1. Keep **Provision Azure Active Directory Users** enabled and select the **Provision Azure Active Directory Users** + link to [configure attribute mappings](#configure-attribute-mappings). + 1. Below the mapping list select the **Show advanced options** checkbox. + 1. Select the **Edit attribute list for customappsso** link. + 1. Ensure the `id` is the primary and required field, and `externalId` is also required. + 1. Select **Save**. +1. Return to the **Provisioning** tab, saving unsaved changes if necessary. +1. Select **Edit attribute mappings**. +1. Under **Mappings**: + 1. Select **Provision Azure Active Directory Groups**. + 1. On the Attribute Mapping page, turn off the **Enabled** toggle. Leaving it turned on doesn't break the SCIM user + provisioning, but it causes errors in Azure Active Directory that may be confusing and misleading. + 1. Select **Save**. +1. Return to the **Provisioning** tab, saving unsaved changes if necessary. +1. Select **Edit attribute mappings**. +1. Turn on the **Provisioning Status** toggle. Synchronization details and any errors appears on the bottom of the + **Provisioning** screen, together with a link to the audit events. -1. You can then test the connection by selecting **Test Connection**. If the connection is successful, save your configuration before moving on. See below for [troubleshooting](#troubleshooting). +WARNING: +Once synchronized, changing the field mapped to `id` and `externalId` may cause a number of errors. These include +provisioning errors, duplicate users, and may prevent existing users from accessing the GitLab group. -#### Configure attribute mapping +#### Configure attribute mappings -Follow [Azure documentation to configure the attribute mapping](https://docs.microsoft.com/en-us/azure/active-directory/app-provisioning/customize-application-attributes). +While [configuring Azure Active Directory for SCIM](#configure-azure-active-directory), you configure attribute mappings. +For an example, see [example configuration](example_saml_config.md#scim-mapping). -The following table below provides an attribute mapping known to work with GitLab. If -your SAML configuration differs from [the recommended SAML settings](index.md#azure-setup-notes), -modify the corresponding `customappsso` settings accordingly. In particular, the `externalId` must -match the [SAML NameID](index.md#nameid). -If a mapping is not listed in the table, use the Azure defaults. -For a list of required attributes, refer to the [SCIM API documentation](../../../api/scim.md). +The following table provides attribute mappings known to work with GitLab. -| Azure Active Directory Attribute | `customappsso` Attribute | Matching precedence | -| -------------------------------- | ------------------------------ | ------------------- | -| `objectId` | `externalId` | 1 | -| `userPrincipalName` | `emails[type eq "work"].value` | | -| `mailNickname` | `userName` | | +| Source attribute | Target attribute | Matching precedence | +|:--------------------|:-------------------------------|:--------------------| +| `objectId` | `externalId` | 1 | +| `userPrincipalName` | `emails[type eq "work"].value` | | +| `mailNickname` | `userName` | | -For guidance, you can view [an example configuration](example_saml_config.md#azure-active-directory). +Each attribute mapping has: -1. Below the mapping list select **Show advanced options > Edit attribute list for AppName**. -1. Ensure the `id` is the primary and required field, and `externalId` is also required. +- An Azure Active Directory attribute (source attribute). +- A `customappsso` attribute (target attribute). +- A matching precedence. - NOTE: - `username` should neither be primary nor required as we don't support - that field on GitLab SCIM yet. +For each attribute: -1. Save all changes. -1. In the **Provisioning** step, set the `Provisioning Status` to `On`. +1. Select the attribute to edit it. +1. Select the required settings. +1. Select **Ok**. -Once enabled, the synchronization details and any errors appears on the -bottom of the **Provisioning** screen, together with a link to the audit events. +If your SAML configuration differs from [the recommended SAML settings](index.md#azure-setup-notes), select the mapping +attributes and modify them accordingly. In particular, the `objectId` source attribute must map to the `externalId` +target attribute. -WARNING: -Once synchronized, changing the field mapped to `id` and `externalId` may cause a number of errors. These include provisioning errors, duplicate users, and may prevent existing users from accessing the GitLab group. +If a mapping is not listed in the table, use the Azure Active Directory defaults. For a list of required attributes, +refer to the [SCIM API documentation](../../../api/scim.md). -### Okta configuration steps +### Configure Okta Before you start this section: - Check that you are using Okta [Lifecycle Management](https://www.okta.com/products/lifecycle-management/) product. This product tier is required to use SCIM on Okta. To check which Okta product you are using, check your signed Okta contract, contact your Okta AE, CSM, or Okta support. -- Complete the [GitLab configuration](#gitlab-configuration) process. +- Complete the [GitLab configuration](#configure-gitlab) process. - Complete the setup for SAML application for [Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/main/), as described in the [Okta setup notes](index.md#okta-setup-notes). - Check that your Okta SAML setup matches our documentation exactly, especially the NameID configuration. Otherwise, the Okta SCIM app may not work properly. @@ -137,7 +159,7 @@ The Okta GitLab application currently only supports SCIM. Continue using the separate Okta [SAML SSO](index.md) configuration along with the new SCIM application described above. An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/216173) to add SAML support to the Okta GitLab application. -### OneLogin +### Configure OneLogin As the developers of this app, OneLogin provides a "GitLab (SaaS)" app in their catalog, which includes a SCIM integration. Please reach out to OneLogin if you encounter issues. @@ -216,17 +238,17 @@ The **Identity** (`extern_uid`) value stored by GitLab is updated by SCIM whenev This value is also used by SCIM to match users on the `id`, and is updated by SCIM whenever the `id` or `externalId` values change. -It is important that this SCIM `id` and SCIM `externalId` are configured to the same value as the SAML `NameId`. SAML responses can be traced using [debugging tools](index.md#saml-debugging-tools), and any errors can be checked against our [SAML troubleshooting docs](index.md#troubleshooting). +It is important that this SCIM `id` and SCIM `externalId` are configured to the same value as the SAML `NameId`. SAML responses can be traced using [debugging tools](troubleshooting.md#saml-debugging-tools), and any errors can be checked against our [SAML troubleshooting docs](troubleshooting.md). ### How do I verify user's SAML NameId matches the SCIM externalId -Admins can use the Admin Area to [list SCIM identities for a user](../../admin_area/#user-identities). +Admins can use the Admin Area to [list SCIM identities for a user](../../admin_area/index.md#user-identities). Group owners can see the list of users and the `externalId` stored for each user in the group SAML SSO Settings page. A possible alternative is to use the [SCIM API](../../../api/scim.md#get-a-list-of-scim-provisioned-users) to manually retrieve the `externalId` we have stored for users, also called the `external_uid` or `NameId`. -To see how the `external_uid` compares to the value returned as the SAML NameId, you can have the user use a [SAML Tracer](index.md#saml-debugging-tools). +To see how the `external_uid` compares to the value returned as the SAML NameId, you can have the user use a [SAML Tracer](troubleshooting.md#saml-debugging-tools). ### Update or fix mismatched SCIM externalId and SAML NameId @@ -241,7 +263,7 @@ that provider may create duplicate users. If the `externalId` for a user is not correct, and also doesn't match the SAML NameID, you can address the problem in the following ways: -- You can have users unlink and relink themselves, based on the ["SAML authentication failed: User has already been taken"](index.md#message-saml-authentication-failed-user-has-already-been-taken) section. +- You can have users unlink and relink themselves, based on the ["SAML authentication failed: User has already been taken"](troubleshooting.md#message-saml-authentication-failed-user-has-already-been-taken) section. - You can unlink all users simultaneously, by removing all users from the SAML app while provisioning is turned on. - It may be possible to use the [SCIM API](../../../api/scim.md#update-a-single-scim-provisioned-user) to manually correct the `externalId` stored for users to match the SAML `NameId`. To look up a user, you need to know the desired value that matches the `NameId` as well as the current `externalId`. @@ -314,4 +336,4 @@ and the error response can include a HTML result of the GitLab URL `https://gitl This error is harmless and occurs because Group provisioning was turned on but GitLab SCIM integration does not support it nor require it. To remove the error, follow the instructions in the Azure configuration guide to disable the option -[`Synchronize Azure Active Directory Groups to AppName`](#azure-configuration-steps). +[`Synchronize Azure Active Directory Groups to AppName`](#configure-azure-active-directory). diff --git a/doc/user/group/saml_sso/troubleshooting.md b/doc/user/group/saml_sso/troubleshooting.md new file mode 100644 index 00000000000..177f33228c0 --- /dev/null +++ b/doc/user/group/saml_sso/troubleshooting.md @@ -0,0 +1,249 @@ +--- +type: reference +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Troubleshooting SAML **(FREE)** + +This page contains possible solutions for problems you might encounter when using: + +- [SAML SSO for GitLab.com groups](index.md). +- The self-managed instance-level [SAML OmniAuth Provider](../../../integration/saml.md). + +## SAML debugging tools + +SAML responses are base64 encoded, so we recommend the following browser plugins to decode them on the fly: + +- [SAML-tracer](https://addons.mozilla.org/en-US/firefox/addon/saml-tracer/) for Firefox. +- [SAML Message Decoder](https://chrome.google.com/webstore/detail/saml-message-decoder/mpabchoaimgbdbbjjieoaeiibojelbhm?hl=en) for Chrome. + +Specific attention should be paid to: + +- The NameID, which we use to identify which user is signing in. If the user has previously signed in, this [must match the value we have stored](#verifying-nameid). +- The presence of a `X509Certificate`, which we require to verify the response signature. +- The `SubjectConfirmation` and `Conditions`, which can cause errors if misconfigured. + +### Generate a SAML response + +Use SAML responses to preview the attribute names and values sent in the assertions list while attempting to sign in +using an identity provider. + +To generate a SAML Response: + +1. Install one of the browser debugging tools previously mentioned. +1. Open a new browser tab. +1. Open the SAML tracer console: + - Chrome: On a context menu on the page, select **Inspect**, then select the **SAML** tab in the opened developer + console. + - Firefox: Select the SAML-tracer icon located on the browser toolbar. +1. Go to the GitLab single sign-on URL for the group in the same browser tab with the SAML tracer open. +1. Select **Authorize** or attempt to log in. A SAML response is displayed in the tracer console that resembles this + [example SAML response](index.md#example-saml-response). +1. Within the SAML tracer, select the **Export** icon to save the response in JSON format. + +## GitLab SAML Testing Environments + +To troubleshoot, [a complete GitLab with SAML testing environment using Docker compose](https://gitlab.com/gitlab-com/support/toolbox/replication/tree/master/compose_files) +is available. + +If you only require a SAML provider for testing, a [quick start guide to start a Docker container](../../../administration/troubleshooting/test_environments.md#saml) with a plug and play SAML 2.0 Identity Provider (identity provider) is available. + +You can test the SaaS feature locally by [enabling SAML for groups on a self-managed instance](../../../integration/saml.md#configuring-group-saml-on-a-self-managed-gitlab-instance). + +## Verifying configuration + +For convenience, we've included some [example resources](../../../user/group/saml_sso/example_saml_config.md) used by our Support Team. While they may help you verify the SAML app configuration, they are not guaranteed to reflect the current state of third-party products. + +## Searching Rails log for a SAML response **(FREE SELF)** + +You can find the base64-encoded SAML Response in the [`production_json.log`](../../../administration/logs/index.md#production_jsonlog). +This response is sent from the identity provider, and contains user information that is consumed by GitLab. +Many errors in the SAML integration can be solved by decoding this response and comparing it to the SAML settings in the GitLab configuration file. + +For example, with SAML for groups, +you should be able to find the base64 encoded SAML response by searching with the following filters: + +- `json.meta.caller_id`: `Groups::OmniauthCallbacksController#group_saml` +- `json.meta.user` or `json.username`: `username` +- `json.method`: `POST` +- `json.path`: `/groups/GROUP-PATH/-/saml/callback` + +In a relevant log entry, the `json.params` should provide a valid response with: + +- `"key": "SAMLResponse"` and the `"value": (full SAML response)`, +- `"key": "RelayState"` with `"value": "/group-path"`, and +- `"key": "group_id"` with `"value": "group-path"`. + +In some cases, if the SAML response is lengthy, you may receive a `"key": "truncated"` with `"value":"..."`. +In these cases, use one of the [SAML debugging tools](#saml-debugging-tools), or for SAML SSO for groups, +a group owner can get a copy of the SAML response from when they select +the "Verify SAML Configuration" button on the group SSO Settings page. + +Use a base64 decoder to see a human-readable version of the SAML response. + +## Configuration errors + +### Invalid audience + +This error means that the identity provider doesn't recognize GitLab as a valid sender and +receiver of SAML requests. Make sure to: + +- Add the GitLab callback URL to the approved audiences of the identity provider server. +- Avoid trailing whitespace in the `issuer` string. + +### Key validation error, Digest mismatch or Fingerprint mismatch + +These errors all come from a similar place, the SAML certificate. SAML requests +must be validated using either a fingerprint, a certificate, or a validator. + +For this requirement, be sure to take the following into account: + +- If a fingerprint is used, it must be the SHA1 fingerprint +- If no certificate is provided in the settings, a fingerprint or fingerprint + validator needs to be provided and the response from the server must contain + a certificate (`<ds:KeyInfo><ds:X509Data><ds:X509Certificate>`) +- If a certificate is provided in the settings, it is no longer necessary for + the request to contain one. In this case the fingerprint or fingerprint + validators are optional + +If none of the above described scenarios is valid, the request +fails with one of the mentioned errors. + +### Missing claims, or `Email can't be blank` errors + +The identity provider server needs to pass certain information in order for GitLab to either +create an account, or match the login information to an existing account. `email` +is the minimum amount of information that needs to be passed. If the identity provider server +is not providing this information, all SAML requests fail. + +Make sure this information is provided. + +Another issue that can result in this error is when the correct information is being sent by +the identity provider, but the attributes don't match the names in the OmniAuth `info` hash. In this case, +you must set `attribute_statements` in the SAML configuration to +[map the attribute names in your SAML Response to the corresponding OmniAuth `info` hash names](../../../integration/saml.md#attribute_statements). + +## User sign in banner error messages + +### Message: "SAML authentication failed: Extern UID has already been taken" + +This error suggests you are signed in as a GitLab user but have already linked your SAML identity to a different GitLab user. Sign out and then try to sign in again using SAML, which should log you into GitLab with the linked user account. + +If you do not wish to use that GitLab user with the SAML login, you can [unlink the GitLab account from the SAML app](index.md#unlinking-accounts). + +### Message: "SAML authentication failed: User has already been taken" + +The user that you're signed in with already has SAML linked to a different identity, or the NameID value has changed. +Here are possible causes and solutions: + +| Cause | Solution | +| ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| You've tried to link multiple SAML identities to the same user, for a given identity provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](index.md#unlinking-accounts) from this GitLab account before attempting to sign in again. | +| The NameID changes every time the user requests SSO identification | [Check the NameID](#verifying-nameid) is not set with `Transient` format, or the NameID is not changing on subsequent requests.| + +### Message: "SAML authentication failed: Email has already been taken" + +| Cause | Solution | +| ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | +| When a user account with the email address already exists in GitLab, but the user does not have the SAML identity tied to their account. | The user needs to [link their account](index.md#user-access-and-management). | + +User accounts are created in one of the following ways: + +- User registration +- Sign in through OAuth +- Sign in through SAML +- SCIM provisioning + +### Message: "SAML authentication failed: Extern UID has already been taken, User has already been taken" + +Getting both of these errors at the same time suggests the NameID capitalization provided by the identity provider didn't exactly match the previous value for that user. + +This can be prevented by configuring the NameID to return a consistent value. Fixing this for an individual user involves changing the identifier for the user. For GitLab.com, the user needs to [unlink their SAML from the GitLab account](index.md#unlinking-accounts). + +### Message: "Request to link SAML account must be authorized" + +Ensure that the user who is trying to link their GitLab account has been added as a user within the identity provider's SAML app. + +Alternatively, the SAML response may be missing the `InResponseTo` attribute in the +`samlp:Response` tag, which is [expected by the SAML gem](https://github.com/onelogin/ruby-saml/blob/9f710c5028b069bfab4b9e2b66891e0549765af5/lib/onelogin/ruby-saml/response.rb#L307-L316). +The identity provider administrator should ensure that the login is +initiated by the service provider and not only the identity provider. + +### Message: "Sign in to GitLab to connect your organization's account" **(PREMIUM SAAS)** + +A user can see this message when they are trying to [manually link SAML to their existing GitLab.com account](index.md#linking-saml-to-your-existing-gitlabcom-account). + +To resolve this problem, the user should check they are using the correct GitLab password to log in. The user first needs +to [reset their password](https://gitlab.com/users/password/new) if both: + +- The account was provisioned by SCIM. +- They are signing in with username and password for the first time. + +## Other user sign in issues + +### Verifying NameID + +In troubleshooting, any authenticated user can use the API to verify the NameID GitLab already has linked to the user by visiting [`https://gitlab.com/api/v4/user`](https://gitlab.com/api/v4/user) and checking the `extern_uid` under identities. + +For self-managed, administrators can use the [users API](../../../api/users.md) to see the same information. + +When using SAML for groups, group members of a role with the appropriate permissions can make use of the [members API](../../../api/members.md) to view group SAML identity information for members of the group. + +This can then be compared to the NameID being sent by the identity provider by decoding the message with a [SAML debugging tool](#saml-debugging-tools). We require that these match in order to identify users. + +### Stuck in a login "loop" + +Ensure that the **GitLab single sign-on URL** (for GitLab.com) or the instance URL (for self-managed) has been configured as "Login URL" (or similarly named field) in the identity provider's SAML app. + +For GitLab.com, alternatively, when users need to [link SAML to their existing GitLab.com account](index.md#linking-saml-to-your-existing-gitlabcom-account), provide the **GitLab single sign-on URL** and instruct users not to use the SAML app on first sign in. + +### Users receive a 404 **(PREMIUM SAAS)** + +Because SAML SSO for groups is a paid feature, your subscription expiring can result in a `404` error when you're signing in using SAML SSO on GitLab.com. +If all users are receiving a `404` when attempting to log in using SAML, confirm +[there is an active subscription](../../../subscriptions/gitlab_com/index.md#view-your-gitlab-saas-subscription) being used in this SAML SSO namespace. + +If you receive a `404` during setup when using "verify configuration", make sure you have used the correct +[SHA-1 generated fingerprint](../../../integration/saml.md#notes-on-configuring-your-identity-provider). + +If a user is trying to sign in for the first time and the GitLab single sign-on URL has not [been configured](index.md#configure-your-identity-provider), they may see a 404. +As outlined in the [user access section](index.md#linking-saml-to-your-existing-gitlabcom-account), a group Owner needs to provide the URL to users. + +### 500 error after login **(FREE SELF)** + +If you see a "500 error" in GitLab when you are redirected back from the SAML +sign-in page, this could indicate that: + +- GitLab couldn't get the email address for the SAML user. Ensure the identity provider provides a claim containing the user's + email address using the claim name `email` or `mail`. +- The certificate set your `gitlab.rb` file for `identity provider_cert_fingerprint` or `identity provider_cert` file is incorrect. +- Your `gitlab.rb` file is set to enable `identity provider_cert_fingerprint`, and `identity provider_cert` is being provided, or the reverse. + +### 422 error after login **(FREE SELF)** + +If you see a "422 error" in GitLab when you are redirected from the SAML +sign-in page, you might have an incorrectly configured Assertion Consumer +Service (ACS) URL on the identity provider. + +Make sure the ACS URL points to `https://gitlab.example.com/users/auth/saml/callback`, where +`gitlab.example.com` is the URL of your GitLab instance. + +If the ACS URL is correct, and you still have errors, review the other +Troubleshooting sections. + +### User is blocked when signing in through SAML **(FREE SELF)** + +The following are the most likely reasons that a user is blocked when signing in through SAML: + +- In the configuration, `gitlab_rails['omniauth_block_auto_created_users'] = true` is set and this is the user's first time signing in. +- [`required_groups`](../../../integration/saml.md#required-groups) are configured but the user is not a member of one. + +## Google workspace troubleshooting tips + +The Google Workspace documentation on [SAML app error messages](https://support.google.com/a/answer/6301076?hl=en) is helpful for debugging if you are seeing an error from Google while signing in. +Pay particular attention to the following 403 errors: + +- `app_not_configured` +- `app_not_configured_for_user` diff --git a/doc/user/group/settings/group_access_tokens.md b/doc/user/group/settings/group_access_tokens.md index c3098bb56c2..eb9c6af9edf 100644 --- a/doc/user/group/settings/group_access_tokens.md +++ b/doc/user/group/settings/group_access_tokens.md @@ -27,6 +27,13 @@ associated with a group rather than a project or user. In self-managed instances, group access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. +WARNING: +The ability to create group access tokens without expiry was +[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and is planned for removal in GitLab +16.0. When this ability is removed, existing group access tokens without an expiry are planned to have an expiry added. +The automatic adding of an expiry occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry +occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. + You can use group access tokens: - On GitLab SaaS if you have the Premium license tier or higher. Group access tokens are not available with a [trial license](https://about.gitlab.com/free-trial/). @@ -43,11 +50,12 @@ configured for personal access tokens. ## Create a group access token using UI -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214045) in GitLab 14.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214045) in GitLab 14.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348660) in GitLab 15.3, default expiration of 30 days and default role of Guest is populated in the UI. To create a group access token: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Enter a name. The token name is visible to any user with permissions to view the group. 1. Optional. Enter an expiry date for the token. The token will expire on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. @@ -104,7 +112,7 @@ or API. However, administrators can use a workaround: To revoke a group access token: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Next to the group access token to revoke, select **Revoke**. @@ -138,7 +146,7 @@ The scope determines the actions you can perform when you authenticate with a gr To enable or disable group access token creation for all sub-groups in a top-level group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. 1. Under **Permissions**, turn on or off **Users can create project access tokens and group access tokens in this group**. @@ -154,8 +162,9 @@ to groups instead of projects. Bot users for groups: - Do not count as licensed seats. - Can have a maximum role of Owner for a group. For more information, see [Create a group access token](../../../api/group_access_tokens.md#create-a-group-access-token). -- The username is set to `group_{project_id}_bot` for the first access token. For example, `project_123_bot`. -- The email is set to `group{group_id}_bot@noreply.{Gitlab.config.gitlab.host}`. For example, `group123_bot@noreply.example.com`. -- All other properties are similar to [bot users for projects](../../project/settings/project_access_tokens.md#bot-users-for-projects) +- Have a username set to `group_{group_id}_bot` for the first access token. For example, `group_123_bot`. +- Have an email set to `group{group_id}_bot@noreply.{Gitlab.config.gitlab.host}`. For example, `group123_bot@noreply.example.com`. + +All other properties are similar to [bot users for projects](../../project/settings/project_access_tokens.md#bot-users-for-projects). For more information, see [Bot users for projects](../../project/settings/project_access_tokens.md#bot-users-for-projects). diff --git a/doc/user/group/settings/import_export.md b/doc/user/group/settings/import_export.md index eb94a181647..7a398c7d086 100644 --- a/doc/user/group/settings/import_export.md +++ b/doc/user/group/settings/import_export.md @@ -28,7 +28,7 @@ Prerequisite: To enable import and export for a group: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility and access controls**. 1. In the **Import sources** section, select the checkboxes for the sources you want. @@ -44,6 +44,7 @@ be imported into the desired group structure. - If imported into a parent group, a subgroup inherits the same level of visibility unless otherwise restricted. - To preserve the member list and their respective permissions on imported groups, review the users in these groups. Make sure these users exist before importing the desired groups. +- Users must set a public email in the source GitLab instance that matches one of their verified emails in the target GitLab instance. ### Exported contents @@ -55,6 +56,7 @@ The following items are exported: - Badges - Subgroups (including all the aforementioned data) - Epics + - Epic resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - Events - [Wikis](../../project/wiki/group.md) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/53247) in GitLab 13.9) @@ -77,7 +79,7 @@ Prerequisites: To export the contents of a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. In the **Advanced** section, select **Export Group**. 1. After the export is generated, you should receive an email with a link to the [exported contents](#exported-contents) diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 12434de5efc..657ef361bd5 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -56,7 +56,7 @@ the private subgroup. To view the subgroups of a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. Select the **Subgroups and projects** tab. 1. To view a nested subgroup, expand a subgroup in the hierarchy list. @@ -84,7 +84,7 @@ You cannot host a GitLab Pages subgroup website with a top-level domain name. Fo To create a subgroup: -1. On the top bar, select **Menu > Groups** and find and select the parent group to add a subgroup to. +1. On the top bar, select **Main menu > Groups** and find and select the parent group to add a subgroup to. 1. On the parent group's overview page, in the top right, select **New subgroup**. 1. Select **Create group**. 1. Fill in the fields. View a list of [reserved names](../../reserved_names.md) that cannot be used as group names. @@ -98,13 +98,13 @@ default: To change who can create subgroups on a group: - As a user with the Owner role on the group: - 1. On the top bar, select **Menu > Groups** and find your group. + 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. 1. Select a role from **Roles allowed to create subgroups**. 1. Select **Save changes**. - As an administrator: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Groups**. 1. In the group's row select **Edit**. 1. Select a role from **Allowed to create subgroups**. @@ -159,7 +159,7 @@ Group permissions for a member can be changed only by: To see if a member has inherited the permissions from a parent group: -1. On the top bar, select **Menu > Groups** and find the group. +1. On the top bar, select **Main menu > Groups** and find the group. 1. Select **Group information > Members**. Members list for an example subgroup _Four_: @@ -201,7 +201,7 @@ role on an ancestor group, add the user to the subgroup again with a higher role ## Mention subgroups Mentioning subgroups ([`@<subgroup_name>`](../../discussions/index.md#mentions)) in issues, commits, and merge requests -notifies all members of that group. Mentioning works the same as for projects and groups, and you can choose the group +notifies all direct members of that group. Inherited members of a sub-group are not notified by mentions. Mentioning works the same as for projects and groups, and you can choose the group of people to be notified. <!-- ## Troubleshooting diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md index 3e41b7b63cc..9078874d32c 100644 --- a/doc/user/group/value_stream_analytics/index.md +++ b/doc/user/group/value_stream_analytics/index.md @@ -35,7 +35,7 @@ Prerequisite: To view value stream analytics for your group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value stream**. 1. To view metrics for a particular stage, select a stage below the **Filter results** text box. 1. Optional. Filter the results: @@ -73,7 +73,7 @@ To view deployment metrics, you must have a To view the DORA metrics and key metrics: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value stream**. 1. Optional. Filter the results: 1. Select the **Filter results** text box. @@ -125,7 +125,7 @@ In GitLab 13.8 and earlier, deployment frequency metrics are calculated based on ### How value stream analytics aggregates data -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335391) in GitLab 14.5 [with a flag](../../../administration/feature_flags.md) named `use_vsa_aggregated_tables`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335391) in GitLab 14.5. > - Filter by stop date toggle [added](https://gitlab.com/gitlab-org/gitlab/-/issues/352428) in GitLab 14.9 > - Data refresh badge [added](https://gitlab.com/gitlab-org/gitlab/-/issues/341739) in GitLab 14.9 > - Filter by stop date toggle [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/84356) in GitLab 14.9 @@ -154,7 +154,7 @@ Value stream analytics shows the median time spent by issues or merge requests i To view the median time spent in each stage by a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value stream**. 1. Optional. Filter the results: 1. Select the **Filter results** text box. @@ -251,7 +251,7 @@ You can change the name of a project environment in your GitLab CI/CD configurat When you create a value stream, you can use GitLab default stages and hide or re-order them to customize. You can also create custom stages in addition to those provided in the default template. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. Select **Create new Value Stream**. 1. Enter a name for the value stream. @@ -275,7 +275,7 @@ If you have recently upgraded to GitLab Premium, it can take up to 30 minutes fo When you create a value stream, you can create and add custom stages that align with your own development workflows. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. Select **Create value stream**. 1. For each stage: @@ -301,7 +301,7 @@ time from a staging environment to production, you could use the following label After you create a value stream, you can customize it to suit your purposes. To edit a value stream: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. In the top right, select the dropdown list, and select a value stream. 1. Next to the value stream dropdown list, select **Edit**. @@ -320,7 +320,7 @@ After you create a value stream, you can customize it to suit your purposes. To To delete a custom value stream: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. In the top right, select the dropdown list and then select the value stream you would like to delete. 1. Select **Delete (name of value stream)**. @@ -336,7 +336,7 @@ To delete a custom value stream: The **Total time chart** shows the average number of days it takes for development cycles to complete. The chart shows data for the last 500 workflow items. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value stream**. 1. Above the **Filter results** box, select a stage: - To view a summary of the cycle time for all stages, select **Overview**. @@ -349,13 +349,19 @@ The chart shows data for the last 500 workflow items. - In the **From** field, select a start date. - In the **To** field, select an end date. -## Tasks by type chart +## View tasks by type -This chart shows a cumulative count of issues and merge requests per day. +The **Tasks by type** chart displays the cumulative number of issues and merge requests per day for your group. -This chart uses the global page filters for displaying data based on the selected -group, projects, and time frame. The chart defaults to showing counts for issues but can be -toggled to show data for merge requests and further refined for specific group-level labels. +The chart uses the global page filters to display data based on the selected +group, projects, and time frame. -By default the top group-level labels (max. 10) are pre-selected, with the ability to -select up to a total of 15 labels. +To view tasks by type: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Below the **Filter results** text box, select **Overview**. The **Tasks by type** chart displays below the **Total time** chart. +1. To switch between the task type, select the **Settings** (**{settings}**) dropdown list + and select **Issues** or **Merge Requests**. +1. To add or remove labels, select the **Settings** (**{settings}**) dropdown list + and select or search for a label. By default the top group-level labels (maximum 10) are selected. You can select a maximum of 15 labels. diff --git a/doc/user/infrastructure/clusters/connect/index.md b/doc/user/infrastructure/clusters/connect/index.md index 37e1024a32c..af728cb5c21 100644 --- a/doc/user/infrastructure/clusters/connect/index.md +++ b/doc/user/infrastructure/clusters/connect/index.md @@ -34,17 +34,17 @@ your cluster's level. **Project-level clusters:** -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. **Group-level clusters:** -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Kubernetes**. **Instance-level clusters:** -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Kubernetes**. ## Security implications for clusters connected with certificates diff --git a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md index ecf93958b1e..f9feef6329c 100644 --- a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Create a Civo Kubernetes cluster -Every new Civo account receives [$250 in credit](https://civo.com/signup) to get started with the GitLab integration with Civo Kubernetes. You can also use a marketplace app to install GitLab on your Civo Kubernetes cluster. +Every new Civo account receives [$250 in credit](https://dashboard.civo.com/signup) to get started with the GitLab integration with Civo Kubernetes. You can also use a marketplace app to install GitLab on your Civo Kubernetes cluster. Learn how to create a new cluster on Civo Kubernetes through [Infrastructure as Code (IaC)](../../index.md). This process uses the Civo @@ -15,7 +15,7 @@ by using the GitLab agent for Kubernetes. **Prerequisites:** -- A [Civo account](https://civo.com/signup). +- A [Civo account](https://dashboard.civo.com/signup). - [A runner](https://docs.gitlab.com/runner/install/) you can use to run the GitLab CI/CD pipeline. **Steps:** @@ -35,7 +35,8 @@ Start by [importing the example project by URL](../../../project/import/repo_by_ To import the project: -1. On the top bar, select **Menu > Create new project**. +1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Import project**. 1. Select **Repository by URL**. 1. For the **Git repository URL**, enter `https://gitlab.com/civocloud/gitlab-terraform-civo.git`. @@ -64,7 +65,7 @@ Use CI/CD environment variables to configure your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Variables**. -1. Set the variable `BASE64_CIVO_TOKEN` to the [token](https://www.civo.com/account/security) from your Civo account. +1. Set the variable `BASE64_CIVO_TOKEN` to the token from your Civo account. 1. Set the variable `TF_VAR_agent_token` to the agent token you received in the previous task. 1. Set the variable `TF_VAR_kas_address` to the agent server address in the previous task. @@ -95,7 +96,7 @@ After configuring your project, manually trigger the provisioning of your cluste When the pipeline finishes successfully, you can see your new cluster: -- In Civo dashboard: on your [Kubernetes tab](https://www.civo.com/account/kubernetes). +- In Civo dashboard: on your Kubernetes tab. - In GitLab: from your project's sidebar, select **Infrastructure > Kubernetes clusters**. ## Use your cluster diff --git a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md index 2f5967bd7ee..126968baee7 100644 --- a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md @@ -34,7 +34,8 @@ Start by [importing the example project by URL](../../../project/import/repo_by_ To import the project: -1. On the top bar, select **Menu > Create new project**. +1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Import project**. 1. Select **Repository by URL**. 1. For the **Git repository URL**, enter `https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-eks.git`. @@ -82,7 +83,6 @@ contains other variables that you can override according to your needs: - `TF_VAR_cluster_version`: Set the version of Kubernetes. - `TF_VAR_instance_type`: Set the instance type for the Kubernetes nodes. - `TF_VAR_instance_count`: Set the number of Kubernetes nodes. -- `TF_VAR_agent_version`: Set the version of the GitLab agent. - `TF_VAR_agent_namespace`: Set the Kubernetes namespace for the GitLab agent. View the [AWS Terraform provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) and the [Kubernetes Terraform provider](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs) documentation for further resource options. diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md index 07d0c722d8b..a23a9e7a6e5 100644 --- a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -41,7 +41,8 @@ Start by [importing the example project by URL](../../../project/import/repo_by_ To import the project: -1. On the top bar, select **Menu > Create new project**. +1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Import project**. 1. Select **Repository by URL**. 1. For the **Git repository URL**, enter `https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-gke.git`. @@ -107,7 +108,6 @@ contains other variables that you can override according to your needs: - `TF_VAR_cluster_description`: Set a description for the cluster. We recommend setting this to `$CI_PROJECT_URL` to create a reference to your GitLab project on your GCP cluster detail page. This way you know which project was responsible for provisioning the cluster you see on the GCP dashboard. - `TF_VAR_machine_type`: Set the machine type for the Kubernetes nodes. - `TF_VAR_node_count`: Set the number of Kubernetes nodes. -- `TF_VAR_agent_version`: Set the version of the GitLab agent. - `TF_VAR_agent_namespace`: Set the Kubernetes namespace for the GitLab agent. Refer to the [Google Terraform provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference) and the [Kubernetes Terraform provider](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs) documentation for further resource options. diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md index 51fd626ce0f..5f77b7e402a 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md @@ -30,8 +30,8 @@ And update the `applications/cert-manager/helmfile.yaml` with a valid email addr ``` NOTE: -If your Kubernetes version is earlier than 1.20 and you are -[migrating from GitLab Managed Apps to a cluster management project](../../../../clusters/migrating_from_gma_to_project_template.md), +If your Kubernetes version is earlier than 1.20 and you are +[migrating from GitLab Managed Apps to a cluster management project](../../../../clusters/migrating_from_gma_to_project_template.md), then you can instead use `- path: applications/cert-manager-legacy/helmfile.yaml` to take over an existing release of cert-manager v0.10. diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md index 422552c5b71..d5eabb9ba46 100644 --- a/doc/user/infrastructure/iac/index.md +++ b/doc/user/infrastructure/iac/index.md @@ -64,7 +64,7 @@ In each GitLab major release (for example, 15.0), the latest templates replace t To use a Terraform template: -1. On the top bar, select **Menu > Projects** and find the project you want to integrate with Terraform. +1. On the top bar, select **Main menu > Projects** and find the project you want to integrate with Terraform. 1. On the left sidebar, select **Repository > Files**. 1. Edit your `.gitlab-ci.yml` file, use the `include` attribute to fetch the Terraform template: diff --git a/doc/user/infrastructure/iac/terraform_state.md b/doc/user/infrastructure/iac/terraform_state.md index 4e78e0bbed5..d4fea6b7dba 100644 --- a/doc/user/infrastructure/iac/terraform_state.md +++ b/doc/user/infrastructure/iac/terraform_state.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2673) in GitLab 13.0. Terraform uses state files to store details about your infrastructure configuration. -With Terraform remote [backends](https://www.terraform.io/language/settings/backends), +With Terraform remote [backends](https://www.terraform.io/language/settings/backends/configuration), you can store the state file in a remote and shared store. GitLab provides a [Terraform HTTP backend](https://www.terraform.io/language/settings/backends/http) @@ -109,7 +109,7 @@ inconsistent. Instead, use a remote storage resource. [initialized for CI/CD](#initialize-a-terraform-state-as-a-backend-by-using-gitlab-cicd). 1. Copy a pre-populated Terraform `init` command: - 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Infrastructure > Terraform**. 1. Next to the environment you want to use, select **Actions** (**{ellipsis_v}**) and select **Copy Terraform init command**. @@ -287,7 +287,7 @@ To read the Terraform state in the target project, you need at least the Develop To view Terraform state files: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Infrastructure > Terraform**. [An epic exists](https://gitlab.com/groups/gitlab-org/-/epics/4563) to track improvements to this UI. diff --git a/doc/user/infrastructure/iac/troubleshooting.md b/doc/user/infrastructure/iac/troubleshooting.md index e187fe54136..3286b550507 100644 --- a/doc/user/infrastructure/iac/troubleshooting.md +++ b/doc/user/infrastructure/iac/troubleshooting.md @@ -97,9 +97,7 @@ As a result, to create a plan and later use the same plan in another CI job, you `Error: Error acquiring the state lock` errors when using `-backend-config=password=$CI_JOB_TOKEN`. This happens because the value of `$CI_JOB_TOKEN` is only valid for the duration of the current job. -Another possible error message for the same problem could be: `Error: Error loading state: HTTP remote state endpoint requires auth`. - -As a workaround, use [http backend configuration variables](https://www.terraform.io/docs/language/settings/backends/http.html#configuration-variables) in your CI job, +As a workaround, use [http backend configuration variables](https://www.terraform.io/language/settings/backends/http#configuration-variables) in your CI job, which is what happens behind the scenes when following the [Get started using GitLab CI](terraform_state.md#initialize-a-terraform-state-as-a-backend-by-using-gitlab-cicd) instructions. @@ -112,8 +110,8 @@ If you don't set `TF_STATE_NAME` or `TF_ADDRESS` in your job, the job fails with To resolve this, ensure that either `TF_ADDRESS` or `TF_STATE_NAME` is accessible in the job that returned the error: -1. Configure the [CI/CD environment scope](../../../ci/variables/#add-a-cicd-variable-to-a-project) for the job. -1. Set the job's [environment](../../../ci/yaml/#environment), matching the environment scope from the previous step. +1. Configure the [CI/CD environment scope](../../../ci/variables/index.md#add-a-cicd-variable-to-a-project) for the job. +1. Set the job's [environment](../../../ci/yaml/index.md#environment), matching the environment scope from the previous step. ### Error refreshing state: HTTP remote state endpoint requires auth diff --git a/doc/user/instance/clusters/index.md b/doc/user/instance/clusters/index.md index a5c1402b9ec..27aa3479b88 100644 --- a/doc/user/instance/clusters/index.md +++ b/doc/user/instance/clusters/index.md @@ -21,7 +21,7 @@ projects. To view the instance level Kubernetes clusters: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Kubernetes**. ## Cluster precedence diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 6a524fe206a..d61190fbd31 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -346,10 +346,13 @@ backslash `\`. Otherwise the diff highlight does not render correctly: [View this topic in GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md#math). Math written in LaTeX syntax is rendered with [KaTeX](https://github.com/KaTeX/KaTeX). +_KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX._ +This syntax also works for the Asciidoctor `:stem: latexmath`. For details, see +the [Asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activating-stem-support). -Math written between dollar signs `$` is rendered inline with the text. Math written -in a [code block](#code-spans-and-blocks) with the language declared as `math` is rendered -on a separate line: +Math written between dollar signs with backticks (``$`...`$``) is rendered +inline with the text. Math written in a [code block](#code-spans-and-blocks) with +the language declared as `math` is rendered on a separate line: ````markdown This math is inline: $`a^2+b^2=c^2`$. @@ -369,10 +372,44 @@ This math is on a separate line: a^2+b^2=c^2 ``` -_KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX._ +#### LaTeX-compatible fencing -This syntax also works for the Asciidoctor `:stem: latexmath`. For details, see -the [Asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activating-stem-support). +> Introduced in GitLab 15.4 [with a flag](../administration/feature_flags.md) named `markdown_dollar_math`. Disabled by default. + +[View this topic in GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md#latex-compatible-fencing). + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `markdown_dollar_math`. +On GitLab.com, this feature is available. +The feature is not ready for production use. + +Math written between dollar signs (`$...$`) is rendered +inline with the text. Math written between double dollar signs (`$$...$$`) is rendered +on a separate line: + +````markdown +This math is inline: $a^2+b^2=c^2$. + +This math is on a separate line: $$a^2+b^2=c^2$$ + +This math is on a separate line: + +$$ +a^2+b^2=c^2 +$$ +```` + +<!-- Uncomment the example below when the flag is enabled on GitLab.com --> +<!-- This math is inline: $a^2+b^2=c^2$. + +This math is on a separate line: $$a^2+b^2=c^2$$ + +This math is on a separate line: + +$$ +a^2+b^2=c^2 +$$ --> ### Task lists @@ -611,9 +648,10 @@ Quote break. If this section isn't rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md#multiline-blockquote). GitLab Flavored Markdown extends the standard Markdown by also supporting multi-line blockquotes -fenced by `>>>`: +fenced by `>>>`, with a blank line before and after the block: ```markdown + >>> If you paste a message from somewhere else @@ -621,6 +659,7 @@ that spans multiple lines, you can quote that without having to manually prepend `>` to every line! >>> + ``` >>> @@ -772,6 +811,8 @@ Combined emphasis with **asterisks and _underscores_**. Strikethrough uses two tildes. ~~Scratch this.~~ ``` +<!-- markdownlint-disable MD050 --> + Emphasis, aka italics, with *asterisks* or _underscores_. Strong emphasis, aka bold, with double **asterisks** or __underscores__. @@ -780,6 +821,8 @@ Combined emphasis with **asterisks and _underscores_**. Strikethrough uses two tildes. ~~Scratch this.~~ +<!-- markdownlint-enable MD050 --> + #### Multiple underscores in words and mid-word emphasis If this section isn't rendered correctly, @@ -1471,7 +1514,9 @@ Press <kbd>Enter</kbd> to go to the next page. ### Tables -Tables are not part of the core Markdown spec, but they are part of GitLab Flavored Markdown. +Tables are not part of the core Markdown specification, but are part of GitLab Flavored Markdown. + +#### Markdown 1. The first line contains the headers, separated by "pipes" (`|`). 1. The second line separates the headers from the cells. @@ -1547,12 +1592,12 @@ but they do not render properly on `docs.gitlab.com`: | cell 3 | <ul><li> - [ ] Task one </li><li> - [ ] Task two </li></ul> | ``` -#### Copy from spreadsheet and paste in Markdown +##### Copy and paste from a spreadsheet > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27205) in GitLab 12.7. If you're working in spreadsheet software (for example, Microsoft Excel, Google -Sheets, or Apple Numbers), GitLab creates a Markdown table when you copy-and-paste +Sheets, or Apple Numbers), GitLab creates a Markdown table when you copy and paste from a spreadsheet. For example, suppose you have the following spreadsheet: @@ -1563,6 +1608,160 @@ entry and paste the spreadsheet:  +#### JSON + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86353) in GitLab 15.3. + +To render tables with JSON code blocks, use the following syntax: + +````markdown +```json:table +{} +``` +```` + +Watch the following video walkthrough of this feature: + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=12yWKw1AdKY">Demo: JSON Tables in Markdown</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/12yWKw1AdKY" frameborder="0" allowfullscreen="true"> </iframe> +</figure> + +The `items` attribute is a list of objects representing the data points. + +````markdown +```json:table +{ + "items" : [ + {"a": "11", "b": "22", "c": "33"} + ] +} +``` +```` + +To specify the table labels, use the `fields` attribute. + +````markdown +```json:table +{ + "fields" : ["a", "b", "c"], + "items" : [ + {"a": "11", "b": "22", "c": "33"} + ] +} +``` +```` + +Not all elements of `items` must have corresponding values in `fields`. + +````markdown +```json:table +{ + "fields" : ["a", "b", "c"], + "items" : [ + {"a": "11", "b": "22", "c": "33"}, + {"a": "211", "c": "233"} + ] +} +``` +```` + +When `fields` is not explicitly specified, the labels are picked from the first element of `items`. + +````markdown +```json:table +{ + "items" : [ + {"a": "11", "b": "22", "c": "33"}, + {"a": "211", "c": "233"} + ] +} +``` +```` + +You can specify custom labels for `fields`. + +````markdown +```json:table +{ + "fields" : [ + {"key": "a", "label": "AA"}, + {"key": "b", "label": "BB"}, + {"key": "c", "label": "CC"} + ], + "items" : [ + {"a": "11", "b": "22", "c": "33"}, + {"a": "211", "b": "222", "c": "233"} + ] +} +``` +```` + +You can enable sorting for individual elements of `fields`. + +````markdown +```json:table +{ + "fields" : [ + {"key": "a", "label": "AA", "sortable": true}, + {"key": "b", "label": "BB"}, + {"key": "c", "label": "CC"} + ], + "items" : [ + {"a": "11", "b": "22", "c": "33"}, + {"a": "211", "b": "222", "c": "233"} + ] +} +``` +```` + +You can use the `filter` attribute to render a table with content filtered dynamically by user input. + +````markdown +```json:table +{ + "fields" : [ + {"key": "a", "label": "AA"}, + {"key": "b", "label": "BB"}, + {"key": "c", "label": "CC"} + ], + "items" : [ + {"a": "11", "b": "22", "c": "33"}, + {"a": "211", "b": "222", "c": "233"} + ], + "filter" : true +} +``` +```` + +By default, every JSON table has the caption `Generated with JSON data`. +You can override this caption by specifying the `caption` attribute. + +````markdown +```json:table +{ + "items" : [ + {"a": "11", "b": "22", "c": "33"} + ], + "caption" : "Custom caption" +} +``` +```` + +If JSON is invalid, an error occurs. + +````markdown +```json:table +{ + "items" : [ + {"a": "11", "b": "22", "c": "33"} + ], +} +``` +```` + ## References - This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). diff --git a/doc/user/namespace/index.md b/doc/user/namespace/index.md index 9ffc65a4e8e..138b0a355ad 100644 --- a/doc/user/namespace/index.md +++ b/doc/user/namespace/index.md @@ -6,11 +6,23 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Namespaces -In GitLab, a *namespace* organizes related projects together. +In GitLab, a *namespace* provides one place to organize your related projects. Projects in one namespace are separate from projects in other namespaces, +which means you can use the same name for projects in different namespaces. + GitLab has two types of namespaces: -- A *personal* namespace, which is based on your username. Projects under a personal namespace must be configured one at a time. -- A *group* or *subgroup* namespace. In these namespaces, you can manage multiple projects at once. +- A *Personal* namespace, which is based on your username and provided to you when you create your account. + - If you change your username, the project and namespace URLs in your account also change. Before you change your username, + read about [repository redirects](../project/repository/index.md#what-happens-when-a-repository-path-changes). + - You cannot create subgroups in a personal namespace. + - Groups in your namespace do not inherit your namespace permissions and group features. + - All the *Personal Projects* created will fall under the scope of this namespace. + +- A *group* or *subgroup* namespace: + - You can create multiple subgroups to manage multiple projects. + - You can change the URL of group and subgroup namespaces. + - You can configure settings specifically for each subgroup and project in the namespace. + - When you create a subgroup, it inherits some of the parent group settings. You can view these in the subgroup **Settings**. To determine whether you're viewing a group or personal namespace, you can view the URL. For example: diff --git a/doc/user/operations_dashboard/index.md b/doc/user/operations_dashboard/index.md index 1b14582a3f0..0a7e9d6395b 100644 --- a/doc/user/operations_dashboard/index.md +++ b/doc/user/operations_dashboard/index.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w The Operations Dashboard provides a summary of each project's operational health, including pipeline and alert status. -To access the dashboard, on the top bar, select **Menu > Operations**. +To access the dashboard, on the top bar, select **Main menu > Operations**. ## Adding a project to the dashboard diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index 4fc55d18253..84f544ec4ad 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -127,7 +127,7 @@ To publish the package with a deploy token: - `<tag>` is the Git tag name of the version you want to publish. To publish a branch, use `branch=<branch>` instead of `tag=<tag>`. -You can view the published package by going to **Packages & Registries > Package Registry** and +You can view the published package by going to **Packages and registries > Package Registry** and selecting the **Composer** tab. ## Publish a Composer package by using CI/CD @@ -145,11 +145,12 @@ You can publish a Composer package to the Package Registry as part of your CI/CD script: - apk add curl - 'curl --header "Job-Token: $CI_JOB_TOKEN" --data tag=<tag> "${CI_API_V4_URL}/projects/$CI_PROJECT_ID/packages/composer"' + environment: production ``` 1. Run the pipeline. -To view the published package, go to **Packages & Registries > Package Registry** and select the **Composer** tab. +To view the published package, go to **Packages and registries > Package Registry** and select the **Composer** tab. ### Use a CI/CD template diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index 7260dbb616c..ed106685b62 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -310,6 +310,7 @@ create_package: - conan new <package-name>/0.1 -t - conan create . <group-name>+<project-name>/stable - CONAN_LOGIN_USERNAME=ci_user CONAN_PASSWORD=${CI_JOB_TOKEN} conan upload <package-name>/0.1@<group-name>+<project-name>/stable --all --remote=gitlab + environment: production ``` Additional Conan images to use as the basis of your CI file are available in the @@ -389,7 +390,7 @@ There are two ways to remove a Conan package from the GitLab Package Registry. - From the GitLab user interface: - Go to your project's **Packages & Registries > Package Registry**. Remove the + Go to your project's **Packages and registries > Package Registry**. Remove the package by selecting **Remove repository** (**{remove}**). ## Search for Conan packages in the Package Registry diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index a203de2ed2c..bfbc400f4dd 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -26,7 +26,7 @@ Registry for your GitLab instance, visit the You can view the Container Registry for a project or group. 1. Go to your project or group. -1. Go to **Packages & Registries > Container Registry**. +1. Go to **Packages and registries > Container Registry**. You can search, sort, filter, and [delete](#delete-images-from-within-gitlab) containers on this page. You can share a filtered view by copying the URL from your browser. @@ -40,7 +40,7 @@ If a project is public, so is the Container Registry. You can view a list of tags associated with a given container image: 1. Go to your project or group. -1. Go to **Packages & Registries > Container Registry**. +1. Go to **Packages and registries > Container Registry**. 1. Select the container image you are interested in. This brings up the Container Registry **Tag Details** page. You can view details about each tag, @@ -55,7 +55,7 @@ tags on this page. You can share a filtered view by copying the URL from your br To download and run a container image hosted in the GitLab Container Registry: 1. Copy the link to your container image: - - Go to your project or group's **Packages & Registries > Container Registry** + - Go to your project or group's **Packages and registries > Container Registry** and find the image you want. - Next to the image name, select **Copy**. @@ -67,6 +67,8 @@ To download and run a container image hosted in the GitLab Container Registry: docker run [options] registry.example.com/group/project/image [arguments] ``` +[Authentication](#authenticate-with-the-container-registry) is needed to download images from private repository. + For more information on running Docker containers, visit the [Docker documentation](https://docs.docker.com/engine/userguide/intro/). @@ -97,15 +99,9 @@ registry.example.com/mynamespace/myproject/image:latest registry.example.com/mynamespace/myproject/my/image:rc1 ``` -## Build and push images by using Docker commands - -To build and push to the Container Registry, you can use Docker commands. - -### Authenticate with the Container Registry +## Authenticate with the Container Registry -Before you can build and push images, you must authenticate with the Container Registry. - -To authenticate, you can use: +To authenticate with the Container Registry, you can use: - A [personal access token](../../profile/personal_access_tokens.md). - A [deploy token](../../project/deploy_tokens/index.md). @@ -121,7 +117,9 @@ To authenticate, run the `docker` command. For example: docker login registry.example.com -u <username> -p <token> ``` -### Build and push images by using Docker commands +## Build and push images by using Docker commands + +Before you can build and push images, you must [authenticate](#authenticate-with-the-container-registry) with the Container Registry. To build and push to the Container Registry: @@ -139,7 +137,7 @@ To build and push to the Container Registry: docker push registry.example.com/group/project/image ``` -To view these commands, go to your project's **Packages & Registries > Container Registry**. +To view these commands, go to your project's **Packages and registries > Container Registry**. ## Build and push by using GitLab CI/CD @@ -301,6 +299,7 @@ deploy: - ./deploy.sh only: - main + environment: production ``` NOTE: @@ -394,7 +393,7 @@ images), are automatically scheduled for deletion after 24 hours if left unrefer To delete images from within GitLab: -1. Navigate to your project's or group's **Packages & Registries > Container Registry**. +1. Navigate to your project's or group's **Packages and registries > Container Registry**. 1. From the **Container Registry** page, you can select what you want to delete, by either: @@ -508,7 +507,7 @@ You can, however, remove the Container Registry for a project: and disable **Container Registry**. 1. Select **Save changes**. -The **Packages & Registries > Container Registry** entry is removed from the project's sidebar. +The **Packages and registries > Container Registry** entry is removed from the project's sidebar. ## Change visibility of the Container Registry diff --git a/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md b/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md index 8f90f42ad08..76e3da9538f 100644 --- a/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md +++ b/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md @@ -125,7 +125,7 @@ upgrading to [GitLab Premium or Ultimate](https://about.gitlab.com/upgrade/). ## Purchase additional data transfer -Read more about managing your [data transfer limits](../../../subscriptions/gitlab_com/#purchase-more-storage-and-transfer). +Read more about managing your [data transfer limits](../../../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer). ## Related issues diff --git a/doc/user/packages/container_registry/reduce_container_registry_storage.md b/doc/user/packages/container_registry/reduce_container_registry_storage.md index 788e57b6410..b3dd8da9b41 100644 --- a/doc/user/packages/container_registry/reduce_container_registry_storage.md +++ b/doc/user/packages/container_registry/reduce_container_registry_storage.md @@ -113,7 +113,7 @@ You can create a cleanup policy in [the API](#use-the-cleanup-policy-api) or the To create a cleanup policy in the UI: -1. For your project, go to **Settings > Packages & Registries**. +1. For your project, go to **Settings > Packages and registries**. 1. Expand the **Clean up image tags** section. 1. Complete the fields. @@ -206,7 +206,7 @@ For self-managed instances, those settings can be updated in the [Rails console] They are also available in the [administrator area](../../admin_area/index.md): -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. Go to **Settings > CI/CD > Container Registry**. ### Use the cleanup policy API @@ -219,7 +219,7 @@ Examples: ```shell curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" \ - --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":"","name_regex_delete":".*","name_regex_keep":".*-main"}}' \ + --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":".*","name_regex_keep":".*-main"}}' \ "https://gitlab.example.com/api/v4/projects/2" ``` diff --git a/doc/user/packages/debian_repository/index.md b/doc/user/packages/debian_repository/index.md index 1d846a60281..4143ab0881f 100644 --- a/doc/user/packages/debian_repository/index.md +++ b/doc/user/packages/debian_repository/index.md @@ -175,3 +175,43 @@ To install a package: ```shell sudo apt-get -y install -t <codename> <package-name> ``` + +## Download a source package + +To download a source package: + +1. Configure the repository: + + If you are using a private project, add your [credentials](#authenticate-to-the-package-registry) to your apt configuration: + + ```shell + echo 'machine gitlab.example.com login <username> password <your_access_token>' \ + | sudo tee /etc/apt/auth.conf.d/gitlab_project.conf + ``` + + Download your distribution key: + + ```shell + sudo mkdir -p /usr/local/share/keyrings + curl --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/<project_id>/debian_distributions/<codename>/key.asc" \ + | \ + gpg --dearmor \ + | \ + sudo tee /usr/local/share/keyrings/<codename>-archive-keyring.gpg \ + > /dev/null + ``` + + Add your project as a source: + + ```shell + echo 'deb-src [ signed-by=/usr/local/share/keyrings/<codename>-archive-keyring.gpg ] https://gitlab.example.com/api/v4/projects/<project_id>/packages/debian <codename> <component1> <component2>' \ + | sudo tee /etc/apt/sources.list.d/gitlab_project-sources.list + sudo apt-get update + ``` + +1. Download the source package: + + ```shell + sudo apt-get source -t <codename> <package-name> + ``` diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index b570bba73e5..1310f8eedaa 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -37,8 +37,8 @@ For a list of planned additions, view the To enable or turn off the Dependency Proxy for a group: -1. On the top bar, select **Menu > Groups** and find your group. -1. On the left sidebar, select **Settings > Packages & Registries**. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > Packages and registries**. 1. Expand the **Dependency Proxy** section. 1. To enable the proxy, turn on **Enable Proxy**. To turn it off, turn the toggle off. @@ -50,8 +50,8 @@ for the entire GitLab instance. To view the Dependency Proxy: -1. On the top bar, select **Menu > Groups** and find your group. -1. On the left sidebar, select **Packages & Registries > Dependency Proxy**. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Packages and registries > Dependency Proxy**. The Dependency Proxy is not available for projects. @@ -175,8 +175,8 @@ You can also use [custom CI/CD variables](../../../ci/variables/index.md#custom- To store a Docker image in Dependency Proxy storage: -1. On the top bar, select **Menu > Groups** and find your group. -1. On the left sidebar, select **Packages & Registries > Dependency Proxy**. +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Packages and registries > Dependency Proxy**. 1. Copy the **Dependency Proxy image prefix**. 1. Use one of these commands. In these examples, the image is `alpine:latest`. 1. You can also pull images by digest to specify exactly which version of an image to pull. diff --git a/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md b/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md index 839684da875..fecf60feeef 100644 --- a/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md +++ b/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md @@ -33,7 +33,7 @@ image or tag from Docker Hub. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340777) in GitLab 14.6 You can enable an automatic time-to-live (TTL) policy for the Dependency Proxy from the user -interface. To do this, navigate to your group's **Settings > Packages & Registries > Dependency Proxy** +interface. To do this, navigate to your group's **Settings > Packages and registries > Dependency Proxy** and enable the setting to automatically clear items from the cache after 90 days. ### Enable cleanup policies with GraphQL diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index d4acb14b9ca..312a2c119d6 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -123,7 +123,7 @@ or the UI. In the UI: -1. For your group, go to **Settings > Packages & Registries**. +1. For your group, go to **Settings > Packages and registries**. 1. Expand the **Package Registry** section. 1. Turn on the **Reject duplicates** toggle. 1. Optional. To allow some duplicate packages, in the **Exceptions** box enter a regex pattern that @@ -215,7 +215,7 @@ It also demonstrates how to manage a semantic version for the generic package: s ### Internal Server error on large file uploads to S3 -S3-compatible object storage [limits the size of a single PUT request to 5GB](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html). If the `aws_signature_version` is set to `2` in the [object storage connection settings](../../../administration/object_storage.md), attempting to publish a package file larger than the 5GB limit can result in a `HTTP 500: Internal Server Error` response. +S3-compatible object storage [limits the size of a single PUT request to 5GB](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html). If the `aws_signature_version` is set to `2` in the [object storage connection settings](../../../administration/object_storage.md), attempting to publish a package file larger than the 5GB limit can result in a `HTTP 500: Internal Server Error` response. If you are receiving `HTTP 500: Internal Server Error` responses when publishing large files to S3, set the `aws_signature_version` to `4`: diff --git a/doc/user/packages/harbor_container_registry/index.md b/doc/user/packages/harbor_container_registry/index.md new file mode 100644 index 00000000000..720e274aee5 --- /dev/null +++ b/doc/user/packages/harbor_container_registry/index.md @@ -0,0 +1,69 @@ +--- +stage: Package +group: Package +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Harbor Registry **(FREE)** + +You can integrate the [Harbor container registry](../../../user/project/integrations/harbor.md#harbor-container-registry-integration) into GitLab and use Harbor as the container registry for your GitLab project to store images. + +## View the Harbor Registry + +You can view the Harbor Registry for a project or group. + +1. On the top bar, select **Main menu > Projects/Groups**. +1. Go to the project or group that you are interested in. +1. On the left sidebar, select **Packages and registries > Harbor Registry**. + +You can search, sort, and filter images on this page. You can share a filtered view by copying the URL from your browser. + +At the project level, you can see **CLI Commands** in the upper right corner, where you can copy +corresponding commands to log in, build images, and push images. **CLI Commands** is not shown at +the group level. + +NOTE: +Default settings for the Harbor integration at the project level are inherited from the group level. + +## Use images from the Harbor Registry + +To download and run a Harbor image hosted in the GitLab Harbor Registry: + +1. Copy the link to your container image: + 1. Go to your project or group's **Packages and registries > Harbor Registry** and find the image you want. + 1. Click the **Copy** icon next to the image name. + +1. Use the command to run the container image you want. + +## View the tags of a specific artifact + +To view the list of tags associated with a specific artifact: + +1. Go to your project or group. +1. Go to **Packages and registries > Harbor Registry**. +1. Click the image name to view its artifacts. +1. Select the artifact you want. + +This brings up the list of tags. You can view the tag count and the time published. + +You can also copy the tag URL and use it to pull the corresponding artifact. + +## Build and push images by using commands + +To build and push to the Harbor Registry: + +1. Authenticate with the Harbor Registry. +1. Run the command to build or push. + +To view these commands, go to your project's **Packages and registries > Harbor Registry > CLI Commands**. + +## Disable the Harbor Registry for a project + +To remove the Harbor Registry for a project: + +1. Go to your project/group's **Settings > Integrations** page. +1. Click **Harbor** under **Active integrations**. +1. Clear the **Active** checkbox under **Enable integration**. +1. Select **Save changes**. + +The **Packages and registries > Harbor Registry** entry is removed from the sidebar. diff --git a/doc/user/packages/infrastructure_registry/index.md b/doc/user/packages/infrastructure_registry/index.md index e6a179c9d12..48cc7b9dea9 100644 --- a/doc/user/packages/infrastructure_registry/index.md +++ b/doc/user/packages/infrastructure_registry/index.md @@ -18,7 +18,7 @@ projects. To view packages within your project: 1. Go to the project. -1. Go to **Packages & Registries > Infrastructure Registry**. +1. Go to **Packages and registries > Infrastructure Registry**. You can search, sort, and filter packages on this page. @@ -49,7 +49,7 @@ You can see the pipeline that published the package as well as the commit and th To download a package: -1. Go to **Packages & Registries > Infrastructure Registry**. +1. Go to **Packages and registries > Infrastructure Registry**. 1. Select the name of the package you want to download. 1. In the **Activity** section, select the name of the package you want to download. @@ -64,7 +64,7 @@ You can delete packages by using [the API](../../../api/packages.md#delete-a-pro To delete a package in the UI, from your project: -1. Go to **Packages & Registries > Infrastructure Registry**. +1. Go to **Packages and registries > Infrastructure Registry**. 1. Find the name of the package you want to delete. 1. Select **Delete**. @@ -75,7 +75,7 @@ The package is permanently deleted. The Infrastructure Registry is automatically enabled. For self-managed instances, a GitLab administrator can -[disable](../../../administration/packages/index.md) **Packages & Registries**, +[disable](../../../administration/packages/index.md) **Packages and registries**, which removes this menu item from the sidebar. You can also remove the Infrastructure Registry for a specific project: diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index eaa04404a1f..957374245d2 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -605,7 +605,7 @@ To publish a package by using Gradle: gradle publish ``` -Now navigate to your project's **Packages & Registries** page and view the published artifacts. +Now navigate to your project's **Packages and registries** page and view the published artifacts. ### Publishing a package with the same name or version @@ -624,7 +624,7 @@ To prevent users from publishing duplicate Maven packages, you can use the [Grap In the UI: -1. For your group, go to **Settings > Packages & Registries**. +1. For your group, go to **Settings > Packages and registries**. 1. Expand the **Package Registry** section. 1. Turn on the **Reject duplicates** toggle. 1. Optional. To allow some duplicate packages, in the **Exceptions** box, enter a regex pattern that matches the names and/or versions of packages you want to allow. @@ -670,14 +670,20 @@ Downloading from gitlab-maven: http://gitlab.example.com/api/v4/projects/PROJECT ### Use Maven with `mvn dependency:get` -You can install packages by using the Maven commands directly. +You can install packages by using the Maven `dependency:get` [command](https://maven.apache.org/plugins/maven-dependency-plugin/get-mojo.html) directly. 1. In your project directory, run: ```shell - mvn dependency:get -Dartifact=com.nickkipling.app:nick-test-app:1.1-SNAPSHOT + mvn dependency:get -Dartifact=com.nickkipling.app:nick-test-app:1.1-SNAPSHOT -DremoteRepositories=gitlab-maven::::<gitlab endpoint url> -s <path to settings.xml> ``` + - `<gitlab endpoint url>` is the URL of the GitLab [endpoint](#use-the-gitlab-endpoint-for-maven-packages). + - `<path to settings.xml>` is the path to the `settings.xml` file that contains the [authentication details](#authenticate-to-the-package-registry-with-maven). + +NOTE: +The repository IDs in the command(`gitlab-maven`) and the `settings.xml` file must match. + The message should show that the package is downloading from the Package Registry: ```shell @@ -697,9 +703,70 @@ dependencies { } ``` +### Request forwarding to Maven Central + +> [Introduced](<https://gitlab.com/gitlab-org/gitlab/-/issues/362657>) behind a [feature flag](../../feature_flags.md), disabled by default in GitLab 15.4 + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `maven_central_request_forwarding`. +On GitLab.com, this feature is not available. + +When a Maven package is not found in the Package Registry, the request is forwarded +to [Maven Central](https://search.maven.org/). + +When the feature flag is enabled, administrators can disable this behavior in the +[Continuous Integration settings](../../admin_area/settings/continuous_integration.md). + +There are many ways to configure your Maven project so that it will request packages +in Maven Central from GitLab. Maven repositories are queried in a +[specific order](https://maven.apache.org/guides/mini/guide-multiple-repositories.html#repository-order). +By default, maven-central is usually checked first through the +[Super POM](https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM), so +GitLab needs to be configured to be queried before maven-central. + +[Using GitLab as a mirror of the central proxy](#setting-gitlab-as-a-mirror-for-the-central-proxy) is one +way to force GitLab to be queried in place of maven-central. + +Maven forwarding is restricted to only the [project level](#project-level-maven-endpoint) and +[group level](#group-level-maven-endpoint) endpoints. The [instance level endpoint](#instance-level-maven-endpoint) +has naming restrictions that prevent it from being used for packages that don't follow that convention and also +introduces too much security risk for supply-chain style attacks. + +#### Setting GitLab as a mirror for the central proxy + +To ensure all package requests are sent to GitLab instead of Maven Central, +you can override Maven Central as the central repository by adding a `<mirror>` +section to your `settings.xml`: + +```xml +<settings> + <servers> + <server> + <id>central-proxy</id> + <configuration> + <httpHeaders> + <property> + <name>Private-Token</name> + <value>{personal_access_token}</value> + </property> + </httpHeaders> + </configuration> + </server> + </servers> + <mirrors> + <mirror> + <id>central-proxy</id> + <name>GitLab proxy of central repo</name> + <url>https://gitlab.example.com/api/v4/projects/{project_id}/packages/maven</url> + <mirrorOf>central</mirrorOf> + </mirror> + </mirrors> +</settings> +``` + ## Remove a package -For your project, go to **Packages & Registries > Package Registry**. +For your project, go to **Packages and registries > Package Registry**. To remove a package, select the red trash icon or, from the package details, the **Delete** button. diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 28de06b2d8a..678f5681890 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -284,7 +284,7 @@ To upload an npm package to your project, run this command: npm publish ``` -To view the package, go to your project's **Packages & Registries**. +To view the package, go to your project's **Packages and registries**. You can also define `"publishConfig"` for your project in `package.json`. For example: @@ -325,6 +325,7 @@ deploy: script: - echo "//${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">.npmrc - npm publish + environment: production ``` See the diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index 2e182d20811..5b1e5bbd304 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -403,6 +403,7 @@ updated: - dotnet nuget push "bin/Release/*.nupkg" --source gitlab only: - main + environment: production ``` 1. Commit the changes and push it to your GitLab repository to trigger a new CI/CD build. diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index 7748780d0e4..fe19c549536 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -26,7 +26,7 @@ Learn how to use the GitLab Package Registry to build your own custom package wo You can view packages for your project or group. 1. Go to the project or group. -1. Go to **Packages & Registries > Package Registry**. +1. Go to **Packages and registries > Package Registry**. You can search, sort, and filter packages on this page. You can share your search results by copying and pasting the URL from your browser. @@ -99,7 +99,7 @@ For information on reducing your storage use for the Package Registry, see The Package Registry is automatically enabled. If you are using a self-managed instance of GitLab, your administrator can remove -the menu item, **Packages & Registries**, from the GitLab sidebar. For more information, +the menu item, **Packages and registries**, from the GitLab sidebar. For more information, see the [administration documentation](../../../administration/packages/index.md). You can also remove the Package Registry for your project specifically: @@ -109,7 +109,7 @@ You can also remove the Package Registry for your project specifically: **Packages** feature. 1. Select **Save changes**. -The **Packages & Registries > Package Registry** entry is removed from the sidebar. +The **Packages and registries > Package Registry** entry is removed from the sidebar. ## Package Registry visibility permissions diff --git a/doc/user/packages/package_registry/reduce_package_registry_storage.md b/doc/user/packages/package_registry/reduce_package_registry_storage.md index cd7dd062f60..d85992fe05d 100644 --- a/doc/user/packages/package_registry/reduce_package_registry_storage.md +++ b/doc/user/packages/package_registry/reduce_package_registry_storage.md @@ -7,12 +7,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reduce Package Registry Storage **(FREE)** Without cleanup, package registries become large over time. When a large number of packages and -their files are added: +their assets are added: - Fetching the list of packages becomes slower. - They take up a large amount of storage space on the server, impacting your [storage usage quota](../../usage_quotas.md). -We recommend deleting unnecessary packages and files. This page offers examples of how to do so. +We recommend deleting unnecessary packages and assets. This page offers examples of how to do so. ## Check Package Registry Storage Use @@ -29,40 +29,40 @@ You can delete packages by using [the API](../../../api/packages.md#delete-a-pro To delete a package in the UI, from your group or project: -1. Go to **Packages & Registries > Package Registry**. +1. Go to **Packages and registries > Package Registry**. 1. Find the name of the package you want to delete. 1. Select **Delete**. The package is permanently deleted. -## Delete files associated with a package +## Delete assets associated with a package -To delete package files, you must have suitable [permissions](../../permissions.md). +To delete package assets, you must have suitable [permissions](../../permissions.md). You can delete packages by using [the API](../../../api/packages.md#delete-a-package-file) or the UI. -To delete package files in the UI, from your group or project: +To delete package assets in the UI, from your group or project: -1. Go to **Packages & Registries > Package Registry**. +1. Go to **Packages and registries > Package Registry**. 1. Find the name of the package you want to delete. 1. Select the package to view additional details. -1. Find the name of the file you would like to delete. -1. Expand the ellipsis and select **Delete file**. +1. Find the name of the assets you would like to delete. +1. Expand the ellipsis and select **Delete asset**. -The package files are permanently deleted. +The package assets are permanently deleted. ## Cleanup policy > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346153) in GitLab 15.2. Depending on the number of packages to remove, the process of manually deleting the packages can take a long time to finish. -A cleanup policy defines a set of rules that, applied to a project, defines which package files you can automatically delete. +A cleanup policy defines a set of rules that, applied to a project, defines which package assets you can automatically delete. ### Enable the cleanup policy By default, the packages cleanup policy is disabled. To enable it: -1. Go to your project **Settings > Packages & Registries**. +1. Go to your project **Settings > Packages and registries**. 1. Expand **Manage storage used by package assets**. 1. Set the rules appropriately. @@ -73,7 +73,7 @@ To access these project settings, you must be at least a maintainer on the relat - `Number of duplicated assets to keep`. The number of duplicated assets to keep. Some package formats allow you to upload more than one copy of an asset. You can limit the number of duplicated assets to keep and automatically - delete the oldest files once the limit is reached. + delete the oldest assets once the limit is reached. ### Set cleanup limits to conserve resources diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index ba9ecbe50a3..302c88bf46f 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -309,7 +309,7 @@ Uploading mypypipackage-0.0.1.tar.gz 100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s] ``` -To view the published package, go to your project's **Packages & Registries** +To view the published package, go to your project's **Packages and registries** page. If you didn't use a `.pypirc` file to define your repository source, you can diff --git a/doc/user/packages/rubygems_registry/index.md b/doc/user/packages/rubygems_registry/index.md index 05113d0bc10..682a3e2ecf1 100644 --- a/doc/user/packages/rubygems_registry/index.md +++ b/doc/user/packages/rubygems_registry/index.md @@ -130,7 +130,7 @@ Pushing gem to https://gitlab.example.com/api/v4/projects/1/packages/rubygems... {"message":"201 Created"} ``` -To view the published gem, go to your project's **Packages & Registries** page. Gems pushed to +To view the published gem, go to your project's **Packages and registries** page. Gems pushed to GitLab aren't displayed in your project's Packages UI immediately. It can take up to 10 minutes to process a gem. diff --git a/doc/user/packages/terraform_module_registry/index.md b/doc/user/packages/terraform_module_registry/index.md index 436c55f9ee0..0a3de25bf7d 100644 --- a/doc/user/packages/terraform_module_registry/index.md +++ b/doc/user/packages/terraform_module_registry/index.md @@ -24,7 +24,7 @@ When you publish a Terraform Module, if it does not exist, it is created. Prerequisites: -- A package with the same name and version must not already exist. +- A package with the same name and version must not already exist in the top-level namespace. - Your project and group names must not include a dot (`.`). For example, `source = "gitlab.example.com/my.group/project.name"`. - You must [authenticate with the API](../../../api/index.md#authentication). If authenticating with a deploy token, it must be configured with the `write_package_registry` scope. diff --git a/doc/user/permissions.md b/doc/user/permissions.md index eca94fce2e9..9a4590d9478 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -88,6 +88,8 @@ The following table lists project permissions available for each role: | [Incident Management](../operations/incident_management/index.md):<br>View [escalation policies](../operations/incident_management/escalation_policies.md) | | ✓ | ✓ | ✓ | ✓ | | [Incident Management](../operations/incident_management/index.md):<br>Manage [on-call schedules](../operations/incident_management/oncall_schedules.md) | | | | ✓ | ✓ | | [Incident Management](../operations/incident_management/index.md):<br>Manage [escalation policies](../operations/incident_management/escalation_policies.md) | | | | ✓ | ✓ | +| [Issue boards](project/issue_board.md):<br>Create or delete lists | | ✓ | ✓ | ✓ | ✓ | +| [Issue boards](project/issue_board.md):<br>Move issues between lists | | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>Add Labels | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>Assign | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>Create (*18*) | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -95,7 +97,7 @@ The following table lists project permissions available for each role: | [Issues](project/issues/index.md):<br>View [Design Management](project/issues/design_management.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>View [related issues](project/issues/related_issues.md) | ✓ | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>Set [weight](project/issues/issue_weight.md) | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ | -| [Issues]](project/issues/index.md):<br>Set [parent epic](group/epics/manage_epics.md#add-an-existing-issue-to-an-epic) | | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Set [parent epic](group/epics/manage_epics.md#add-an-existing-issue-to-an-epic) | | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>View [confidential issues](project/issues/confidential_issues.md) | (*2*) | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>Close / reopen (*19*) | | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>Lock threads | | ✓ | ✓ | ✓ | ✓ | @@ -119,7 +121,7 @@ The following table lists project permissions available for each role: | [Merge requests](project/merge_requests/index.md):<br>Add labels | | | ✓ | ✓ | ✓ | | [Merge requests](project/merge_requests/index.md):<br>Lock threads | | | ✓ | ✓ | ✓ | | [Merge requests](project/merge_requests/index.md):<br>Manage or accept | | | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>[Resolve a thread](discussions/#resolve-a-thread) | | | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>[Resolve a thread](discussions/index.md#resolve-a-thread) | | | ✓ | ✓ | ✓ | | [Merge requests](project/merge_requests/index.md):<br>Manage [merge approval rules](project/merge_requests/approvals/settings.md) (project settings) | | | | ✓ | ✓ | | [Merge requests](project/merge_requests/index.md):<br>Delete | | | | | ✓ | | [Metrics dashboards](../operations/metrics/dashboards/index.md):<br>Manage user-starred metrics dashboards (*6*) | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -348,11 +350,6 @@ from pushing to a protected branch. Read through the documentation on Find the current permissions on the value stream analytics dashboard, as described in [related documentation](analytics/value_stream_analytics.md#access-permissions-for-value-stream-analytics). -### Issue board permissions - -Find the current permissions for interacting with the issue board feature in the -[issue boards permissions page](project/issue_board.md#permissions). - ### File Locking permissions **(PREMIUM)** The user that locks a file or directory is the only one that can edit and push their changes back to the repository where the locked objects are located. @@ -386,7 +383,7 @@ The following table lists group permissions available for each role: | Pull a container image using the dependency proxy | ✓ | ✓ | ✓ | ✓ | ✓ | | View Contribution analytics | ✓ | ✓ | ✓ | ✓ | ✓ | | View group [epic](group/epics/index.md) | ✓ | ✓ | ✓ | ✓ | ✓ | -| View [group wiki](project/wiki/group.md) pages | ✓ (6) | ✓ | ✓ | ✓ | ✓ | +| View [group wiki](project/wiki/group.md) pages | ✓ (5) | ✓ | ✓ | ✓ | ✓ | | View [Insights](project/insights/index.md) | ✓ | ✓ | ✓ | ✓ | ✓ | | View [Insights](project/insights/index.md) charts | ✓ | ✓ | ✓ | ✓ | ✓ | | View [Issue analytics](analytics/issue_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -398,13 +395,13 @@ The following table lists group permissions available for each role: | Pull [packages](packages/index.md) | | ✓ | ✓ | ✓ | ✓ | | Delete [packages](packages/index.md) | | | | ✓ | ✓ | | Create/edit/delete [Maven and generic package duplicate settings](packages/generic_packages/index.md#do-not-allow-duplicate-generic-packages) | | | | ✓ | ✓ | -| Pull a Container Registry image | ✓ (7) | ✓ | ✓ | ✓ | ✓ | +| Pull a Container Registry image | ✓ (6) | ✓ | ✓ | ✓ | ✓ | | Remove a Container Registry image | | | ✓ | ✓ | ✓ | | View [Group DevOps Adoption](group/devops_adoption/index.md) | | ✓ | ✓ | ✓ | ✓ | | View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | | View [Productivity analytics](analytics/productivity_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | Create and edit [group wiki](project/wiki/group.md) pages | | | ✓ | ✓ | ✓ | -| Create project in group | | | ✓ (3)(5) | ✓ (3) | ✓ (3) | +| Create project in group | | | ✓ (2)(4) | ✓ (2) | ✓ (2) | | Create/edit/delete group milestones | | ✓ | ✓ | ✓ | ✓ | | Create/edit/delete iterations | | ✓ | ✓ | ✓ | ✓ | | Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | @@ -412,10 +409,10 @@ The following table lists group permissions available for each role: | Purge the dependency proxy for a group | | | | | ✓ | | Create/edit/delete dependency proxy [cleanup policies](packages/dependency_proxy/reduce_dependency_proxy_storage.md#cleanup-policies) | | | | ✓ | ✓ | | Use [security dashboard](application_security/security_dashboard/index.md) | | | ✓ | ✓ | ✓ | -| View group Audit Events | | | ✓ (7) | ✓ (7) | ✓ | +| View group Audit Events | | | ✓ (6) | ✓ (6) | ✓ | | Create subgroup | | | | ✓ (1) | ✓ | | Delete [group wiki](project/wiki/group.md) pages | | | ✓ | ✓ | ✓ | -| Edit [epic](group/epics/index.md) comments (posted by any user) | | | | ✓ (2) | ✓ (2) | +| Edit [epic](group/epics/index.md) comments (posted by any user) | | | | ✓ | ✓ | | List group deploy tokens | | | | ✓ | ✓ | | Manage [group push rules](group/access_and_permissions.md#group-push-rules) | | | | ✓ | ✓ | | View/manage group-level Kubernetes cluster | | | | ✓ | ✓ | @@ -426,14 +423,14 @@ The following table lists group permissions available for each role: | Delete group [epic](group/epics/index.md) | | | | | ✓ | | Disable notification emails | | | | | ✓ | | Edit group settings | | | | | ✓ | -| Edit [SAML SSO](group/saml_sso/index.md) | | | | | ✓ (4) | +| Edit [SAML SSO](group/saml_sso/index.md) | | | | | ✓ (3) | | Filter members by 2FA status | | | | | ✓ | | Manage group level CI/CD variables | | | | | ✓ | | Manage group members | | | | | ✓ | | Share (invite) groups with groups | | | | | ✓ | | View 2FA status of members | | | | | ✓ | -| View [Billing](../subscriptions/gitlab_com/index.md#view-your-gitlab-saas-subscription) | | | | | ✓ (4) | -| View group [Usage Quotas](usage_quotas.md) page | | | | | ✓ (4) | +| View [Billing](../subscriptions/gitlab_com/index.md#view-your-gitlab-saas-subscription) | | | | | ✓ (3) | +| View group [Usage Quotas](usage_quotas.md) page | | | | | ✓ (3) | | Manage group runners | | | | | ✓ | | [Migrate groups](group/import/index.md) | | | | | ✓ | | Manage [subscriptions, and purchase CI/CD minutes and storage](../subscriptions/gitlab_com/index.md) | | | | | ✓ | @@ -441,14 +438,13 @@ The following table lists group permissions available for each role: <!-- markdownlint-disable MD029 --> 1. Groups can be set to allow either Owners, or Owners and users with the Maintainer role, to [create subgroups](group/subgroups/index.md#create-a-subgroup). -2. Introduced in GitLab 12.2. -3. Default project creation role can be changed at: +2. Default project creation role can be changed at: - The [instance level](admin_area/settings/visibility_and_access_controls.md#define-which-roles-can-create-projects). - The [group level](group/manage.md#specify-who-can-add-projects-to-a-group). -4. Does not apply to subgroups. -5. Developers can push commits to the default branch of a new project only if the [default branch protection](group/manage.md#change-the-default-branch-protection-of-a-group) is set to "Partially protected" or "Not protected". -6. In addition, if your group is public or internal, all users who can see the group can also see group wiki pages. -7. Users can only view events based on their individual actions. +3. Does not apply to subgroups. +4. Developers can push commits to the default branch of a new project only if the [default branch protection](group/manage.md#change-the-default-branch-protection-of-a-group) is set to "Partially protected" or "Not protected". +5. In addition, if your group is public or internal, all users who can see the group can also see group wiki pages. +6. Users can only view events based on their individual actions. <!-- markdownlint-enable MD029 --> @@ -470,7 +466,8 @@ project and should only have access to that project. External users: -- Can only create projects (including forks), subgroups, and snippets within the top-level group to which they belong. +- Cannot create project, groups, and snippets within their personal namespaces. +- Can only create projects (including forks), subgroups, and snippets within top-level groups to which they are explicitly granted access. - Can only access public projects and projects to which they are explicitly granted access, thus hiding all other internal or private ones from them (like being logged out). @@ -496,7 +493,7 @@ An administrator can flag a user as external by either of the following methods: - [Through the API](../api/users.md#user-modification). - Using the GitLab UI: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users** to create a new user or edit an existing one. There, you can find the option to flag the user as external. @@ -510,7 +507,7 @@ Additionally, users can be set as external users using: By default, new users are not set as external users. This behavior can be changed by an administrator: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Account and limit** section. diff --git a/doc/user/profile/account/create_accounts.md b/doc/user/profile/account/create_accounts.md index 694ed02a694..e3f7d47038d 100644 --- a/doc/user/profile/account/create_accounts.md +++ b/doc/user/profile/account/create_accounts.md @@ -33,7 +33,7 @@ Prerequisites: To create a user manually: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users** (`/admin/users`). 1. Select **New user**. 1. Complete the fields. diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index 5ec29814e06..d2e0c1ad834 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -28,7 +28,7 @@ As a user, to delete your own account: As an administrator, to delete a user account: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select a user. 1. Under the **Account** tab, select: diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 4c859d98004..e4cf905bbce 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -111,8 +111,8 @@ the README file with information, it's included on your profile page. To create a new project and add its README to your profile: -1. On the top bar, select **Menu > Project**. -1. Select **Create new project**. +1. On the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Create blank project**. 1. Enter the project details: - In the **Project name** field, enter the name for your new project. @@ -205,7 +205,7 @@ To set your current status: 1. Select a value from the **Clear status after** dropdown list. 1. Select **Set status**. Alternatively, you can select **Remove status** to remove your user status entirely. -You can also set your current status by [using the API](../../api/users.md#user-status). +You can also set your current status from [your user settings](#access-your-user-settings) or by [using the API](../../api/users.md#user-status). If you select the **Busy** checkbox, remember to clear it when you become available again. @@ -421,7 +421,7 @@ When you sign in to the main GitLab application, a `_gitlab_session` cookie is set. When you close your browser, the cookie is cleared client-side and it expires after a set duration. GitLab administrators can determine the duration: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Account and limit**. The set duration is in **Session duration (minutes)**. diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index 0245f0615e0..1b737f14f68 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -108,7 +108,7 @@ To select a notification level for a group, use either of these methods: Or: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. Select the notification dropdown, next to the bell icon (**{notifications}**). 1. Select the desired [notification level](#notification-levels). @@ -139,7 +139,7 @@ To select a notification level for a project, use either of these methods: Or: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Select the notification dropdown, next to the bell icon (**{notifications}**). 1. Select the desired [notification level](#notification-levels). diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 427c412219a..2fd18f583a4 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -19,6 +19,13 @@ Personal access tokens can be an alternative to [OAuth2](../../api/oauth2.md) an In both cases, you authenticate with a personal access token in place of your password. +WARNING: +The ability to create personal access tokens without expiry was +[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and is planned for removal in GitLab +16.0. When this ability is removed, existing personal access tokens without an expiry are planned to have an expiry added. +The automatic adding of an expiry occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry +occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. + Personal access tokens are: - Required when [two-factor authentication (2FA)](account/two_factor_authentication.md) is enabled. @@ -40,6 +47,8 @@ Use impersonation tokens to automate authentication as a specific user. ## Create a personal access token +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348660) in GitLab 15.3, default expiration of 30 days is populated in the UI. + You can create as many personal access tokens as you like. 1. In the top-right corner, select your avatar. diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index e99a2dad980..31ab802a8b8 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -116,12 +116,11 @@ between the fixed (max. `1280px`) and the fluid (`100%`) application layout. NOTE: While `1280px` is the standard max width when using fixed layout, some pages still use 100% width, depending on the content. -### Default dashboard +### Dashboard For users who have access to a large number of projects but only keep up with a -select few, the amount of activity on the default Dashboard page can be -overwhelming. Changing this setting allows you to redefine your default -dashboard. +select few, the amount of activity on the your dashboard can be +overwhelming. Changing this setting allows you to redefine what is displayed by default. You can include the following options for your default dashboard view: diff --git a/doc/user/project/badges.md b/doc/user/project/badges.md index 2f9e04fb828..cf0ff4ed8b9 100644 --- a/doc/user/project/badges.md +++ b/doc/user/project/badges.md @@ -21,7 +21,7 @@ If you find that you have to add the same badges to several projects, you may wa To add a new badge to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Badges**. 1. Under "Link", enter the URL that the badges should point to and under @@ -41,7 +41,7 @@ A common project badge presents the GitLab CI pipeline status. To add this badge to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Badges**. 1. Under **Name**, enter _Pipeline Status_. @@ -68,7 +68,7 @@ If you need individual badges for each project, either: To add a new badge to a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Badges**. 1. Under "Link", enter the URL that the badges should point to and under @@ -118,7 +118,7 @@ https://gitlab.example.com/<project_path>/-/raw/<default_branch>/my-image.svg To add a new badge to a group or project with a custom image: -1. On the top bar, select **Menu** and find your group or project. +1. On the top bar, select **Main menu** and find your group or project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Badges**. 1. Under **Name**, enter the name for the badge. diff --git a/doc/user/project/canary_deployments.md b/doc/user/project/canary_deployments.md index f8494116655..aac704e2cdd 100644 --- a/doc/user/project/canary_deployments.md +++ b/doc/user/project/canary_deployments.md @@ -38,8 +38,8 @@ want to make sure the performance stays the same, or improves. Developers need to be careful when using canaries with user-facing changes, because by default, requests from the same user are randomly distributed between canary and non-canary pods, which could result in confusion or even errors. If needed, you -may want to consider -[setting `service.spec.sessionAffinity` to `ClientIP` in your Kubernetes service definitions](https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies), +may want to consider +[setting `service.spec.sessionAffinity` to `ClientIP` in your Kubernetes service definitions](https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies), but that is beyond the scope of this document. ## Advanced traffic control with Canary Ingress diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index be73f2c9a01..d9339291328 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -56,7 +56,7 @@ cluster certificates: 1. Go to your: - Project's **Infrastructure > Kubernetes clusters** page, for a project-level cluster. - Group's **Kubernetes** page, for a group-level cluster. - - **Menu > Admin > Kubernetes**, for an instance-level cluster. + - **Main menu > Admin > Kubernetes**, for an instance-level cluster. 1. Select **Integrate with a cluster certificate**. 1. Under the **Create new cluster** tab, select **Amazon EKS** to display an `Account ID` and `External ID` needed for later steps. @@ -248,7 +248,7 @@ For example, the following policy document allows assuming a role whose name sta To configure Amazon authentication in GitLab, generate an access key for the IAM user in the Amazon AWS console, and follow these steps: -1. In GitLab, on the top bar, select **Menu > Admin > Settings > General** and expand the **Amazon EKS** section. +1. In GitLab, on the top bar, select **Main menu > Admin > Settings > General** and expand the **Amazon EKS** section. 1. Check **Enable Amazon EKS integration**. 1. Enter your **Account ID**. 1. Enter your [access key and ID](#eks-access-key-and-id). diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md index c55c11151ce..d7137c18a03 100644 --- a/doc/user/project/clusters/add_existing_cluster.md +++ b/doc/user/project/clusters/add_existing_cluster.md @@ -68,7 +68,7 @@ To add a Kubernetes cluster to your project, group, or instance: 1. Navigate to your: 1. Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. 1. Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. - 1. **Menu > Admin > Kubernetes** page, for an instance-level cluster. + 1. **Main menu > Admin > Kubernetes** page, for an instance-level cluster. 1. On the **Kubernetes clusters** page, select the **Connect with a certificate** option from the **Actions** dropdown menu. 1. On the **Connect a cluster** page, fill in the details: 1. **Kubernetes cluster name** (required) - The name you wish to give the cluster. diff --git a/doc/user/project/clusters/add_gke_clusters.md b/doc/user/project/clusters/add_gke_clusters.md index bfaf9aab7b7..6ed02838e9b 100644 --- a/doc/user/project/clusters/add_gke_clusters.md +++ b/doc/user/project/clusters/add_gke_clusters.md @@ -51,7 +51,7 @@ Note the following: cluster's pod address IP range is set to `/16` instead of the regular `/14`. `/16` is a CIDR notation. - GitLab requires basic authentication enabled and a client certificate issued for the cluster to - set up an [initial service account](cluster_access.md). In + set up an [initial service account](cluster_access.md). In [GitLab versions 11.10 and later](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/58208), the cluster creation process explicitly requests GKE to create clusters with basic authentication enabled and a client certificate. @@ -63,7 +63,7 @@ cluster certificates: - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. - - **Menu > Admin > Kubernetes** page, for an instance-level cluster. + - **Main menu > Admin > Kubernetes** page, for an instance-level cluster. 1. Select **Integrate with a cluster certificate**. 1. Under the **Create new cluster** tab, select **Google GKE**. 1. Connect your Google account if you haven't done already by clicking the diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index f1004a40a13..2fba00ae940 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -19,7 +19,7 @@ When you successfully connect an existing cluster using cluster certificates, th 1. Go to your: - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. - - **Menu > Admin > Kubernetes** page, for an instance-level cluster. + - **Main menu > Admin > Kubernetes** page, for an instance-level cluster. 1. Select the name of the cluster you want to disable. 1. Toggle **GitLab Integration** off (in gray). 1. Select **Save changes**. diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index f89d863e83b..940b58103f5 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -24,5 +24,5 @@ to a single project. To view project-level Kubernetes clusters: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. diff --git a/doc/user/project/code_intelligence.md b/doc/user/project/code_intelligence.md index 7f35caf2a68..860ebfbed14 100644 --- a/doc/user/project/code_intelligence.md +++ b/doc/user/project/code_intelligence.md @@ -17,7 +17,10 @@ development environments (IDE), including: Code Intelligence is built into GitLab and powered by [LSIF](https://lsif.dev/) (Language Server Index Format), a file format for precomputed code -intelligence data. +intelligence data. GitLab processes one LSIF file per project, and +Code Intelligence does not support different LSIF files per branch. +Follow epic [#4212, Code intelligence enhancements](https://gitlab.com/groups/gitlab-org/-/epics/4212) +for progress on upcoming enhancements. NOTE: You can automate this feature in your applications by using [Auto DevOps](../../topics/autodevops/index.md). @@ -59,13 +62,5 @@ under the **References** tab: ## Language support Generating an LSIF file requires a language server indexer implementation for the -relevant language. - -| Language | Implementation | -|---|---| -| Go | [`sourcegraph/lsif-go`](https://github.com/sourcegraph/lsif-go) | -| JavaScript | [`sourcegraph/lsif-node`](https://github.com/sourcegraph/lsif-node) | -| TypeScript | [`sourcegraph/lsif-node`](https://github.com/sourcegraph/lsif-node) | - -View a complete list of [available LSIF indexers](https://lsif.dev/#implementations-server) on their website and +relevant language. View a complete list of [available LSIF indexers](https://lsif.dev/#implementations-server) on their website and refer to their documentation to see how to generate an LSIF file for your specific language. diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 41afbdada6b..63010610605 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -116,7 +116,7 @@ To display the deploy boards for a specific [environment](../../ci/environments/ Kubernetes. NOTE: - Matching based on the Kubernetes `app` label was removed in + Matching based on the Kubernetes `app` label was removed in [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/14020). To migrate, please apply the required annotations (see above) and re-deploy your application. If you are using Auto DevOps, this will diff --git a/doc/user/project/deploy_keys/index.md b/doc/user/project/deploy_keys/index.md index a9f19d27416..f424ec529b2 100644 --- a/doc/user/project/deploy_keys/index.md +++ b/doc/user/project/deploy_keys/index.md @@ -54,7 +54,7 @@ For example: To view the deploy keys available to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Deploy keys**. @@ -72,7 +72,7 @@ Prerequisites: - [Generate an SSH key pair](../../ssh.md#generate-an-ssh-key-pair). Put the private SSH key on the host that requires access to the repository. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Deploy keys**. 1. Complete the fields. @@ -92,7 +92,7 @@ Prerequisites: To create a public deploy key: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Deploy Keys**. 1. Select **New deploy key**. 1. Complete the fields. @@ -109,7 +109,7 @@ Prerequisites: To grant a public deploy key access to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Deploy keys**. 1. Select **Publicly accessible deploy keys**. @@ -129,7 +129,7 @@ Prerequisites: To disable a deploy key: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Deploy keys**. 1. Select **Disable** (**{cancel}**). diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index 595f5e541b7..04a2eeacffb 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -31,7 +31,7 @@ You can create as many deploy tokens as you need from the settings of your project. Alternatively, you can also create [group-scoped deploy tokens](#group-deploy-token). 1. Sign in to your GitLab account. -1. On the top bar, select **Menu > Projects** or **Menu > Groups** to find your project or group. +1. On the top bar, select **Main menu > Projects** or **Main menu > Groups** to find your project or group. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Deploy tokens**. 1. Choose a name, and optionally, an expiration date and username for the token. @@ -51,7 +51,7 @@ Deploy tokens expire at midnight UTC on the date you define. To revoke a deploy token: -1. On the top bar, select **Menu > Projects** or **Menu > Groups** to find your project or group. +1. On the top bar, select **Main menu > Projects** or **Main menu > Groups** to find your project or group. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Deploy tokens**. 1. In the **Active Deploy Tokens** section, by the token you want to revoke, select **Revoke**. @@ -190,7 +190,8 @@ To pull images from the Dependency Proxy, you must: ### GitLab deploy token -> Support for `gitlab-deploy-token` at the group level [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214014) in GitLab 15.1 [with a flag](../../../administration/feature_flags.md) named `ci_variable_for_group_gitlab_deploy_token`. Enabled by default. +> - Support for `gitlab-deploy-token` at the group level [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214014) in GitLab 15.1 [with a flag](../../../administration/feature_flags.md) named `ci_variable_for_group_gitlab_deploy_token`. Enabled by default. +> - [Feature flag `ci_variable_for_group_gitlab_deploy_token`](https://gitlab.com/gitlab-org/gitlab/-/issues/363621) removed in GitLab 15.4. There's a special case when it comes to deploy tokens. If a user creates one named `gitlab-deploy-token`, the username and token of the deploy token is diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index 5df3a973cca..4050fa34026 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -32,7 +32,7 @@ directory in your repository. To create an issue description template: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository**. 1. Next to the default branch, select **{plus}**. 1. Select **New file**. @@ -51,7 +51,7 @@ push to your default branch. To create a merge request description template: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository**. 1. Next to the default branch, select **{plus}**. 1. Select **New file**. @@ -103,7 +103,7 @@ As a result, you can use the same templates in issues and merge requests in all To re-use templates [you've created](../project/description_templates.md#create-an-issue-template): -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Templates**. 1. From the dropdown list, select your template project as the template repository at group level. @@ -134,10 +134,9 @@ To set a default description template for merge requests, either: This [doesn't overwrite](#priority-of-default-description-templates) the default template if one has been set in the project settings. - Users on GitLab Premium and higher: set the default template in project settings: - 1. On the top bar, select **Menu > Projects** and find your project. - 1. On the left sidebar, select **Settings**. - 1. Expand **Merge requests**. - 1. Fill in the **Default description template for merge requests** text area. + 1. On the top bar, select **Main menu > Projects** and find your project. + 1. On the left sidebar, select **Settings > Merge requests**. + 1. In the **Merge commit message template** section, fill in **Default description template for merge requests**. 1. Select **Save changes**. To set a default description template for issues, either: @@ -147,7 +146,7 @@ To set a default description template for issues, either: This [doesn't overwrite](#priority-of-default-description-templates) the default template if one has been set in the project settings. - Users on GitLab Premium and higher: set the default template in project settings: - 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings**. 1. Expand **Default issue template**. 1. Fill in the **Default description template for issues** text area. diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index 4810fb96ed3..f1b9bde6cd0 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -220,7 +220,7 @@ To view the user who locked the file (if it was not you), hover over the button. To view and remove file locks: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Locked Files**. This list shows all the files locked either through LFS or GitLab UI. diff --git a/doc/user/project/git_attributes.md b/doc/user/project/git_attributes.md index 90f64b7262c..f2e4b65e3d4 100644 --- a/doc/user/project/git_attributes.md +++ b/doc/user/project/git_attributes.md @@ -23,5 +23,5 @@ ignored. ## Syntax Highlighting The `.gitattributes` file can be used to define which language to use when -syntax highlighting files and diffs. See +syntax highlighting files and diffs. See ["Syntax Highlighting"](highlighting.md) for more information. diff --git a/doc/user/project/import/clearcase.md b/doc/user/project/import/clearcase.md index d9ad0c57d79..2d9f92c38e4 100644 --- a/doc/user/project/import/clearcase.md +++ b/doc/user/project/import/clearcase.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w tools developed by IBM which also include a centralized version control system similar to Git. -A good read of ClearCase's basic concepts is can be found in this +A good read of ClearCase's basic concepts is can be found in this [StackOverflow post](https://stackoverflow.com/a/645771/974710). The following table illustrates the main differences between ClearCase and Git: diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index a3dfa3edff0..c04f734e8bb 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -26,7 +26,7 @@ If you are importing from GitHub Enterprise to a self-managed GitLab instance: - You must first enable [GitHub integration](../../../integration/github.md). - To import projects from GitHub Enterprise to GitLab.com, use the [Import API](../../../api/import.md). -- If GitLab is behind a HTTP/HTTPS proxy, you must populate the [allowlist for local requests](../../../security/webhooks.md#allowlist-for-local-requests) +- If GitLab is behind a HTTP/HTTPS proxy, you must populate the [allowlist for local requests](../../../security/webhooks.md#create-an-allowlist-for-local-requests) with `github.com` and `api.github.com` to solve the hostname. For more information, read the issue [Importing a GitHub project requires DNS resolution even when behind a proxy](https://gitlab.com/gitlab-org/gitlab/-/issues/37941). @@ -62,7 +62,7 @@ For this association to succeed, each GitHub author and assignee in the reposito must meet one of the following conditions prior to the import: - Have previously logged in to a GitLab account using the GitHub icon. -- Have a GitHub account with a [public-facing email address](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-github-user-account/managing-email-preferences/setting-your-commit-email-address) +- Have a GitHub account with a [public-facing email address](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address) that matches their GitLab account's email address. GitLab content imports that use GitHub accounts require that the GitHub public-facing email address is populated. This means @@ -76,7 +76,7 @@ field to be populated so you may have to add it on existing accounts. Before you begin, ensure that any GitHub users who you want to map to GitLab users have either: - A GitLab account that has logged in using the GitHub icon. -- A GitLab account with an email address that matches the [publicly visible email address](https://docs.github.com/en/rest/reference/users#get-a-user) in the profile of the GitHub user +- A GitLab account with an email address that matches the [publicly visible email address](https://docs.github.com/en/rest/users#get-a-user) in the profile of the GitHub user If you are importing to GitLab.com, you can alternatively import GitHub repositories using a [personal access token](#use-a-github-token). We do not recommend this method, as it does not associate all user activity (such as issues and pull requests) with matching GitLab users. @@ -171,12 +171,15 @@ The following items of a project are imported: - Repository description. - Git repository data. +- Branch protection rules. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22650) in GitLab 15.4. - Issues. - Pull requests. - Wiki pages. - Milestones. - Labels. - Release note descriptions. +- Release note attachments. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15620) in GitLab 15.4 with `github_importer_attachments_import` + [feature flag](../../../administration/feature_flags.md) disabled by default. - Pull request review comments. - Regular issue and pull request comments. - [Git Large File Storage (LFS) Objects](../../../topics/git/lfs/index.md). @@ -184,6 +187,8 @@ The following items of a project are imported: - Pull request "merged by" information (GitLab.com and GitLab 13.7 and later). - Pull request comments replies in discussions ([GitLab.com and GitLab 14.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/336596)). - Diff Notes suggestions ([GitLab.com and GitLab 14.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/340624)). +- Issue events and pull requests events. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7673) in GitLab 15.4 with `github_importer_issue_events_import` + [feature flag](../../../administration/feature_flags.md) disabled by default. References to pull requests and issues are preserved. Each imported repository maintains visibility level unless that [visibility level is restricted](../../public_access.md#restrict-use-of-public-or-internal-projects), in which case it diff --git a/doc/user/project/import/gitlab_com.md b/doc/user/project/import/gitlab_com.md index 4103367accc..8d30a9c7f52 100644 --- a/doc/user/project/import/gitlab_com.md +++ b/doc/user/project/import/gitlab_com.md @@ -5,29 +5,30 @@ group: Import info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Project importing from GitLab.com to your private GitLab instance **(FREE)** +# Import a project from GitLab.com to your private GitLab instance **(FREE)** -You can import your existing GitLab.com projects to your GitLab instance, but keep in -mind that it is possible only if GitLab.com integration is enabled on your GitLab instance. -[Read more about GitLab.com integration for self-managed GitLab instances](../../../integration/gitlab.md). +You can import your existing GitLab.com projects to your GitLab instance. -To get to the importer page you need to go to "New project" page. +Prerequisite: -NOTE: -If you are interested in importing Wiki and merge request data to your new instance, -you'll need to follow the instructions for [exporting a project](../settings/import_export.md#export-a-project-and-its-data) +- GitLab.com integration must be enabled on your GitLab instance. + [Read more about GitLab.com integration for self-managed GitLab instances](../../../integration/gitlab.md). - +To import a GitLab.com project to your self-managed GitLab instance: -Go to the **Import Projects** tab, then select **GitLab.com**, and you are redirected to GitLab.com -for permission to access your projects. After accepting, you are automatically redirected to the importer. +1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. +1. Select **Import project**. +1. Select **GitLab.com**. +1. Give GitLab.com permission to access your projects. +1. Select **Import**. - +The importer imports your repository and issues. +When the importer is done, a new GitLab project is created with your imported data. -To import a project, select **Import**. The importer imports your repository and issues. -Once the importer is done, a new GitLab project is created with your imported data. +## Related topics -## Automate group and project import **(PREMIUM)** - -For information on automating user, group, and project import API calls, see -[Automate group and project import](index.md#automate-group-and-project-import). +- To automate user, group, and project import API calls, see + [Automate group and project import](index.md#automate-group-and-project-import). +- To import Wiki and merge request data to your new instance, + see [exporting a project](../settings/import_export.md#export-a-project-and-its-data). diff --git a/doc/user/project/import/img/gitlab_importer.png b/doc/user/project/import/img/gitlab_importer.png Binary files differdeleted file mode 100644 index 27d42eb492e..00000000000 --- a/doc/user/project/import/img/gitlab_importer.png +++ /dev/null diff --git a/doc/user/project/import/img/gitlab_new_project_page_v12_2.png b/doc/user/project/import/img/gitlab_new_project_page_v12_2.png Binary files differdeleted file mode 100644 index ff6e5dbf4a1..00000000000 --- a/doc/user/project/import/img/gitlab_new_project_page_v12_2.png +++ /dev/null diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index 432f043f945..72d533efd1b 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -85,7 +85,7 @@ Migrate the assets in this order: Keep in mind the limitations of the [import/export feature](../settings/import_export.md#items-that-are-exported). -You must still migrate your [Container Registry](../../packages/container_registry/) +You must still migrate your [Container Registry](../../packages/container_registry/index.md) over a series of Docker pulls and pushes. Re-run any CI pipelines to retrieve any build artifacts. ## Migrate from GitLab.com to self-managed GitLab @@ -142,5 +142,10 @@ GitLab from: - Bitbucket Server - Bitbucket Data Center -See the [Quick Start Guide](https://gitlab.com/gitlab-org/professional-services-automation/tools/migration/congregate/-/blob/master/docs/using-congregate.md#quick-start) -to learn how to use this approach for migrating users, groups, and projects at scale. +For more information, see: + +- [Quick Start](https://gitlab.com/gitlab-org/professional-services-automation/tools/migration/congregate/-/blob/master/docs/using-congregate.md#quick-start). +- [Frequently Asked Migration Questions](https://gitlab.com/gitlab-org/professional-services-automation/tools/migration/congregate/-/blob/master/customer/famq.md), + including settings that need checking afterwards and other limitations. + +For support, customers must enter into a paid engagement with GitLab Professional Services. diff --git a/doc/user/project/import/repo_by_url.md b/doc/user/project/import/repo_by_url.md index 5163f957171..d64bea2bb41 100644 --- a/doc/user/project/import/repo_by_url.md +++ b/doc/user/project/import/repo_by_url.md @@ -9,7 +9,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can import your existing repositories by providing the Git URL: -1. On the top bar, select **Menu > Create new project**. +1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select the **Import project** tab. 1. Select **Repository by URL**. 1. Enter a **Git repository URL**. diff --git a/doc/user/project/import/svn.md b/doc/user/project/import/svn.md index b88abf91ae1..d6bcb0a2018 100644 --- a/doc/user/project/import/svn.md +++ b/doc/user/project/import/svn.md @@ -5,186 +5,87 @@ group: Import info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Migrating from SVN to GitLab **(FREE)** +# Migrate from Subversion to GitLab **(FREE)** -Subversion (SVN) is a central version control system (VCS) while -Git is a distributed version control system. There are some major differences -between the two, for more information consult your favorite search engine. +GitLab uses Git as its version control system. If you're using Subversion (SVN) as your version control system, +you can migrate to using a Git repository in GitLab using `svn2git`. -There are two approaches to SVN to Git migration: +You can follow the steps on this page to migrate to Git if your SVN repository: -- [Git/SVN Mirror](#smooth-migration-with-a-gitsvn-mirror-using-subgit) which: - - Makes the GitLab repository to mirror the SVN project. - - Git and SVN repositories are kept in sync; you can use either one. - - Smoothens the migration process and allows you to manage migration risks. +- Has a standard format (trunk, branches, and tags). +- Is not nested. -- [Cut over migration](#cut-over-migration-with-svn2git) which: - - Translates and imports the existing data and history from SVN to Git. - - Is a fire and forget approach, good for smaller teams. +For a non-standard repository see the [`svn2git` documentation](https://github.com/nirvdrum/svn2git). -## Smooth migration with a Git/SVN mirror using SubGit +We recommend a hard cut over from SVN to Git and GitLab. Run the migration command once and then have all users use the +new GitLab repository immediately. -[SubGit](https://subgit.com) is a tool for a smooth, stress-free SVN to Git -migration. It creates a writable Git mirror of a local or remote Subversion -repository and that way you can use both Subversion and Git as long as you like. -It requires access to your GitLab server as it talks with the Git repositories -directly in a file system level. +## Install `svn2git` -### SubGit prerequisites +Install `svn2git` on a local workstation rather than the GitLab server: -1. Install Oracle JRE 1.8 or newer. On Debian-based Linux distributions you can - follow [this article](http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html). -1. Download SubGit from <https://subgit.com/download>. -1. Unpack the downloaded SubGit zip archive to the `/opt` directory. The `subgit` - command is available at `/opt/subgit-VERSION/bin/subgit`. +- On all systems you can install as a Ruby gem if you already have Ruby and Git installed: -### SubGit configuration + ```shell + sudo gem install svn2git + ``` -The first step to mirror you SVN repository in GitLab is to create a new empty -project that is used as a mirror. For Omnibus installations the path to -the repository is -`/var/opt/gitlab/git-data/repositories/USER/REPO.git` by default. For -installations from source, the default repository directory is -`/home/git/repositories/USER/REPO.git`. For convenience, assign this path to a -variable: +- On Debian-based Linux distributions you can install the native packages: -```shell -GIT_REPO_PATH=/var/opt/gitlab/git-data/repositories/USER/REPOS.git -``` + ```shell + sudo apt-get install git-core git-svn ruby + ``` -SubGit keeps this repository in sync with a remote SVN project. For -convenience, assign your remote SVN project URL to a variable: +## Prepare an authors file (recommended) -```shell -SVN_PROJECT_URL=http://svn.company.com/repos/project -``` +Prepare an authors file so `svn2git` can map SVN authors to Git authors. If you choose not to create the authors file, +commits are not attributed to the correct GitLab user. -Next you need to run SubGit to set up a Git/SVN mirror. Make sure the following -`subgit` command is ran on behalf of the same user that keeps ownership of -GitLab Git repositories (by default `git`): +To map authors, you must map every author present on changes in the SVN repository. If you don't, the +migration fails and you have to update the author file accordingly. -```shell -subgit configure --layout auto $SVN_PROJECT_URL $GIT_REPO_PATH -``` +1. Search through the SVN repository and output a list of authors: -Adjust authors and branches mappings, if necessary. Open with your favorite -text editor: + ```shell + svn log --quiet | grep -E "r[0-9]+ \| .+ \|" | cut -d'|' -f2 | sed 's/ //g' | sort | uniq + ``` -```shell -edit $GIT_REPO_PATH/subgit/authors.txt -edit $GIT_REPO_PATH/subgit/config -``` +1. Use the output from the last command to construct the authors file. Create a file called `authors.txt` and add one + mapping per line. For example: -For more information regarding the SubGit configuration options, refer to -[SubGit's documentation](https://subgit.com/documentation/) website. + ```plaintext + sidneyjones = Sidney Jones <sidneyjones@example.com> + ``` -### Initial translation +## Migrate SVN repository to Git repository -Now that SubGit has configured the Git/SVN repositories, run `subgit` to perform the -initial translation of existing SVN revisions into the Git repository: +`svn2git` supports excluding certain file paths, branches, tags, and more. See +the [`svn2git` documentation](https://github.com/nirvdrum/svn2git) or run `svn2git --help` for full documentation on all of +the available options. -```shell -subgit install $GIT_REPO_PATH -``` +For each repository to migrate: -After the initial translation is completed, `subgit` keeps the Git repository and the SVN -project sync - new Git commits are translated to -SVN revisions and new SVN revisions are translated to Git commits. Mirror -works transparently and does not require any special commands. +1. Create a new directory and change into it. +1. For repositories that: -If you would prefer to perform one-time cut over migration with `subgit`, use -the `import` command instead of `install`: + - Don't require a username and password, run: -```shell -subgit import $GIT_REPO_PATH -``` + ```shell + svn2git https://svn.example.com/path/to/repo --authors /path/to/authors.txt + ``` -### SubGit licensing + - Do require a username and password, run: -Running SubGit in a mirror mode requires a -[registration](https://subgit.com/pricing). Registration is free for open -source, academic and startup projects. + ```shell + svn2git https://svn.example.com/path/to/repo --authors /path/to/authors.txt --username <username> --password <password> + ``` -### SubGit support +1. Create a new GitLab project for your migrated code. +1. Copy the SSH or HTTP(S) repository URL from the GitLab project page. +1. Add the GitLab repository as a Git remote and push all the changes. This pushes all commits, branches, and tags. -For any questions related to SVN to GitLab migration with SubGit, you can -contact the SubGit team directly at [support@subgit.com](mailto:support@subgit.com). - -## Cut over migration with svn2git - -NOTE: -Any issues with svn2git should be directed to the [relevant project and maintainer](https://github.com/nirvdrum/svn2git). -Check for existing issues and history for update frequency. - -If you are currently using an SVN repository, you can migrate the repository -to Git and GitLab. We recommend a hard cut over - run the migration command once -and then have all developers start using the new GitLab repository immediately. -Otherwise, it's hard to keep changing in sync in both directions. The conversion -process should be run on a local workstation. - -Install `svn2git`. On all systems you can install as a Ruby gem if you already -have Ruby and Git installed. - -```shell -sudo gem install svn2git -``` - -On Debian-based Linux distributions you can install the native packages: - -```shell -sudo apt-get install git-core git-svn ruby -``` - -Optionally, prepare an authors file so `svn2git` can map SVN authors to Git authors. -If you choose not to create the authors file then commits are not attributed -to the correct GitLab user. Some users may not consider this a big issue while -others want to ensure they complete this step. If you choose to map authors, -you must map every author present on changes in the SVN -repository. If you don't, the conversion fails and you have to update -the author file accordingly. The following command searches through the -repository and output a list of authors. - -```shell -svn log --quiet | grep -E "r[0-9]+ \| .+ \|" | cut -d'|' -f2 | sed 's/ //g' | sort | uniq -``` - -Use the output from the last command to construct the authors file. -Create a file called `authors.txt` and add one mapping per line. - -```plaintext -janedoe = Jane Doe <janedoe@example.com> -johndoe = John Doe <johndoe@example.com> -``` - -If your SVN repository is in the standard format (trunk, branches, tags, -not nested) the conversion is simple. For a non-standard repository see -[svn2git documentation](https://github.com/nirvdrum/svn2git). The following -command will checkout the repository and do the conversion in the current -working directory. Be sure to create a new directory for each repository before -running the `svn2git` command. The conversion process takes some time. - -```shell -svn2git https://svn.example.com/path/to/repo --authors /path/to/authors.txt -``` - -If your SVN repository requires a username and password add the -`--username <username>` and `--password <password>` flags to the above command. -`svn2git` also supports excluding certain file paths, branches, tags, and so on. See -[svn2git documentation](https://github.com/nirvdrum/svn2git) or run -`svn2git --help` for full documentation on all of the available options. - -Create a new GitLab project, into which you push your converted code. -Copy the SSH or HTTP(S) repository URL from the project page. Add the GitLab -repository as a Git remote and push all the changes. This pushes all commits, -branches and tags. - -```shell -git remote add origin git@gitlab.com:<group>/<project>.git -git push --all origin -git push --tags origin -``` - -## Contribute to this guide - -We welcome all contributions that would expand this guide with instructions on -how to migrate from SVN and other version control systems. + ```shell + git remote add origin git@gitlab.example.com:<group>/<project>.git + git push --all origin + git push --tags origin + ``` diff --git a/doc/user/project/insights/index.md b/doc/user/project/insights/index.md index 53547e69e00..81293fb1645 100644 --- a/doc/user/project/insights/index.md +++ b/doc/user/project/insights/index.md @@ -1,6 +1,6 @@ --- -stage: none -group: unassigned +stage: Manage +group: Optimize info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -158,7 +158,7 @@ Supported values are: | `stacked-bar` |  | NOTE: -The `dora` data source only supports the `bar` chart type. +The `dora` data source supports the `bar` and `line` chart types. ### `query` @@ -352,7 +352,7 @@ dora: title: "DORA charts" charts: - title: "DORA deployment frequency" - type: bar # only bar chart is supported at the moment + type: bar # or line query: data_source: dora params: diff --git a/doc/user/project/integrations/asana.md b/doc/user/project/integrations/asana.md index a10e261f10e..07b37b5be43 100644 --- a/doc/user/project/integrations/asana.md +++ b/doc/user/project/integrations/asana.md @@ -32,7 +32,7 @@ In Asana, create a Personal Access Token. Complete these steps in GitLab: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Asana**. 1. Ensure that the **Active** toggle is enabled. diff --git a/doc/user/project/integrations/bamboo.md b/doc/user/project/integrations/bamboo.md index 75f099268cb..7b39f6c7162 100644 --- a/doc/user/project/integrations/bamboo.md +++ b/doc/user/project/integrations/bamboo.md @@ -36,7 +36,7 @@ integration in GitLab. ## Configure GitLab -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Atlassian Bamboo**. 1. Ensure the **Active** checkbox is selected. diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index ac4b9d0769b..f058950e0f4 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -14,7 +14,7 @@ You can configure Bugzilla as an To enable the Bugzilla integration in a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Bugzilla**. 1. Select the checkbox under **Enable integration**. diff --git a/doc/user/project/integrations/custom_issue_tracker.md b/doc/user/project/integrations/custom_issue_tracker.md index 71ca9f4f640..c3794aa1a2b 100644 --- a/doc/user/project/integrations/custom_issue_tracker.md +++ b/doc/user/project/integrations/custom_issue_tracker.md @@ -20,7 +20,7 @@ on the left sidebar in your project. To enable a custom issue tracker in a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Custom issue tracker**. 1. Select the checkbox under **Enable integration**. diff --git a/doc/user/project/integrations/discord_notifications.md b/doc/user/project/integrations/discord_notifications.md index 3780ea37c0b..dffcc780206 100644 --- a/doc/user/project/integrations/discord_notifications.md +++ b/doc/user/project/integrations/discord_notifications.md @@ -26,7 +26,7 @@ and configure it in GitLab. With the webhook URL created in the Discord channel, you can set up the Discord Notifications service in GitLab. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Discord Notifications**. 1. Ensure that the **Active** toggle is enabled. diff --git a/doc/user/project/integrations/emails_on_push.md b/doc/user/project/integrations/emails_on_push.md index c1c48c7fb12..37d9a86f8fa 100644 --- a/doc/user/project/integrations/emails_on_push.md +++ b/doc/user/project/integrations/emails_on_push.md @@ -11,7 +11,7 @@ that is pushed to your project. To enable emails on push: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Emails on push**. 1. In the **Recipients** section, provide a list of emails separated by spaces or newlines. diff --git a/doc/user/project/integrations/ewm.md b/doc/user/project/integrations/ewm.md index b02f1a06e96..45f3653757d 100644 --- a/doc/user/project/integrations/ewm.md +++ b/doc/user/project/integrations/ewm.md @@ -14,7 +14,7 @@ This IBM product was [formerly named Rational Team Concert](https://jazz.net/blo To enable the EWM integration, in a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **EWM**. 1. Select the checkbox under **Enable integration**. diff --git a/doc/user/project/integrations/github.md b/doc/user/project/integrations/github.md index c07142d6edf..4be541d99cb 100644 --- a/doc/user/project/integrations/github.md +++ b/doc/user/project/integrations/github.md @@ -29,7 +29,7 @@ Complete these steps on GitHub: Complete these steps in GitLab: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **GitHub**. 1. Ensure the **Active** checkbox is selected. @@ -39,7 +39,7 @@ Complete these steps in GitLab: 1. Optional. Select **Test settings**. 1. Select **Save changes**. -After configuring the integration, see [Pipelines for external pull requests](../../../ci/ci_cd_for_external_repos/#pipelines-for-external-pull-requests) +After configuring the integration, see [Pipelines for external pull requests](../../../ci/ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests) to configure pipelines to run for open pull requests. ### Static or dynamic status check names diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index dc56c2669f8..afc379e7a07 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -74,7 +74,6 @@ Slack user on GitLab.com. The only difference with the [manually configurable Slack slash commands](slack_slash_commands.md) is that all the commands should be prefixed with the `/gitlab` keyword. -We are working on making this configurable in the future. For example, to show the issue number `1001` under the `gitlab-org/gitlab` project, you would do: diff --git a/doc/user/project/integrations/hangouts_chat.md b/doc/user/project/integrations/hangouts_chat.md index fbfa7d914a5..b2586383b43 100644 --- a/doc/user/project/integrations/hangouts_chat.md +++ b/doc/user/project/integrations/hangouts_chat.md @@ -10,7 +10,7 @@ Integrate your project to send notifications from GitLab to a room of your choice in [Google Chat](https://chat.google.com/) (former Google Hangouts). -## How it works +## Integration workflow To enable this integration, first you need to create a webhook for the room in Google Chat where you want to receive the notifications from your project. @@ -23,9 +23,9 @@ notifications to Google Chat:  -## In Google Chat +## Enable the integration in Google Chat -Select a room and create a webhook: +To enable the integration in Google Chat: 1. Enter the room where you want to receive notifications from GitLab. 1. Open the room dropdown menu on the top-left and select **Manage webhooks**. @@ -36,9 +36,22 @@ Select a room and create a webhook: For further details, see [the Google Chat documentation for configuring webhooks](https://developers.google.com/chat/how-tos/webhooks). -## In GitLab +### Enable threads in Google Chat -Enable the Google Chat integration in GitLab: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27823) in GitLab 15.4. + +To enable threaded notifications for the same GitLab object (for example, an issue or merge request): + +1. Go to [Google Chat](https://chat.google.com/). +1. In **Spaces**, select **+ > Create space**. +1. Enter the space name and (optionally) other details, and select **Use threaded replies**. +1. Select **Create**. + +You cannot enable threaded replies for existing Google Chat spaces. + +## Enable the integration in GitLab + +To enable the integration in GitLab: 1. In your project, go to **Settings > Integrations** and select **Google Chat**. 1. Scroll down to the end of the page where you find a **Webhook** field. diff --git a/doc/user/project/integrations/harbor.md b/doc/user/project/integrations/harbor.md index da35f0dc226..535703ff59e 100644 --- a/doc/user/project/integrations/harbor.md +++ b/doc/user/project/integrations/harbor.md @@ -25,7 +25,7 @@ In the Harbor instance, ensure that: GitLab supports integrating Harbor projects at the group or project level. Complete these steps in GitLab: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Harbor**. 1. Turn on the **Active** toggle under **Enable Integration**. @@ -42,7 +42,9 @@ After the Harbor integration is activated: - The global variables `$HARBOR_USERNAME`, `$HARBOR_HOST`, `$HARBOR_OCI`, `$HARBOR_PASSWORD`, `$HARBOR_URL`, and `$HARBOR_PROJECT` are created for CI/CD use. - The project-level integration settings override the group-level integration settings. -## Secure your requests to the Harbor APIs +## Security considerations + +### Secure your requests to the Harbor APIs For each API request through the Harbor integration, the credentials for your connection to the Harbor API use the `username:password` combination. The following are suggestions for safe use: @@ -51,6 +53,12 @@ the `username:password` combination. The following are suggestions for safe use: - Follow the principle of least privilege (for access on Harbor) with your credentials. - Have a rotation policy on your credentials. +### CI/CD variable security + +Malicious code pushed to your `.gitlab-ci.yml` file could compromise your variables, including +`$HARBOR_PASSWORD`, and send them to a third-party server. For more details, see +[CI/CD variable security](../../../ci/variables/index.md#cicd-variable-security). + ## Examples of Harbor variables in CI/CD ### Push a Docker image with kaniko diff --git a/doc/user/project/integrations/img/mattermost_add_slash_command.png b/doc/user/project/integrations/img/mattermost_add_slash_command.png Binary files differdeleted file mode 100644 index 7759efa183c..00000000000 --- a/doc/user/project/integrations/img/mattermost_add_slash_command.png +++ /dev/null diff --git a/doc/user/project/integrations/img/mattermost_bot_auth.png b/doc/user/project/integrations/img/mattermost_bot_auth.png Binary files differdeleted file mode 100644 index a05d8da1237..00000000000 --- a/doc/user/project/integrations/img/mattermost_bot_auth.png +++ /dev/null diff --git a/doc/user/project/integrations/img/mattermost_bot_available_commands.png b/doc/user/project/integrations/img/mattermost_bot_available_commands.png Binary files differdeleted file mode 100644 index 3232ccc3451..00000000000 --- a/doc/user/project/integrations/img/mattermost_bot_available_commands.png +++ /dev/null diff --git a/doc/user/project/integrations/img/mattermost_gitlab_token.png b/doc/user/project/integrations/img/mattermost_gitlab_token.png Binary files differdeleted file mode 100644 index 63140503824..00000000000 --- a/doc/user/project/integrations/img/mattermost_gitlab_token.png +++ /dev/null diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index 3ef40fdfe44..18e827f8df8 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -18,7 +18,7 @@ Prerequisites: To view the available integrations for your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. You can also view and manage integration settings across [all projects in an instance or group](../../admin_area/settings/project_integration_management.md). @@ -76,6 +76,7 @@ You can configure the following integrations. | [Pumble](pumble.md) | Send event notifications to a Pumble channel. | **{dotted-circle}** No | | Pushover | Get real-time notifications on your device. | **{dotted-circle}** No | | [Redmine](redmine.md) | Use Redmine as the issue tracker. | **{dotted-circle}** No | +| [Shimo Workspace](shimo.md) | Use Shimo instead of the GitLab Wiki. | **{dotted-circle}** No | | [Slack application](gitlab_slack_application.md) | Use Slack's official GitLab application. | **{dotted-circle}** No | | [Slack notifications](slack.md) | Send notifications about project events to Slack. | **{dotted-circle}** No | | [Slack slash commands](slack_slash_commands.md) | Enable slash commands in a workspace. | **{dotted-circle}** No | diff --git a/doc/user/project/integrations/irker.md b/doc/user/project/integrations/irker.md index b2c2aea2c2b..5f7de09cc9d 100644 --- a/doc/user/project/integrations/irker.md +++ b/doc/user/project/integrations/irker.md @@ -39,7 +39,7 @@ network. For more details, read ## Complete these steps in GitLab -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **irker (IRC gateway)**. 1. Ensure that the **Active** toggle is enabled. diff --git a/doc/user/project/integrations/mattermost.md b/doc/user/project/integrations/mattermost.md index 7dd4c1d1a8b..12575e34058 100644 --- a/doc/user/project/integrations/mattermost.md +++ b/doc/user/project/integrations/mattermost.md @@ -39,7 +39,7 @@ Display name override is not enabled by default, you need to ask your administra After the Mattermost instance has an incoming webhook set up, you can set up GitLab to send the notifications: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Mattermost notifications**. 1. Select the GitLab events to generate notifications for. For each event you select, input the Mattermost channel diff --git a/doc/user/project/integrations/mattermost_slash_commands.md b/doc/user/project/integrations/mattermost_slash_commands.md index 768acb02ee6..28a5f2eec18 100644 --- a/doc/user/project/integrations/mattermost_slash_commands.md +++ b/doc/user/project/integrations/mattermost_slash_commands.md @@ -6,48 +6,46 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Mattermost slash commands **(FREE)** -If your team uses [Mattermost](https://mattermost.com/) as a chat service, you can -integrate GitLab commands into Mattermost chat. This integration enables users to -run common operations, such as creating a GitLab issue, from the Mattermost chat -environment. +You can use slash commands to run common GitLab operations, like creating an issue, +from a [Mattermost](https://mattermost.com/) chat environment. GitLab can also send events (such as `issue created`) to Mattermost as part of the -separately configured [Mattermost Notifications Service](mattermost.md). +separately configured [Mattermost notifications](mattermost.md). -## Prerequisites +## Configuration options -Mattermost [3.4 or later](https://mattermost.com/blog/category/platform/releases/) is required. -GitLab provides different methods of configuring Mattermost slash commands, depending -on your configuration: +GitLab provides different ways to configure Mattermost slash commands. For any of these options, +you must have Mattermost [3.4 or later](https://mattermost.com/blog/category/platform/releases/). - **Omnibus GitLab installations**: Mattermost is bundled with - [Omnibus GitLab](https://docs.gitlab.com/omnibus/). To configure Mattermost for Omnibus GitLab, read the - [Omnibus GitLab Mattermost documentation](../../../integration/mattermost/index.md). + [Omnibus GitLab](https://docs.gitlab.com/omnibus/). To configure Mattermost for Omnibus GitLab, + read the [Omnibus GitLab Mattermost documentation](../../../integration/mattermost/index.md). - **If Mattermost is installed on the same server as GitLab**, use the - [automated configuration](#automated-configuration). -- **For all other installations**, use the [manual configuration](#manual-configuration). + [automated configuration](#configure-automatically). +- **For all other installations**, use the [manual configuration](#configure-manually). -## Automated configuration +## Configure automatically -If Mattermost is installed on the same server as GitLab, the configuration process can be -done for you by GitLab. +If Mattermost is installed on the same server as GitLab, +you can automatically configure Mattermost slash commands: -Go to the Mattermost Slash Command service on your project and select **Add to Mattermost**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Integrations**. +1. In **Add an integration**, select **Mattermost slash commands**. +1. In **Enable integration**, ensure the **Active** checkbox is selected. +1. Select **Add to Mattermost**, and select **Save changes**. -## Manual configuration +## Configure manually To manually configure slash commands in Mattermost, you must: -1. [Enable custom slash commands](#enable-custom-slash-commands) in Mattermost. -1. [Get configuration values](#get-configuration-values-from-gitlab) from GitLab. -1. [Create a new slash command](#create-a-slash-command) in Mattermost. -1. [Provide the Mattermost token](#provide-the-mattermost-token-to-gitlab) to GitLab. +1. [Enable custom slash commands in Mattermost](#enable-custom-slash-commands-in-mattermost). + (This step is required only for installations from source.) +1. [Get configuration values from GitLab](#get-configuration-values-from-gitlab). +1. [Create a slash command in Mattermost](#create-a-slash-command-in-mattermost). +1. [Provide the Mattermost token to GitLab](#provide-the-mattermost-token-to-gitlab). -### Enable custom slash commands - -NOTE: -Omnibus GitLab installations are preconfigured. This step is required only for -installations from source. +### Enable custom slash commands in Mattermost To enable custom slash commands from the Mattermost administrator console: @@ -58,80 +56,68 @@ To enable custom slash commands from the Mattermost administrator console: - **Enable Custom Slash Commands** - **Enable integrations to override usernames** - **Enable integrations to override profile picture icons** -1. Select **Save**, but do not close this browser tab, because you need it in +1. Select **Save**, but do not close this browser tab. You need it in a later step. ### Get configuration values from GitLab -After you enable custom slash commands in Mattermost, you need configuration -information from GitLab. To get this information: +To get configuration values from GitLab: -1. In a different browser tab than your current Mattermost session, sign in to +1. In a different browser tab, sign in to GitLab as a user with administrator access. -1. On the top bar, select **Menu > Admin**. -1. In the left menu, select **Settings > Integrations**, then select - **Mattermost slash commands**. -1. GitLab displays potential values for Mattermost settings. Copy the **Request URL** - as you need it for the next step. All other values are suggestions. -1. Do not close this browser tab, because you need it in future steps. - -Next, create a slash command in Mattermost with the values from GitLab. +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > Integrations**. +1. Select **Mattermost slash commands**. GitLab displays potential values for Mattermost settings. +1. Copy the **Request URL** value. All other values are suggestions. +1. Do not close this browser tab. You need it in a later step. -### Create a slash command +### Create a slash command in Mattermost -To create a slash command, you need the values you obtained from GitLab in -the previous step: +To create a slash command in Mattermost: -1. In the Mattermost tab you left open when you - [enabled custom slash commands](#enable-custom-slash-commands), go to your - team page. +1. [In the Mattermost browser tab](#enable-custom-slash-commands-in-mattermost), + go to your team page. 1. Select the **{ellipsis_v}** **Settings** icon, and select **Integrations**. -1. In the left menu, select **Slash commands**. -1. Select **Add Slash Command**: - -  +1. On the left sidebar, select **Slash commands**. +1. Select **Add Slash Command**. 1. Provide a **Display Name** and **Description** for your new command. -1. Provide a **Command Trigger Word** according to your application's configuration: +1. Provide a **Command Trigger Word** based on your application's configuration: - - **If you intend to only connect one project to your Mattermost team**: Use + - **If you intend to only connect one project to your Mattermost team**, use `/gitlab` for your trigger word. - - **If you intend to connect multiple projects**: Use a trigger word that relates + - **If you intend to connect multiple projects**, use a trigger word that relates to your project, such as `/project-name` or `/gitlab-project-name`. -1. For **Request URL**, provide the value you copied from GitLab when you - [viewed configuration values](#get-configuration-values-from-gitlab). -1. For all other values, you may use the suggestions from GitLab or use your +1. For **Request URL**, [paste the value you copied from GitLab](#get-configuration-values-from-gitlab). +1. For all other values, you may use the suggestions from GitLab or your preferred values. -1. Copy the **Token** value, as you need it in a later step, and select **Done**. +1. Copy the **Token** value, and select **Done**. ### Provide the Mattermost token to GitLab -When you create a new slash command in Mattermost, it generates a token you must +Creating a slash command in Mattermost generates a token you must provide to GitLab: -1. In the GitLab browser tab from - [getting configuration values from GitLab](#get-configuration-values-from-gitlab), - select the **Active** checkbox to enable this configuration. -1. In the **Token** field, paste the token you obtained from Mattermost. - ensure that the **Active** toggle is enabled. - -  - -1. Select **Save changes** for the changes to take effect. +1. [In the GitLab browser tab](#get-configuration-values-from-gitlab), + select the **Active** checkbox. +1. In the **Token** text box, [paste the token you copied from Mattermost](#create-a-slash-command-in-mattermost). +1. Select **Save changes**. Your slash command can now communicate with your GitLab project. -## Authorizing Mattermost to interact with GitLab +## Connect your GitLab account to Mattermost -The first time a user interacts with the newly created slash commands, -Mattermost triggers an authorization process. +Prerequisite: - +- To run [slash commands](#available-slash-commands), you must have + [permission](../../permissions.md#project-members-permissions) to + perform the action in the GitLab project. -This connects your Mattermost user with your GitLab user. You can -see all authorized chat accounts in your profile's page under **Chat**. +To interact with GitLab using Mattermost slash commands: -When the authorization process is complete, you can start interacting with -GitLab using the Mattermost commands. +1. In a Mattermost chat environment, run your new slash command. +1. Select **connect your GitLab account** to authorize access. + +You can see all authorized chat accounts in your Mattermost profile page under **Chat**. ## Available slash commands @@ -139,30 +125,21 @@ The available slash commands for Mattermost are: | Command | Description | Example | | ------- | ----------- | ------- | -| <kbd>/<trigger> issue new <title> <kbd>⇧ Shift</kbd>+<kbd>↵ Enter</kbd> <description></kbd> | Create a new issue in the project that `<trigger>` is tied to. `<description>` is optional. | `/gitlab issue new We need to change the homepage` | -| <kbd>/<trigger> issue show <issue-number></kbd> | Show the issue with ID `<issue-number>` from the project that `<trigger>` is tied to. | `/gitlab issue show 42` | -| <kbd>/<trigger> deploy <environment> to <environment></kbd> | Start the CI job that deploys from one environment to another, for example `staging` to `production`. CI/CD must be [properly configured](../../../ci/yaml/index.md). | `/gitlab deploy staging to production` | - -To see a list of available commands to interact with GitLab, type the -trigger word followed by <kbd>help</kbd>. Example: `/gitlab help` - - +| `/<trigger> issue new <title>` <kbd>Shift</kbd>+<kbd>Enter</kbd> `<description>` | Create a new issue in the project that `<trigger>` is tied to. `<description>` is optional. | `/gitlab issue new We need to change the homepage` | +| `/<trigger> issue show <issue-number>` | Show the issue with ID `<issue-number>` from the project that `<trigger>` is tied to. | `/gitlab issue show 42` | +| `/<trigger> deploy <environment> to <environment>` | Start the CI/CD job that deploys from one environment to another (for example, `staging` to `production`). CI/CD must be [properly configured](../../../ci/yaml/index.md). | `/gitlab deploy staging to production` | +| `/<trigger> help` | View a list of available slash commands. | `/gitlab help` | -## Permissions +## Related topics -The permissions to run the [available commands](#available-slash-commands) derive from -the [permissions you have on the project](../../permissions.md#project-members-permissions). +- [Mattermost slash commands](https://developers.mattermost.com/integrate/slash-commands/) +- [Omnibus GitLab Mattermost](../../../integration/mattermost/index.md) ## Troubleshooting -If an event is not being triggered, confirm that the channel you're using is a public one. -Mattermost webhooks do not have access to private channels. - -If a private channel is required, you can edit the webhook's channel in Mattermost and -select a private channel. It is not possible to use different channels for -different types of notifications. All events are sent to the specified channel. - -## Further reading +When a Mattermost slash command does not trigger an event in GitLab: -- [Mattermost slash commands documentation](https://docs.mattermost.com/developer/slash-commands.html) -- [Omnibus GitLab Mattermost](../../../integration/mattermost/) +- Ensure you're using a public channel. + Mattermost webhooks do not have access to private channels. +- If you require a private channel, edit the webhook channel, + and select a private one. All events are sent to the specified channel. diff --git a/doc/user/project/integrations/microsoft_teams.md b/doc/user/project/integrations/microsoft_teams.md index 6679bab745b..2e6954390fb 100644 --- a/doc/user/project/integrations/microsoft_teams.md +++ b/doc/user/project/integrations/microsoft_teams.md @@ -35,7 +35,7 @@ After you configure Microsoft Teams to receive notifications, you must configure GitLab to send the notifications: 1. Sign in to GitLab as an administrator. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Microsoft Teams notifications**. 1. To enable the integration, select **Active**. diff --git a/doc/user/project/integrations/pivotal_tracker.md b/doc/user/project/integrations/pivotal_tracker.md index 7f5414b86de..a0798da21f0 100644 --- a/doc/user/project/integrations/pivotal_tracker.md +++ b/doc/user/project/integrations/pivotal_tracker.md @@ -37,7 +37,7 @@ In Pivotal Tracker, [create an API token](https://www.pivotaltracker.com/help/ar Complete these steps in GitLab: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Pivotal Tracker**. 1. Ensure that the **Active** toggle is enabled. diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 068a2810a53..c1181169261 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -62,7 +62,7 @@ GitLab can use these to access the resource. More information about authenticati service account can be found at Google's documentation for [Authenticating from a service account](https://cloud.google.com/iap/docs/authentication-howto#authenticating_from_a_service_account). -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Prometheus**. 1. For **API URL**, provide the domain name or IP address of your server, such as @@ -83,7 +83,7 @@ You can configure [Thanos](https://thanos.io/) as a drop-in replacement for Prom with GitLab. Use the domain name or IP address of the Thanos server you'd like to integrate with. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Prometheus**. 1. Provide the domain name or IP address of your server, for example diff --git a/doc/user/project/integrations/pumble.md b/doc/user/project/integrations/pumble.md index cd28a7c0048..0eb3a38bb86 100644 --- a/doc/user/project/integrations/pumble.md +++ b/doc/user/project/integrations/pumble.md @@ -25,8 +25,8 @@ notifications: 1. To enable the integration for your group or project: 1. In your group or project, on the left sidebar, select **Settings > Integrations**. -1. To enable the integration for your instance: - 1. On the top bar, select **Menu > Admin**. +1. To enable the integration for your instance: + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Integrations**. 1. Select the **Pumble** integration. 1. Ensure that the **Active** toggle is enabled. @@ -36,4 +36,4 @@ notifications: 1. Optional. To test the integration, select **Test settings**. 1. Select **Save changes**. -The Pumble channel begins to receive all applicable GitLab events. +The Pumble channel begins to receive all applicable GitLab events. diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index a989b418199..bc1d299dccf 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -10,7 +10,7 @@ Use [Redmine](https://www.redmine.org/) as the issue tracker. To enable the Redmine integration in a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Redmine**. 1. Select the checkbox under **Enable integration**. diff --git a/doc/user/project/integrations/servicenow.md b/doc/user/project/integrations/servicenow.md index fdcbb498621..dc6c2da0d91 100644 --- a/doc/user/project/integrations/servicenow.md +++ b/doc/user/project/integrations/servicenow.md @@ -15,7 +15,7 @@ With the GitLab spoke in ServiceNow, you can automate actions for GitLab projects, groups, users, issues, merge requests, branches, and repositories. For a full list of features, see the -[GitLab spoke documentation](https://docs.servicenow.com/bundle/orlando-servicenow-platform/page/administer/integrationhub-store-spokes/concept/gitlab-spoke.html). +[GitLab spoke documentation](https://docs.servicenow.com/bundle/sandiego-application-development/page/administer/integrationhub-store-spokes/concept/gitlab-spoke.html). You must [configure GitLab as an OAuth2 authentication service provider](../../../integration/oauth_provider.md), which involves creating an application and then providing the Application ID diff --git a/doc/user/project/integrations/shimo.md b/doc/user/project/integrations/shimo.md new file mode 100644 index 00000000000..ea92b8b3f0b --- /dev/null +++ b/doc/user/project/integrations/shimo.md @@ -0,0 +1,34 @@ +--- +stage: Ecosystem +group: Integrations +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Shimo Workspace integration **(FREE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343386) in GitLab 14.5 with a feature flag named `shimo_integration`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/345356) in GitLab 15.4. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/345356) in GitLab 15.4. [Feature flag `shimo_integration`](https://gitlab.com/gitlab-org/gitlab/-/issues/345356) removed. + +[Shimo](https://shimo.im/) is a productivity suite that includes documents, spreadsheets, and slideshows in one interface. With this integration, you can use the Shimo Wiki directly within GitLab instead of the [GitLab group/project wiki](../wiki/index.md). + +## Configure settings in GitLab + +To enable the Shimo Workspace integration for your group or project: + +1. On the top bar, select **Main menu** and find your group or project. +1. On the left sidebar, select **Settings > Integrations**. +1. In **Add an integration**, select **Shimo Workspace**. +1. In **Enable integration**, ensure the **Active** checkbox is selected. +1. Provide the **Shimo Workspace URL** you want to link to your group or project (for example, `https://shimo.im/space/aBAYV6VvajUP873j`). +1. Select **Save changes**. + +On the left sidebar, **Shimo** now appears instead of **Wiki**. + +## View the Shimo Workspace + +To view the Shimo Workspace from your group or project: + +1. On the top bar, select **Main menu** and find your group or project. +1. On the left sidebar, select **Shimo**. +1. On the **Shimo Workspace** page, select **Go to Shimo Workspace**. diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md index dd0f57570aa..ae2e57a6d7f 100644 --- a/doc/user/project/integrations/slack.md +++ b/doc/user/project/integrations/slack.md @@ -22,7 +22,7 @@ to control GitLab from Slack. Slash commands are configured separately. ## Configure GitLab -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Slack notifications**. 1. In the **Enable integration** section, select the **Active** checkbox. diff --git a/doc/user/project/integrations/slack_slash_commands.md b/doc/user/project/integrations/slack_slash_commands.md index 5ad344a7d8e..67d6befb5fc 100644 --- a/doc/user/project/integrations/slack_slash_commands.md +++ b/doc/user/project/integrations/slack_slash_commands.md @@ -21,7 +21,7 @@ For GitLab.com, use the [GitLab Slack app](gitlab_slack_application.md) instead. Slack slash command integrations are scoped to a project. -1. In GitLab, on the top bar, select **Menu > Projects** and find your project. +1. In GitLab, on the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Slack slash commands**. Leave this browser tab open. 1. Open a new browser tab, sign in to your Slack team, and [start a new Slash Commands integration](https://my.slack.com/services/new/slash-commands). diff --git a/doc/user/project/integrations/unify_circuit.md b/doc/user/project/integrations/unify_circuit.md index 1e607d89e80..91beefd30ab 100644 --- a/doc/user/project/integrations/unify_circuit.md +++ b/doc/user/project/integrations/unify_circuit.md @@ -15,7 +15,7 @@ copy its URL. In GitLab: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **Unify Circuit**. 1. Turn on the **Active** toggle. diff --git a/doc/user/project/integrations/webex_teams.md b/doc/user/project/integrations/webex_teams.md index e8b2470cf13..0b487e29d26 100644 --- a/doc/user/project/integrations/webex_teams.md +++ b/doc/user/project/integrations/webex_teams.md @@ -27,7 +27,7 @@ notifications: 1. Navigate to: - **Settings > Integrations** in a project to enable the integration at the project level. - **Settings > Integrations** in a group to enable the integration at the group level. - - On the top bar, select **Menu > Admin**. Then, in the left sidebar, + - On the top bar, select **Main menu > Admin**. Then, in the left sidebar, select **Settings > Integrations** to enable an instance-level integration. 1. Select the **Webex Teams** integration. 1. Ensure that the **Active** toggle is enabled. diff --git a/doc/user/project/integrations/youtrack.md b/doc/user/project/integrations/youtrack.md index 25fc9c4e1c3..e6071e7517d 100644 --- a/doc/user/project/integrations/youtrack.md +++ b/doc/user/project/integrations/youtrack.md @@ -14,7 +14,7 @@ You can configure YouTrack as an To enable the YouTrack integration in a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **YouTrack**. 1. Select the checkbox under **Enable integration**. diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 787e990526d..916d566bb20 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -8,14 +8,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w The issue board is a software project management tool used to plan, organize, and visualize a workflow for a feature or product release. -It can be used as a [Kanban](https://en.wikipedia.org/wiki/Kanban_(development)) or a + +You can use it as a [Kanban](https://en.wikipedia.org/wiki/Kanban_(development)) or a [Scrum](https://en.wikipedia.org/wiki/Scrum_(software_development)) board. -It pairs issue tracking and project management, keeping everything together, -so that you don't need to jump between different platforms to organize your workflow. +Issue boards pair issue tracking and project management, keeping everything together, +so you can organize your workflow on a single platform. -Issue boards build on the existing [issue tracking functionality](issues/index.md) and -[labels](labels.md). Your issues appear as cards in vertical lists, organized by their assigned +Issue boards use [issues](issues/index.md) and [labels](labels.md). +Your issues appear as cards in vertical lists, organized by their assigned labels, [milestones](#milestone-lists), or [assignees](#assignee-lists). Issue boards help you to visualize and manage your entire process in GitLab. @@ -31,21 +32,20 @@ boards in the same project.  -Different issue board features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), -as shown in the following table: +Different issue board features are available in different [GitLab tiers](https://about.gitlab.com/pricing/): | Tier | Number of project issue boards | Number of [group issue boards](#group-issue-boards) | [Configurable issue boards](#configurable-issue-boards) | [Assignee lists](#assignee-lists) | | -------- | ------------------------------ | --------------------------------------------------- | ------------------------------------------------------- | --------------------------------- | -| Free | Multiple | 1 | No | No | -| Premium | Multiple | Multiple | Yes | Yes | -| Ultimate | Multiple | Multiple | Yes | Yes | +| Free | Multiple | 1 | **{dotted-circle}** No | **{dotted-circle}** No | +| Premium | Multiple | Multiple | **{check-circle}** Yes | **{check-circle}** Yes | +| Ultimate | Multiple | Multiple | **{check-circle}** Yes | **{check-circle}** Yes | -To learn more, visit [GitLab Enterprise features for issue boards](#gitlab-enterprise-features-for-issue-boards) below. +Read more about [GitLab Enterprise features for issue boards](#gitlab-enterprise-features-for-issue-boards).  <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -Watch a [video presentation](https://youtu.be/vjccjHI7aGI) of +Watch a [video presentation](https://youtu.be/vjccjHI7aGI) (April 2020) of the issue board feature. ## Multiple issue boards @@ -68,17 +68,25 @@ GitLab automatically loads the last board you visited. ### Create an issue board +Prerequisites: + +- You must have at least the Reporter role for the project. + To create a new issue board: -1. Select the dropdown with the current board name in the upper left corner of the issue boards page. +1. Select the dropdown list with the current board name in the upper left corner of the issue boards page. 1. Select **Create new board**. 1. Enter the new board's name and select its scope: milestone, labels, assignee, or weight. ### Delete an issue board +Prerequisites: + +- You must have at least the Reporter role for the project. + To delete the currently active issue board: -1. Select the dropdown with the current board name in the upper left corner of the issue boards page. +1. Select the dropdown list with the current board name in the upper left corner of the issue boards page. 1. Select **Delete board**. 1. Select **Delete** to confirm. @@ -192,12 +200,11 @@ card includes: - Issue number - Assignee -## Permissions +## Ordering issues in a list -Users with at least the Reporter role can use all the functionality of the -issue board feature to create or delete lists. They can also drag issues from one list to another. +Prerequisites: -## Ordering issues in a list +- You must have at least the Reporter role for the project. When an issue is created, the system assigns a relative order value that is greater than the maximum value of that issue's project or root group. This means the issue will be at the bottom of any issue list that @@ -275,16 +282,19 @@ Users on GitLab Free can use a single group issue board. ### Assignee lists **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5784) in GitLab 11.0. - As in a regular list showing all issues with a chosen label, you can add an assignee list that shows all issues assigned to a user. -You can have a board with both label lists and assignee lists. To add an -assignee list: +You can have a board with both label lists and assignee lists. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +To add an assignee list: 1. Select **Create list**. 1. Select **Assignee**. -1. In the dropdown, select a user. +1. In the dropdown list, select a user. 1. Select **Add to board**. Now that the assignee list is added, you can assign or unassign issues to that user @@ -295,14 +305,18 @@ To remove an assignee list, just as with a label list, select the trash icon. ### Milestone lists **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6469) in GitLab 11.2. - You're also able to create lists of a milestone. These are lists that filter issues by the assigned -milestone, giving you more freedom and visibility on the issue board. To add a milestone list: +milestone, giving you more freedom and visibility on the issue board. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +To add a milestone list: 1. Select **Create list**. 1. Select **Milestone**. -1. In the dropdown, select a milestone. +1. In the dropdown list, select a milestone. 1. Select **Add to board**. Like the assignee lists, you're able to [drag issues](#move-issues-and-lists) @@ -316,14 +330,17 @@ As in other list types, select the trash icon to remove a list. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250479) in GitLab 13.11 [with a flag](../../administration/feature_flags.md) named `iteration_board_lists`. Enabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75404) in GitLab 14.6. Feature flag `iteration_board_lists` removed. -You're also able to create lists of an iteration. -These lists filter issues by the assigned iteration. +You can create lists of issues in an iteration. + +Prerequisites: + +- You must have at least the Reporter role for the project. To add an iteration list: 1. Select **Create list**. 1. Select **Iteration**. -1. In the dropdown, select an iteration. +1. In the dropdown list, select an iteration. 1. Select **Add to board**. Like the milestone lists, you're able to [drag issues](#move-issues-and-lists) @@ -344,9 +361,13 @@ This feature is available both at the project and group level. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For a video overview, see [Epics Swimlanes Walkthrough - 13.6](https://www.youtube.com/watch?v=nHC7-kz5P2g) (November 2020). +Prerequisites: + +- You must have at least the Reporter role for the project. + To group issues by epic in an issue board: -1. Select the **Group by** dropdown button. +1. Select **Group by**. 1. Select **Epic**.  @@ -375,8 +396,7 @@ You can also [drag issues](#move-issues-and-lists) to change their position and ## Work In Progress limits **(PREMIUM)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11403) in GitLab 12.7 -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. You can set a Work In Progress (WIP) limit for each issue list on an issue board. When a limit is set, the list's header shows the number of issues in the list and the soft limit of issues. @@ -389,6 +409,10 @@ Examples: - You have a list with five issues with a limit of five. When you move another issue to that list, the list's header displays **6/5**, with the six shown in red. +Prerequisites: + +- You must have at least the Reporter role for the project. + To set a WIP limit for a list: 1. Navigate to a Project or Group board of which you're a member. @@ -399,8 +423,7 @@ To set a WIP limit for a list: ## Blocked issues **(PREMIUM)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34723) in GitLab 12.8. -> - [View blocking issues when hovering over blocked icon](https://gitlab.com/gitlab-org/gitlab/-/issues/210452) in GitLab 13.10. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210452) in GitLab 13.10: View blocking issues when hovering over the "blocked" icon. If an issue is [blocked by another issue](issues/related_issues.md#blocking-issues), an icon appears next to its title to indicate its blocked status. @@ -423,9 +446,6 @@ When you hover over the blocked icon (**{issue-block}**), a detailed information - Change issue labels (by dragging an issue between lists). - Close an issue (by dragging it to the **Closed** list). -If you're not able to do some of the things above, make sure you have the right -[permissions](#permissions). - ### Edit an issue > Editing title, iteration, and confidentiality [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/248908) in GitLab 14.1. @@ -433,6 +453,10 @@ If you're not able to do some of the things above, make sure you have the right You can edit an issue without leaving the board view. To open the right sidebar, select an issue card (not its title). +Prerequisites: + +- You must have at least the Reporter role for the project. + You can edit the following issue attributes in the right sidebar: - Assignees @@ -462,6 +486,10 @@ at the end of the lists, before **Closed**. To move and reorder lists, drag them Removing a list doesn't have any effect on issues and labels, as it's just the list view that's removed. You can always create it again later if you need. +Prerequisites: + +- You must have at least the Reporter role for the project. + To remove a list from an issue board: 1. On the top of the list you want to remove, select the **List settings** icon (**{settings}**). @@ -473,6 +501,10 @@ To remove a list from an issue board: > The **Add issues** button was [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/57329) in GitLab 13.11. +Prerequisites: + +- You must have at least the Reporter role for the project. + If your board is scoped to one or more attributes, go to the issues you want to add and apply the same attributes as your board scope. @@ -488,6 +520,11 @@ The issue should now show in the `Doing` list on your issue board. > The **Remove from board** button was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/229507) in GitLab 13.10. When an issue should no longer belong to a list, you can remove it. + +Prerequisites: + +- You must have at least the Reporter role for the project. + The steps depend on the scope of the list: 1. To open the right sidebar, select the issue card. @@ -502,6 +539,10 @@ The steps depend on the scope of the list: You can use the filters on top of your issue board to show only the results you want. It's similar to the filtering used in the [issue tracker](issues/index.md). +Prerequisites: + +- You must have at least the Reporter role for the project. + You can filter by the following: - Assignee @@ -577,6 +618,40 @@ into a different list. Learn about possible effects in [Dragging issues between To move a list, select its top bar, and drag it horizontally. You can't move the **Open** and **Closed** lists, but you can hide them when editing an issue board. +#### Move an issue to the start of the list + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367473) in GitLab 15.4. + +You can move issues to the top of the list with a menu shortcut. + +Your issue is moved to the top of the list even if other issues are hidden by a filter. + +Prerequisites: + +- You must at least have the Reporter role for the project. + +To move an issue to the start of the list: + +1. In an issue board, hover over the card of the issue you want to move. +1. Select the vertical ellipsis (**{ellipsis_v}**), then **Move to start of list**. + +#### Move an issue to the end of the list + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367473) in GitLab 15.4. + +You can move issues to the bottom of the list with a menu shortcut. + +Your issue is moved to the bottom of the list even if other issues are hidden by a filter. + +Prerequisites: + +- You must at least have the Reporter role for the project. + +To move an issue to the end of the list: + +1. In an issue board, hover over the card of the issue you want to move. +1. Select the vertical ellipsis (**{ellipsis_v}**), then **Move to end of list**. + #### Dragging issues between lists To move an issue to another list, select the issue card and drag it onto that list. @@ -593,8 +668,7 @@ and the target list. ### Multi-select issue cards -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18954) in GitLab 12.4. -> - [Placed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61955) behind a [feature flag](../feature_flags.md), disabled by default in GitLab 14.0. +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61955) behind a [feature flag](../feature_flags.md) named `board_multi_select` in GitLab 14.0. Disabled by default. FLAG: On self-managed GitLab, by default this feature is not available. To make it available, ask an @@ -605,6 +679,10 @@ The feature is not ready for production use. You can select multiple issue cards, then drag the group to another position within the list, or to another list. This makes it faster to reorder many issues at once. +Prerequisites: + +- You must have at least the Reporter role for the project. + To select and move multiple cards: 1. Select each card with <kbd>Control</kbd>+`Click` on Windows or Linux, or <kbd>Command</kbd>+`Click` on MacOS. @@ -612,22 +690,6 @@ To select and move multiple cards:  -### First time using an issue board - -> - The automatic creation of the **To Do** and **Doing** lists [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202144) in GitLab 13.5. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/270583) in GitLab 13.7. In GitLab 13.7 and later, the **To Do** and **Doing** columns are not automatically created. - -WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/270583) in GitLab 13.7. -The **To Do** and **Doing** columns are no longer automatically created. - -In GitLab 13.5 and 13.6, the first time you open an issue board, you are presented with the default lists -(**Open**, **To Do**, **Doing**, and **Closed**). - -If the **To Do** and **Doing** labels don't exist in the project or group, they are created, and -their lists appear as empty. If any of them already exists, the list is filled with the issues that -have that label. - ## Tips A few things to remember: diff --git a/doc/user/project/issues/associate_zoom_meeting.md b/doc/user/project/issues/associate_zoom_meeting.md index 41de91d9bd7..ef864dc2743 100644 --- a/doc/user/project/issues/associate_zoom_meeting.md +++ b/doc/user/project/issues/associate_zoom_meeting.md @@ -8,8 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16609) in GitLab 12.4. -In order to communicate synchronously for incidents management, -GitLab allows to associate a Zoom meeting with an issue. +To communicate synchronously for incidents management, +you can associate a Zoom meeting with an issue. After you start a Zoom call for a fire-fight, you need a way to associate the conference call with an issue. This is so that your team members can join swiftly without requesting a link. @@ -36,6 +36,9 @@ You are only allowed to attach a single Zoom meeting to an issue. If you attempt to add a second Zoom meeting using the `/zoom` quick action, it doesn't work. You need to [remove it](#removing-an-existing-zoom-meeting-from-an-issue) first. +Users on GitLab Premium and higher can also +[add multiple Zoom links to incidents](../../../operations/incident_management/linked_resources.md#link-zoom-meetings-from-an-incident). + ## Removing an existing Zoom meeting from an issue Similarly to adding a Zoom meeting, you can remove it with a quick action: diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 402ce4bebec..5a1e66c8f7d 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -22,7 +22,7 @@ confidential checkbox and hit **Save changes**. When you create a confidential issue in a project, the project becomes listed in the **Contributed projects** section in your [profile](../../profile/index.md). **Contributed projects** does not show information about the confidential issue; it only shows the project name. - + ## Modify issue confidentiality @@ -39,9 +39,12 @@ The second way is to locate the **Confidentiality** section in the sidebar and s |  |  | Every change from regular to confidential and vice versa, is indicated by a -system note in the issue's comments. +system note in the issue's comments: - + + +- **{eye-slash}** The issue is made confidential. +- **{eye}** The issue is made public. When an issue is made confidential, only users with at least the Reporter role for the project have access to the issue. @@ -51,7 +54,7 @@ the issue even if they were actively participating before the change. ## Confidential issue indicators There are a few things that visually separate a confidential issue from a -regular one. In the issues index page view, you can see the eye-slash (**{eye-slash}**) icon +regular one. In the issues index page view, you can see the confidential (**{eye-slash}**) icon next to the issues that are marked as confidential:  @@ -61,7 +64,7 @@ you cannot see confidential issues at all. --- -Likewise, while inside the issue, you can see the eye-slash icon right next to +Likewise, while inside the issue, you can see the confidential (**{eye-slash}**) icon right next to the issue number. There is also an indicator in the comment area that the issue you are commenting on is confidential. diff --git a/doc/user/project/issues/img/confidential_issues_create.png b/doc/user/project/issues/img/confidential_issues_create.png Binary files differdeleted file mode 100644 index 0a141eb39f8..00000000000 --- a/doc/user/project/issues/img/confidential_issues_create.png +++ /dev/null diff --git a/doc/user/project/issues/img/confidential_issues_create_v15_4.png b/doc/user/project/issues/img/confidential_issues_create_v15_4.png Binary files differnew file mode 100644 index 00000000000..ff489ad8605 --- /dev/null +++ b/doc/user/project/issues/img/confidential_issues_create_v15_4.png diff --git a/doc/user/project/issues/img/confidential_issues_system_notes.png b/doc/user/project/issues/img/confidential_issues_system_notes.png Binary files differdeleted file mode 100644 index 355be80ecb6..00000000000 --- a/doc/user/project/issues/img/confidential_issues_system_notes.png +++ /dev/null diff --git a/doc/user/project/issues/img/confidential_issues_system_notes_v15_4.png b/doc/user/project/issues/img/confidential_issues_system_notes_v15_4.png Binary files differnew file mode 100644 index 00000000000..e448f609112 --- /dev/null +++ b/doc/user/project/issues/img/confidential_issues_system_notes_v15_4.png diff --git a/doc/user/project/issues/img/issue_weight_v13_11.png b/doc/user/project/issues/img/issue_weight_v13_11.png Binary files differdeleted file mode 100644 index 842c154ea49..00000000000 --- a/doc/user/project/issues/img/issue_weight_v13_11.png +++ /dev/null diff --git a/doc/user/project/issues/img/related_issue_block_v15_3.png b/doc/user/project/issues/img/related_issue_block_v15_3.png Binary files differindex 827ddeabf10..942f7a33fe0 100644 --- a/doc/user/project/issues/img/related_issue_block_v15_3.png +++ b/doc/user/project/issues/img/related_issue_block_v15_3.png diff --git a/doc/user/project/issues/img/related_issues_add_v15_3.png b/doc/user/project/issues/img/related_issues_add_v15_3.png Binary files differindex 7c6edf61427..28739c0b909 100644 --- a/doc/user/project/issues/img/related_issues_add_v15_3.png +++ b/doc/user/project/issues/img/related_issues_add_v15_3.png diff --git a/doc/user/project/issues/issue_weight.md b/doc/user/project/issues/issue_weight.md index fcc53a239dc..21d7fb0a764 100644 --- a/doc/user/project/issues/issue_weight.md +++ b/doc/user/project/issues/issue_weight.md @@ -13,15 +13,56 @@ When you have a lot of issues, it can be hard to get an overview. With weighted issues, you can get a better idea of how much time, value, or complexity a given issue has or costs. -You can set the weight of an issue during its creation, by changing the -value in the dropdown menu. You can set it to a non-negative integer -value from 0, 1, 2, and so on. -You can remove weight from an issue as well. -A user with a Reporter role (or above) can set the weight. +## View the issue weight -This value appears on the right sidebar of an individual issue, as well as -in the issues page next to a weight icon (**{weight}**). +You can view the issue weight on: -As an added bonus, you can see the total sum of all issues on the milestone page. +- The right sidebar of each issue. +- The issues page, next to a weight icon (**{weight}**). +- [Issue boards](../issue_board.md), next to a weight icon (**{weight}**). +- The [milestone](../milestones/index.md) page, as a total sum of issue weights. - +## Set the issue weight + +Prerequisites: + +- You must have at least the Reporter role for the project. + +You can set the issue weight when you create or edit an issue. + +You must enter whole, positive numbers. + +When you change the weight of an issue, the new value overwrites the previous value. + +### When you create an issue + +To set the issue weight when you [create an issue](managing_issues.md#create-an-issue), enter a +number under **Weight**. + +### From an existing issue + +To set the issue weight from an existing issue: + +1. Go to the issue. +1. On the right sidebar, in the **Weight** section, select **Edit**. +1. Enter the new weight. +1. Select any area outside the dropdown list. + +### From an issue board + +To set the issue weight when you [edit an issue from an issue board](../issue_board.md#edit-an-issue): + +1. Go to your issue board. +1. Select an issue card (not its title). +1. On the right sidebar, in the **Weight** section, select **Edit**. +1. Enter the new weight. +1. Select any area outside the dropdown list. + +## Remove issue weight + +Prerequisites: + +- You must have at least the Reporter role for the project. + +To remove the issue weight, follow the same steps as when you [set the issue weight](#set-the-issue-weight), +and select **remove weight**. diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 40e5a6d6a92..b4edb238479 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -33,7 +33,7 @@ Prerequisites: To create an issue: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Either: - On the left sidebar, select **Issues**, and then, in the top right corner, select **New issue**. @@ -56,7 +56,7 @@ Prerequisites: To create an issue from a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Issues**. 1. In the top right corner, select **Select project to create issue**. 1. Select the project you'd like to create an issue for. The button now reflects the selected @@ -103,7 +103,7 @@ Prerequisites: To create an issue from a project issue board: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Select **Issues > Boards**. 1. At the top of a board list, select **New issue** (**{plus-square}**). 1. Enter the issue's title. @@ -111,7 +111,7 @@ To create an issue from a project issue board: To create an issue from a group issue board: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. Select **Issues > Boards**. 1. At the top of a board list, select **New issue** (**{plus-square}**). 1. Enter the issue's title. @@ -138,7 +138,7 @@ Prerequisites: To email an issue to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Select **Issues**. 1. At the bottom of the page, select **Email a new issue to this project**. 1. To copy the email address, select **Copy** (**{copy-to-clipboard}**). @@ -271,7 +271,7 @@ Prerequisites: To edit multiple issues at the same time: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Issues**. 1. Select **Edit issues**. A sidebar on the right of your screen appears. 1. Select the checkboxes next to each issue you want to edit. @@ -304,7 +304,7 @@ Prerequisites: To edit multiple issues at the same time: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Issues**. 1. Select **Edit issues**. A sidebar on the right of your screen appears. 1. Select the checkboxes next to each issue you want to edit. @@ -458,7 +458,8 @@ The default issue closing pattern regex: #### Disable automatic issue closing -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/19754) in GitLab 12.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/19754) in GitLab 12.7. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/240922) in GitLab 15.4: The referenced issue's project setting is checked instead of the project of the commit or merge request. You can disable the automatic issue closing feature on a per-project basis in the [project's settings](../settings/index.md). @@ -469,23 +470,18 @@ Prerequisites: To disable automatic issue closing: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Default branch**. -1. Select **Auto-close referenced issues on default branch**. +1. Clear the **Auto-close referenced issues on default branch** checkbox. 1. Select **Save changes**. Referenced issues are still displayed, but are not closed automatically. -The automatic issue closing is disabled by default in a project if the project has the issue tracker -disabled. If you want to enable automatic issue closing, make sure to -[enable GitLab Issues](../settings/index.md#configure-project-visibility-features-and-permissions). - Changing this setting applies only to new merge requests or commits. Already closed issues remain as they are. -If issue tracking is enabled, disabling automatic issue closing only applies to merge requests -attempting to automatically close issues in the same project. -Merge requests in other projects can still close another project's issues. +Disabling automatic issue closing only applies to issues in the project where the setting was disabled. +Merge requests and commits in this project can still close another project's issues. #### Customize the issue closing pattern **(FREE SELF)** @@ -602,7 +598,7 @@ GitLab displays the results on-screen, but you can also > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/39908) in GitLab 12.1. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Issues > List**. 1. In the **Search** box, type the issue ID. For example, enter filter `#10` to return only issue 10. @@ -644,10 +640,15 @@ To copy the issue's email address: > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/17589) in GitLab 14.5. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/17589) in GitLab 14.9. Feature flags `real_time_issue_sidebar` and `broadcast_issue_updates` removed. -Assignees in the sidebar are updated in real time. -When you're viewing an issue and somebody changes its assignee, +Some sections of the right sidebar are updated in real time. +When you're viewing an issue and somebody changes one of the values, you can see the change without having to refresh the page. +The following sections are updated in real time: + +- [Assignee](#assignee) +- Labels, [if enabled](../labels.md#real-time-changes-to-labels) + ## Assignee An issue can be assigned to one or [more users](multiple_assignees_for_issues.md). @@ -685,6 +686,7 @@ Up to five similar issues, sorted by most recently updated, are displayed below > - Health status of closed issues [can't be edited](https://gitlab.com/gitlab-org/gitlab/-/issues/220867) in GitLab 13.4 and later. > - Issue health status visible in issue lists [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/45141) in GitLab 13.6. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/213567) in GitLab 13.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218618) in GitLab 15.4: health status is visible on issue cards in issue boards. To help you track issue statuses, you can assign a status to each issue. This status marks issues as progressing as planned or needing attention to keep on schedule. @@ -703,7 +705,11 @@ To edit health status of an issue: - Needs attention (amber) - At risk (red) -You can then see the issue's status in the issues list and the epic tree. +You can see the issue’s health status in: + +- Issues list +- Epic tree +- Issue cards in issue boards After an issue is closed, its health status can't be edited and the **Edit** button becomes disabled until the issue is reopened. diff --git a/doc/user/project/issues/related_issues.md b/doc/user/project/issues/related_issues.md index 9dc361b403f..d1e62a76103 100644 --- a/doc/user/project/issues/related_issues.md +++ b/doc/user/project/issues/related_issues.md @@ -29,7 +29,7 @@ Prerequisites: To link one issue to another: -1. In the **Linked issues** section of an issue, +1. In the **Linked items** section of an issue, select the add linked issue button (**{plus}**). 1. Select the relationship between the two issues. Either: - **relates to** @@ -61,7 +61,7 @@ You can also add a linked issue from a commit message or the description in anot ## Remove a linked issue -In the **Linked issues** section of an issue, select the remove button (**{close}**) on the +In the **Linked items** section of an issue, select the remove button (**{close}**) on the right-side of each issue token to remove. Due to the bi-directional relationship, the relationship no longer appears in either issue. diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 62c5489d9c3..13c93fadf6e 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -58,7 +58,7 @@ You can also assign and unassign labels with [quick actions](quick_actions.md): To view the **project's labels**: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Labels**. Or: @@ -75,7 +75,7 @@ project or group path where it was created. To view the **group's labels**: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Group information > Labels**. Or: @@ -97,7 +97,7 @@ Prerequisites: To create a project label: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Labels**. 1. Select **New label**. 1. In the **Title** field, enter a short, descriptive name for the label. You @@ -131,7 +131,7 @@ To do so: To create a group label: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Group information > Labels**. 1. Select **New label**. 1. In the **Title** field, enter a short, descriptive name for the label. You @@ -171,7 +171,7 @@ Prerequisites: To edit a **project** label: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Labels**. 1. Next to the label you want to edit, select **Edit** (**{pencil}**). @@ -179,7 +179,7 @@ To edit a **project** label: To edit a **group** label: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Group information > Labels**. 1. Next to the label you want to edit, select **Edit** (**{pencil}**). @@ -197,7 +197,7 @@ Prerequisites: To delete a **project** label: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Labels**. 1. Either: @@ -210,7 +210,7 @@ To delete a **project** label: To delete a **group** label: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Group information > Labels**. 1. Either: @@ -240,7 +240,7 @@ Prerequisites: To promote a project label to a group label: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Labels**. 1. Next to the **Subscribe** button, select the three dots (**{ellipsis_v}**) and select **Promote to group label**. @@ -291,7 +291,7 @@ Prerequisites: To add the default labels to the project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Labels**. 1. Select **Generate a default set of labels**. @@ -430,7 +430,7 @@ Prerequisites: To prioritize a label: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Labels**. 1. Next to a label you want to prioritize, select the star (**{star-o}**). diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 4a6272a0ca3..8d8169e8454 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -68,7 +68,7 @@ Prerequisite: To add a user to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. Select **Invite members**. 1. Enter an email address and select a [role](../../permissions.md). @@ -106,7 +106,7 @@ Prerequisite: To add groups to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. Select **Invite a group**. 1. Select a group. @@ -131,7 +131,7 @@ Prerequisite: To import users: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. Select **Import from a project**. 1. Select the project. You can view only the projects for which you're a maintainer. @@ -175,7 +175,7 @@ Prerequisites: To remove a member from a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. Next to the project member you want to remove, select **Remove member**. 1. Optional. In the confirmation box, select the @@ -197,7 +197,7 @@ You can filter and sort members in a project. ### Display inherited members -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. In the **Filter members** box, select `Membership` `=` `Inherited`. 1. Press <kbd>Enter</kbd>. @@ -206,7 +206,7 @@ You can filter and sort members in a project. ### Display direct members -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. In the **Filter members** box, select `Membership` `=` `Direct`. 1. Press <kbd>Enter</kbd>. @@ -229,7 +229,7 @@ You can sort members by **Account**, **Access granted**, **Max role**, or **Last GitLab users can request to become a member of a project. -1. On the top bar, select **Menu > Projects** and find the project you want to be a member of. +1. On the top bar, select **Main menu > Projects** and find the project you want to be a member of. 1. By the project name, select **Request Access**.  @@ -253,7 +253,7 @@ Prerequisite: - You must be the project owner. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. Under **Project visibility**, select **Users can request access**. diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index 3d5b855a9d3..ee161deaabb 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -48,6 +48,7 @@ After sharing 'Project Acme' with 'Engineering': - The group is listed in the **Groups** tab. - The project is listed on the group dashboard. +- All members, including members from the ancestors of the 'Engineering' group, gain access to 'Project Acme' with an access level based on the outcome of [maximum access level](#maximum-access-level). When you share a project, be aware of the following restrictions and outcomes: @@ -83,7 +84,7 @@ The following outcomes occur: ## Share project with group lock -It is possible to prevent projects in a group from +It is possible to prevent projects in a group from [sharing a project with another group](../members/share_project_with_groups.md). This allows for tighter control over project access. diff --git a/doc/user/project/merge_requests/approvals/rules.md b/doc/user/project/merge_requests/approvals/rules.md index c9278c19322..32548215054 100644 --- a/doc/user/project/merge_requests/approvals/rules.md +++ b/doc/user/project/merge_requests/approvals/rules.md @@ -32,8 +32,9 @@ use the default approval rules from the target (upstream) project, not the sourc To add a merge request approval rule: -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**, and then select **Add approval rule**. +1. Go to your project and select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval rules**. +1. Select **Add approval rule**. 1. Add a human-readable **Rule name**. 1. Set the number of required approvals in **Approvals required**. A value of `0` makes [the rule optional](#configure-optional-approval-rules), and any number greater than `0` @@ -65,8 +66,9 @@ to existing merge requests: To edit a merge request approval rule: -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**, and then select **Edit**. +1. Go to your project and select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval rules**. +1. Select **Edit** next to the rule you want to edit. 1. Optional. Change the **Rule name**. 1. Set the number of required approvals in **Approvals required**. The minimum value is `0`. 1. Add or remove eligible approvers, as needed: @@ -155,11 +157,11 @@ approve in these ways: If you add [code owners](../../code_owners.md) to your repository, the owners of files become eligible approvers in the project. To enable this merge request approval rule: -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**. -1. Locate **All eligible users** and select the number of approvals required: +1. Go to your project and select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval rules**. +1. Locate the **All eligible users** rule, and select the number of approvals required: - +  You can also [require code owner approval](../../protected_branches.md#require-code-owner-approval-on-a-protected-branch) @@ -182,9 +184,10 @@ granting them push access: and select the Reporter role for the user. 1. [Share the project with your group](../../members/share_project_with_groups.md#share-a-project-with-a-group-of-users), based on the Reporter role. -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**. -1. Select **Add approval rule** or **Update approval rule** and target the protected branch. +1. Go to your project and select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval rules**, and either: + - For a new rule, select **Add approval rule** and target the protected branch. + - For an existing rule, select **Edit** and target the protected branch. 1. [Add the group](../../../group/manage.md#create-a-group) to the permission list.  @@ -226,12 +229,12 @@ Approval rules are often relevant only to specific branches, like your approval rule for certain branches: 1. [Create an approval rule](#add-an-approval-rule). -1. Go to your project and select **Settings**. -1. Expand **Merge request (MR) approvals**. +1. Go to your project and select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval rules**. 1. Select a **Target branch**: - To apply the rule to all branches, select **All branches**. - To apply the rule to all protected branches, select **All protected branches** (GitLab 15.3 and later). - - To apply the rule to a specific branch, select it from the list: + - To apply the rule to a specific branch, select it from the list. 1. To enable this configuration, read [Code Owner's approvals for protected branches](../../protected_branches.md#require-code-owner-approval-on-a-protected-branch). diff --git a/doc/user/project/merge_requests/approvals/settings.md b/doc/user/project/merge_requests/approvals/settings.md index 3ca8ddb508a..4fdf6d46b8b 100644 --- a/doc/user/project/merge_requests/approvals/settings.md +++ b/doc/user/project/merge_requests/approvals/settings.md @@ -16,8 +16,8 @@ those rules are applied as a merge request moves toward completion. To view or edit merge request approval settings: -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**. +1. Go to your project and select **Settings > Merge requests**. +1. Expand **Approvals**. ### Approval settings @@ -44,9 +44,9 @@ You can further define what happens to existing approvals when commits are added By default, the author of a merge request cannot approve it. To change this setting: -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**. -1. Clear the **Prevent approval by author** checkbox. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval settings** and + clear the **Prevent approval by author** checkbox. 1. Select **Save changes**. Authors can edit the approval rule in an individual merge request and override @@ -68,9 +68,9 @@ the project level or [instance level](../../../admin_area/merge_requests_approva you can prevent committers from approving merge requests that are partially their own. To do this: -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**. -1. Select the **Prevent approvals by users who add commits** checkbox. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval settings** and + select **Prevent approvals by users who add commits**. If this checkbox is cleared, an administrator has disabled it [at the instance level](../../../admin_area/merge_requests_approvals.md), and it can't be changed at the project level. @@ -94,9 +94,9 @@ By default, users can override the approval rules you [create for a project](rul on a per-merge-request basis. If you don't want users to change approval rules on merge requests, you can disable this setting: -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**. -1. Select the **Prevent editing approval rules in merge requests** checkbox. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval settings** and + select **Prevent editing approval rules in merge requests**. 1. Select **Save changes**. This change affects all open merge requests. @@ -112,9 +112,9 @@ permission enables an electronic signature for approvals, such as the one define 1. Enable password authentication for the web interface, as described in the [sign-in restrictions documentation](../../../admin_area/settings/sign_in_restrictions.md#password-authentication-enabled). -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**. -1. Select the **Require user password to approve** checkbox. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval settings** and + select **Require user password to approve**. 1. Select **Save changes**. ## Remove all approvals when commits are added to the source branch @@ -123,9 +123,9 @@ By default, an approval on a merge request remains in place, even if you add mor after the approval. If you want to remove all existing approvals on a merge request when more changes are added to it: -1. Go to your project and select **Settings > General**. -1. Expand **Merge request (MR) approvals**. -1. Select the **Remove all approvals when commits are added to the source branch** checkbox. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval settings** and + select **Remove all approvals when commits are added to the source branch**. 1. Select **Save changes**. Approvals aren't removed when a merge request is [rebased from the UI](../methods/index.md#rebasing-in-semi-linear-merge-methods) @@ -143,10 +143,9 @@ Prerequisite: To do this: -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge request approvals**. -1. Select **Remove approvals by Code Owners if their files changed**. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval settings** and + select **Remove approvals by Code Owners if their files changed**. 1. Select **Save changes**. ## Code coverage check approvals diff --git a/doc/user/project/merge_requests/cherry_pick_changes.md b/doc/user/project/merge_requests/cherry_pick_changes.md index 14f3979cf34..2040995280e 100644 --- a/doc/user/project/merge_requests/cherry_pick_changes.md +++ b/doc/user/project/merge_requests/cherry_pick_changes.md @@ -7,61 +7,106 @@ type: reference, concepts # Cherry-pick changes **(FREE)** -GitLab implements Git's powerful feature to -[cherry-pick any commit](https://git-scm.com/docs/git-cherry-pick "Git cherry-pick documentation") -with a **Cherry-pick** button in merge requests and commit details. +In Git, *cherry-picking* is taking a single commit from one branch and adding it +as the latest commit on another branch. The rest of the commits in the source branch +are not added to the target. You should cherry-pick a commit when you need the +change contained in a single commit, but you can't or don't want to pull the +entire contents of that branch into another. -## Cherry-pick a merge request +You can use the GitLab UI to cherry-pick single commits or entire merge requests. +You can even cherry-pick a commit from [a fork of your project](#cherry-pick-into-a-project). -After the merge request has been merged, a **Cherry-pick** button displays -to cherry-pick the changes introduced by that merge request. +NOTE: +Support for tracking commits cherry-picked from the command line +is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/202215). - +## Cherry-pick example + +In this example of cherry-picking, a Git repository has two branches: `develop` and `main`. +This example shows a cherry-picked commit from one branch being added to another: + +```mermaid +gitGraph + commit id: "A" + branch develop + commit id:"B" + checkout main + commit id:"C" + checkout develop + commit id:"D" + checkout main + commit id:"E" + cherry-pick id:"B" + commit id:"G" + checkout develop + commit id:"H" +``` -After you select that button, a modal displays a -[branch filter search box](../repository/branches/index.md#branch-filter-search-box) -where you can choose to either: +In this example, a cherry-pick of commit `B` from the `develop` branch is added +after commit `E` in the `main` branch. -- Cherry-pick the changes directly into the selected branch. -- Create a new merge request with the cherry-picked changes. +Commit `G` is added after the cherry-pick. -### Track a cherry-pick +## Cherry-pick all changes from a merge request -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2675) in GitLab 12.9. +After a merge request is merged, you can cherry-pick all changes introduced +by the merge request: -When you cherry-pick a merge commit, GitLab displays a system note to the related merge -request thread. It crosslinks the new commit and the existing merge request. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests**, and find your merge request. +1. Scroll to the merge request reports section, and find the **Merged by** report. +1. In the top right, select **Cherry-pick**: - +  +1. In the modal window, select the project and branch to cherry-pick into. +1. Optional. Select **Start a new merge request with these changes**. +1. Select **Cherry-pick**. -Each deployment's [list of associated merge requests](../../../api/deployments.md#list-of-merge-requests-associated-with-a-deployment) includes cherry-picked merge commits. +## Cherry-pick a single commit -NOTE: -We only track cherry-pick executed from GitLab (both UI and API). Support for tracking cherry-picked commits through the command line -is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/202215). +You can cherry-pick a single commit from multiple locations in your GitLab project. -## Cherry-pick a commit +### From a project's commit list -You can cherry-pick a commit from the commit details page: +To cherry-pick a commit from the list of all commits for a project: - +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Commits**. +1. Select the title of the commit you want to cherry-pick. +1. In the modal window, select the project and branch to cherry-pick into. +1. Optional. Select **Start a new merge request with these changes**. +1. Select **Cherry-pick**. -Similar to cherry-picking a merge request, you can cherry-pick the changes -directly into the target branch or create a new merge request to cherry-pick the -changes. +### From a merge request -When cherry-picking merge commits, the mainline is always the -first parent. If you want to use a different mainline, you need to do that -from the command line. +You can cherry-pick commits from any merge request in your project, regardless of +whether the merge request is open or closed. To cherry-pick a commit from the +list of commits included in a merge request: -Here's a quick example to cherry-pick a merge commit using the second parent as the -mainline: +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests**, and find your merge request. +1. In the merge request's secondary menu, select **Commits** to display the commit details page. +1. Select the title of the commit you want to cherry-pick. +1. In the top right corner, select **Options > Cherry-pick** to show the cherry-pick modal. +1. In the modal window, select the project and branch to cherry-pick into. +1. Optional. Select **Start a new merge request with these changes**. +1. Select **Cherry-pick**. -```shell -git cherry-pick -m 2 7a39eb0 -``` +### From the file view of a repository -### Cherry-pick into a project +You can cherry-pick from the list of previous commits affecting an individual file +when you view that file in your project's Git repository: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Files** and go to the file + changed by the commit. +1. Select **History**, then select the title of the commit you want to cherry-pick. +1. In the top right corner, select **Options > Cherry-pick** to show the cherry-pick modal. +1. In the modal window, select the project and branch to cherry-pick into. +1. Optional. Select **Start a new merge request with these changes**. +1. Select **Cherry-pick**. + +## Cherry-pick into a project > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21268) in GitLab 13.11 behind a [feature flag](../../feature_flags.md), disabled by default. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/324154) in GitLab 14.0. @@ -70,25 +115,38 @@ You can cherry-pick merge requests from the same project, or forks of the same project, from the GitLab user interface: 1. In the merge request's secondary menu, select **Commits** to display the commit details page. -1. Select the **Options** dropdown and select **Cherry-pick** to show the cherry-pick modal. +1. In the top right corner, select **Options > Cherry-pick** to show the cherry-pick modal. 1. In **Pick into project** and **Pick into branch**, select the destination project and branch:  1. Optional. Select **Start a new merge request** if you're ready to create a merge request. 1. Select **Cherry-pick**. +## View system notes for cherry-picked commits + +When you cherry-pick a merge commit in the GitLab UI or API, GitLab adds a system note +to the related merge request thread in the format **{cherry-pick-commit}** +`[USER]` **picked the changes into the branch** `[BRANCHNAME]` with commit** `[SHA]` `[DATE]`: + + + +The system note crosslinks the new commit and the existing merge request. +Each deployment's [list of associated merge requests](../../../api/deployments.md#list-of-merge-requests-associated-with-a-deployment) includes cherry-picked merge commits. + ## Related topics -- The [Commits API](../../../api/commits.md) enables you to add custom messages - to changes you cherry-pick through the API. +- Use the [Commits API](../../../api/commits.md) to add custom messages + to changes when you use the API to cherry-pick. -<!-- ## Troubleshooting +## Troubleshooting -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### Selecting a different parent commit when cherry-picking -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +When you cherry-pick a merge commit in the GitLab UI, the mainline is always the +first parent. Use the command line to cherry-pick with a different mainline. + +Here's a quick example to cherry-pick a merge commit using the second parent as the +mainline: + +```shell +git cherry-pick -m 2 7a39eb0 +``` diff --git a/doc/user/project/merge_requests/commit_templates.md b/doc/user/project/merge_requests/commit_templates.md index 6f9bc452b96..99a1739b1a4 100644 --- a/doc/user/project/merge_requests/commit_templates.md +++ b/doc/user/project/merge_requests/commit_templates.md @@ -29,8 +29,8 @@ Prerequisite: To do this: -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General** and expand **Merge requests**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. 1. Depending on the type of template you want to create, scroll to either [**Merge commit message template**](#default-template-for-merge-commits) or [**Squash commit message template**](#default-template-for-squash-commits). diff --git a/doc/user/project/merge_requests/creating_merge_requests.md b/doc/user/project/merge_requests/creating_merge_requests.md index f30b20e9d34..1a44126f7ff 100644 --- a/doc/user/project/merge_requests/creating_merge_requests.md +++ b/doc/user/project/merge_requests/creating_merge_requests.md @@ -14,7 +14,7 @@ There are many different ways to create a merge request. You can create a merge request from the list of merge requests. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left menu, select **Merge requests**. 1. In the top right, select **New merge request**. 1. Select a source and target branch and then **Compare branches and continue**. @@ -43,7 +43,7 @@ You can create a merge request when you add, edit, or upload a file to a reposit You can create a merge request when you create a branch. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left menu, select **Repository > Branches**. 1. Type a branch name and select **New branch**. 1. Above the file list, on the right side, select **Create merge request**. @@ -90,7 +90,7 @@ to reduce the need for editing merge requests manually through the UI. You can create a merge request from your fork to contribute back to the main project. -1. On the top bar, select **Menu > Project**. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Select your fork of the repository. 1. On the left menu, go to **Merge requests**, and select **New merge request**. 1. In the **Source branch** drop-down list box, select the branch in your forked repository as the source branch. @@ -120,7 +120,7 @@ Prerequisites: To create a merge request by sending an email: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left menu, select **Merge requests**. 1. In the top right, select **Email a new merge request to this project**. An email address is displayed. Copy this address. @@ -165,7 +165,7 @@ scenarios when you create a new merge request: To have merge requests from a fork by default target your own fork (instead of the upstream project), you can change the default. -1. On the top bar, select **Menu > Project**. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left menu, select **Settings > General > Merge requests**. 1. In the **Target project** section, select the option you want to use for your default target project. diff --git a/doc/user/project/merge_requests/csv_export.md b/doc/user/project/merge_requests/csv_export.md index 893b2bc6811..f997898f5a5 100644 --- a/doc/user/project/merge_requests/csv_export.md +++ b/doc/user/project/merge_requests/csv_export.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -12,7 +12,7 @@ Export all the data collected from a project's merge requests into a comma-separ To export merge requests to a CSV file: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests** . 1. Add any searches or filters. This can help you keep the size of the CSV file under the 15MB limit. The limit ensures the file can be emailed to a variety of email providers. diff --git a/doc/user/project/merge_requests/dependencies.md b/doc/user/project/merge_requests/dependencies.md new file mode 100644 index 00000000000..5b88e69357c --- /dev/null +++ b/doc/user/project/merge_requests/dependencies.md @@ -0,0 +1,163 @@ +--- +stage: Create +group: Code Review +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference, concepts +--- + +# Merge request dependencies **(PREMIUM)** + +A single feature can span several merge requests, spread out across multiple projects, +and the order in which the work merges can be significant. Use merge request dependencies +when it's important to merge work in a specific order. Some examples: + +- Ensure changes to a required library are merged before changes to a project that + imports the library. +- Prevent a documentation-only merge request from merging before the feature work + is itself merged. +- Require a merge request updating a permissions matrix to merge, before merging work + from someone who hasn't yet been granted permissions. + +If your project `me/myexample` imports a library from `myfriend/library`, +you might want to update your project to use a new feature in `myfriend/library`. +However, if you merge changes to your project before the external library adds the +new feature, you would break the default branch in your project. A merge request +dependency prevents your work from merging too soon: + +```mermaid +graph TB + A['me/myexample' project] + B['myfriend/library' project] + C[Merge request #1:<br>Create new version 2.5] + D[Merge request #2:<br>Add version 2.5<br>to build] + A-->|contains| D + B---->|contains| C + D-.->|depends on| C + C-.->|blocks| D +``` + +You could mark your `me/myexample` merge request as a [draft](drafts.md) +and explain why in the comments. However, this approach is manual and does not scale, especially +if your merge request relies on several others in multiple projects. Instead, +use the draft (or ready) state to track the readiness of an individual +merge request, and a merge request dependency to enforce merge order. + +NOTE: +Merge request dependencies are a **PREMIUM** feature, but this restriction is +enforced only for the dependent merge request. A merge request in a **PREMIUM** +project can depend on a merge request in a **FREE** project, but a merge request +in a **FREE** project cannot be marked as dependent. + +## View dependencies for a merge request + +If a merge request is dependent on another, the merge request reports section shows +information about the dependency: + + + +To view dependency information on a merge request: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and identify your merge request. +1. Scroll to the merge request reports area. Dependent merge requests display information + about the total number of dependencies set, such as + **(status-warning)** **Depends on 1 merge request being merged**. +1. Select **Expand** to view the title, milestone, assignee, and pipeline status + of each dependency. + +Until your merge request's dependencies all merge, your merge request +cannot be merged. The message +**Merge blocked: you can only merge after the above items are resolved** displays. + +### Closed merge requests + +Closed merge requests still prevent their dependents from being merged, because +a merge request can close regardless of whether or not the planned work actually merged. + +If a merge request closes and the dependency is no longer relevant, +remove it as a dependency to unblock the dependent merge request. + +## Create a new dependent merge request + +When you create a new merge request, you can prevent it from merging until after +other specific work merges, even if the merge request is in a different project. + +Prerequisites: + +- You must have at least the Developer role or be allowed to create merge requests in the project. +- The dependent merge request must be in a project in a **PREMIUM** or higher tier. + +To create a new merge request and mark it as dependent on another: + +1. [Create a new merge request](creating_merge_requests.md). +1. In **Merge request dependencies**, paste either the reference or the full URL + to the merge requests that should merge before this work merges. References + are in the form of `path/to/project!merge_request_id`. +1. Select **Create merge request**. + +## Edit a merge request to add a dependency + +You can edit an existing merge request and mark it as dependent on another. + +Prerequisite: + +- You must have at least the Developer role or be allowed to edit merge requests in the project. + +To do this: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and identify your merge request. +1. Select **Edit**. +1. In **Merge request dependencies**, paste either the reference or the full URL + to the merge requests that should merge before this work merges. References + are in the form of `path/to/project!merge_request_id`. + +## Remove a dependency from a merge request + +You can edit a dependent merge request and remove a dependency. + +Prerequisite: + +- You must have a role in the project that allows you to edit merge requests. + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and identify your merge request. +1. Select **Edit**. +1. Scroll to **Merge request dependencies** and select **Remove** next to the reference + for each dependency you want to remove. + + NOTE: + Dependencies for merge requests you don't have access to are displayed as + **1 inaccessible merge request**, and can be removed the same way. +1. Select **Save changes**. + +## Troubleshooting + +### API support for managing merge request dependencies + +No API support exists for managing dependencies. For more information, read +[issue #12551](https://gitlab.com/gitlab-org/gitlab/-/issues/12551). + +### Preserving dependencies on project import or export + +Dependencies are not preserved when projects are imported or exported. For more +information, read [issue #12549](https://gitlab.com/gitlab-org/gitlab/-/issues/12549). + +### Complex merge order dependencies are unsupported + +GitLab supports direct dependencies between merge requests, but does not support +[indirect (nested) dependencies](https://gitlab.com/gitlab-org/gitlab/-/issues/11393). + +Acceptable dependency patterns include: + +- A single merge request can directly depend on a single merge request. +- A single merge request can directly depend on multiple merge requests. +- Multiple merge requests can directly depend on a single merge request. + +The indirect, nested dependency between `myfriend/library!10` and `mycorp/example!100` shown in this example is not supported: + +```mermaid +graph LR; + A[myfriend/library!10]-->|depends on| B[herfriend/another-lib!1] + B-->|depends on| C[mycorp/example!100] +``` diff --git a/doc/user/project/merge_requests/drafts.md b/doc/user/project/merge_requests/drafts.md index 4bb6034c0bd..695c6d7e612 100644 --- a/doc/user/project/merge_requests/drafts.md +++ b/doc/user/project/merge_requests/drafts.md @@ -20,6 +20,7 @@ the **Merge** button until you remove the **Draft** flag: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32692) in GitLab 13.2, Work-In-Progress (WIP) merge requests were renamed to **Draft**. > - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/228685) all support for using **WIP** in GitLab 14.8. > - **Mark as draft** and **Mark as ready** buttons [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227421) in GitLab 13.5. +> `/draft` quick action as a toggle [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92654) in GitLab 15.4. There are several ways to flag a merge request as a draft: @@ -29,8 +30,7 @@ There are several ways to flag a merge request as a draft: below the **Title** field. - **Commenting in an existing merge request**: Add the `/draft` [quick action](../quick_actions.md#issues-merge-requests-and-epics) - in a comment. This quick action is a toggle, and can be repeated to change the status - back to Ready. + in a comment. GitLab 15.4 [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92654) the toggle behavior of `/draft`. To mark a merge request as ready, use `/ready`. - **Creating a commit**: Add `draft:`, `Draft:`, `fixup!`, or `Fixup!` to the beginning of a commit message targeting the merge request's source branch. This is not a toggle, and adding this text again in a later commit doesn't mark the diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/project/merge_requests/getting_started.md index 09ee828ffd3..9475c0d60ab 100644 --- a/doc/user/project/merge_requests/getting_started.md +++ b/doc/user/project/merge_requests/getting_started.md @@ -66,7 +66,7 @@ After you have created the merge request, you can also: - [Discuss](../../discussions/index.md) your implementation with your team in the merge request thread. - [Perform inline code reviews](reviews/index.md). -- Add [merge request dependencies](merge_request_dependencies.md) to restrict it to be merged only when other merge requests have been merged. +- Add [merge request dependencies](dependencies.md) to restrict it to be merged only when other merge requests have been merged. - Preview continuous integration [pipelines on the merge request widget](widgets.md). - Preview how your changes look directly on your deployed application with [Review Apps](widgets.md#live-preview-with-review-apps). - [Allow collaboration on merge requests across forks](allow_collaboration.md). diff --git a/doc/user/project/merge_requests/img/cancel-mwps_v15_4.png b/doc/user/project/merge_requests/img/cancel-mwps_v15_4.png Binary files differnew file mode 100644 index 00000000000..7ed780d4389 --- /dev/null +++ b/doc/user/project/merge_requests/img/cancel-mwps_v15_4.png diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png b/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png Binary files differdeleted file mode 100644 index c98821548f8..00000000000 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png b/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png Binary files differdeleted file mode 100644 index 8b51503419b..00000000000 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/cherry_pick_mr_timeline_v12_9.png b/doc/user/project/merge_requests/img/cherry_pick_mr_timeline_v12_9.png Binary files differdeleted file mode 100644 index 919b576fcc6..00000000000 --- a/doc/user/project/merge_requests/img/cherry_pick_mr_timeline_v12_9.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/cherry_pick_mr_timeline_v15_4.png b/doc/user/project/merge_requests/img/cherry_pick_mr_timeline_v15_4.png Binary files differnew file mode 100644 index 00000000000..d18c4aaec20 --- /dev/null +++ b/doc/user/project/merge_requests/img/cherry_pick_mr_timeline_v15_4.png diff --git a/doc/user/project/merge_requests/img/cherry_pick_v15_4.png b/doc/user/project/merge_requests/img/cherry_pick_v15_4.png Binary files differnew file mode 100644 index 00000000000..174bb113961 --- /dev/null +++ b/doc/user/project/merge_requests/img/cherry_pick_v15_4.png diff --git a/doc/user/project/merge_requests/img/dependencies_edit_v12_4.png b/doc/user/project/merge_requests/img/dependencies_edit_v12_4.png Binary files differdeleted file mode 100644 index 4edf0648794..00000000000 --- a/doc/user/project/merge_requests/img/dependencies_edit_v12_4.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/dependencies_view_v15_3.png b/doc/user/project/merge_requests/img/dependencies_view_v15_3.png Binary files differnew file mode 100644 index 00000000000..d044e28046f --- /dev/null +++ b/doc/user/project/merge_requests/img/dependencies_view_v15_3.png diff --git a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png b/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png Binary files differdeleted file mode 100644 index 9487264b41a..00000000000 --- a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_status.png b/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_status.png Binary files differdeleted file mode 100644 index 70fa2efc855..00000000000 --- a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_status.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/mwps_v15_4.png b/doc/user/project/merge_requests/img/mwps_v15_4.png Binary files differnew file mode 100644 index 00000000000..f042912d470 --- /dev/null +++ b/doc/user/project/merge_requests/img/mwps_v15_4.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 35ec075c674..500fb95c193 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -27,7 +27,7 @@ You can view merge requests for your project, group, or yourself. To view all merge requests for a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests**. Or, to use a [keyboard shortcut](../../shortcuts.md), press <kbd>g</kbd> + <kbd>m</kbd>. @@ -36,7 +36,7 @@ Or, to use a [keyboard shortcut](../../shortcuts.md), press <kbd>g</kbd> + <kbd> To view merge requests for all projects in a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Merge requests**. If your group contains subgroups, this view also displays merge requests from the subgroup projects. @@ -160,13 +160,11 @@ change and whether you need access to a development environment: ## Assign a user to a merge request -When a merge request is created, it's assigned by default to the person who created it. -This person owns the merge request, but isn't responsible for [reviewing it](reviews/index.md). -To assign the merge request to someone else, use the `/assign @user` +To assign the merge request to a user, use the `/assign @user` [quick action](../quick_actions.md#issues-merge-requests-and-epics) in a text area in a merge request, or: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests** and find your merge request. 1. On the right sidebar, expand the right sidebar and locate the **Assignees** section. 1. Select **Edit**. @@ -186,7 +184,7 @@ accountable for it: To assign multiple assignees to a merge request, use the `/assign @user` [quick action](../quick_actions.md#issues-merge-requests-and-epics) in a text area, or: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests** and find your merge request. 1. On the right sidebar, expand the right sidebar and locate the **Assignees** section. 1. Select **Edit** and, from the dropdown list, select all users you want diff --git a/doc/user/project/merge_requests/merge_request_dependencies.md b/doc/user/project/merge_requests/merge_request_dependencies.md index 6bfef6ab134..6242a77e931 100644 --- a/doc/user/project/merge_requests/merge_request_dependencies.md +++ b/doc/user/project/merge_requests/merge_request_dependencies.md @@ -1,141 +1,11 @@ --- -stage: Create -group: Code Review -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, concepts +redirect_to: 'dependencies.md' +remove_date: '2022-11-22' --- -# Merge request dependencies **(PREMIUM)** +This document was moved to [another location](dependencies.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9688) in GitLab 12.2. -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17291) from "Cross-project dependencies" to "Merge request dependencies" in GitLab 12.4. -> - Intra-project MR dependencies were [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16799) in GitLab 12.4. - -Merge request dependencies allows a required order of merging -between merge requests to be expressed. If a merge request "depends on" another, -then it cannot be merged until its dependency is itself merged. - -NOTE: -Merge requests dependencies are a **PREMIUM** feature, but this restriction is -only enforced for the dependent merge request. A merge request in a **FREE** -project can be a dependency of a **PREMIUM** merge request, but not -the other way around. - -## Use cases - -- Ensure changes to a library are merged before changes to a project that - imports the library. -- Prevent a documentation-only merge request from being merged before the merge request - implementing the feature to be documented. -- Require a merge request updating a permissions matrix to be merged before merging a - merge request from someone who hasn't yet been granted permissions. - -It is common for a single logical change to span several merge requests, spread -out across multiple projects, and the order in which they are merged can be -significant. - -For example, given a project `mycorp/awesome-project` that imports a library -at `myfriend/awesome-lib`, adding a feature in `awesome-project` may **also** -require changes to `awesome-lib`, and so necessitate two merge requests. Merging -the `awesome-project` merge request before the `awesome-lib` one would -break the default branch. - -The `awesome-project` merge request could be [marked as **Draft**](drafts.md), -and the reason for the draft stated included in the comments. However, this -requires the state of the `awesome-lib` merge request to be manually -tracked, and doesn't scale well if the `awesome-project` merge request -depends on changes to **several** other projects. - -By making the `awesome-project` merge request depend on the -`awesome-lib` merge request instead, this relationship is -automatically tracked by GitLab, and the draft state can be used to -communicate the readiness of the code in each individual merge request -instead. - -## Configuration - -To continue the above example, you can configure a dependency when creating the -new merge request in `awesome-project` (or by editing it, if it already exists). -The dependency needs to be configured on the **dependent** merge -request. There is a **Merge request dependencies** section in the form: - - - -Anyone who can edit a merge request can change the list of dependencies. - -New dependencies can be added by reference, or by URL. To remove a dependency, -press the **X** by its reference. - -As dependencies can be specified across projects, it's possible that someone else -has added a dependency for a merge request in a project you don't have access to. -These are shown as a simple count: - - - -If necessary, you can remove all the dependencies like this by pressing the -**X**, just as you would for a single, visible dependency. - -Once you're finished, press the **Save changes** button to submit the request, -or **Cancel** to return without making any changes. - -The list of configured dependencies, and the status of each one, is shown in the -merge request widget: - - - -Until all dependencies have, themselves, been merged, the **Merge** -button is disabled for the dependent merge request. In -particular, note that **closed merge requests** still prevent their -dependents from being merged - it is impossible to automatically -determine whether the dependency expressed by a closed merge request -has been satisfied in some other way or not. - -If a merge request has been closed **and** the dependency is no longer relevant, -it must be removed as a dependency, following the instructions above, before -merge. - -## Limitations - -- API support: [issue #12551](https://gitlab.com/gitlab-org/gitlab/-/issues/12551) -- Dependencies are not preserved across project export/import: [issue #12549](https://gitlab.com/gitlab-org/gitlab/-/issues/12549) -- Complex merge order dependencies are not supported: [issue #11393](https://gitlab.com/gitlab-org/gitlab/-/issues/11393) - -The last item merits a little more explanation. Dependencies between merge -requests can be described as a graph of relationships. The simplest possible -graph has one merge request that depends upon another: - -```mermaid -graph LR; - myfriend/awesome-lib!10-->mycorp/awesome-project!100; -``` - -A more complex (and still supported) graph might have one merge request that -directly depends upon several others: - -```mermaid -graph LR; - myfriend/awesome-lib!10-->mycorp/awesome-project!100; - herfriend/another-lib!1-->mycorp/awesome-project!100; -``` - -Several different merge requests can also directly depend upon the -same merge request: - -```mermaid -graph LR; - herfriend/another-lib!1-->myfriend/awesome-lib!10; - herfriend/another-lib!1-->mycorp/awesome-project!100; -``` - -What is **not** supported is a "deep", or "nested" graph of dependencies. For example: - -```mermaid -graph LR; - herfriend/another-lib!1-->myfriend/awesome-lib!10; - myfriend/awesome-lib!10-->mycorp/awesome-project!100; -``` - -In this example, `myfriend/awesome-lib!10` depends on `herfriend/another-lib!1`, -and is itself a dependent of `mycorp/awesome-project!100`. This means that -`myfriend/awesome-lib!10` becomes an **indirect** dependency of -`mycorp/awesome-project!100`, which is not yet supported. +<!-- This redirect file can be deleted after <2022-11-22>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index 9182cf11566..57c4ff455cb 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -7,125 +7,143 @@ type: reference, concepts # Merge when pipeline succeeds **(FREE)** -When reviewing a merge request that looks ready to merge but still has a -pipeline running, you can set it to merge automatically when the -pipeline succeeds. This way, you don't have to wait for the pipeline to -finish and remember to merge the request manually. +If you review a merge request and it's ready to merge, but the pipeline hasn't +completed yet, you can set it to merge when the pipeline succeeds (MWPS). You don't +have to remember later to merge the work manually: - + -## How it works +If the pipeline succeeds, the merge request is merged. If the pipeline fails, the +author can either retry any failed jobs, or push new commits to fix the failure: -When you select "Merge When Pipeline Succeeds", the status of the merge -request is updated to show the impending merge. If you can't wait -for the pipeline to succeed, you can choose **Merge immediately** -in the dropdown menu on the right of the main button. +- If a retried job succeeds on the second try, the merge request is merged. +- If new commits are added to the merge request, GitLab cancels the MWPS request + to ensure the new changes are reviewed before merge. -The author of the merge request and project members with the Developer role can -cancel the automatic merge at any time before the pipeline finishes. +## Set a merge request to MWPS - +Prerequisites: -When the pipeline succeeds, the merge request is automatically merged. -When the pipeline fails, the author gets a chance to retry any failed jobs, -or to push new commits to fix the failure. +- You must have at least the Developer role in the project. +- If the project is configured to require it, all threads in the + merge request [must be resolved](../../discussions/index.md#resolve-a-thread). +- The merge request must have received all required approvals. -When the jobs are retried and succeed on the second try, the merge request -is automatically merged. When the merge request is updated with -new commits, the automatic merge is canceled to allow the new -changes to be reviewed. +To do this when pushing from the command line, use the `merge_request.merge_when_pipeline_succeeds` +[push option](../push_options.md). -By default, all threads must be resolved before you see the **Merge when -pipeline succeeds** button. If someone adds a new comment after -the button is selected, but before the jobs in the CI pipeline are -complete, the merge is blocked until you resolve all existing threads. +To do this from the GitLab user interface: -## Only allow merge requests to be merged if the pipeline succeeds +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests**. +1. Scroll to the merge request reports section. +1. Optional. Select your desired merge options, such as **Delete source branch**, + **Squash commits**, or **Edit commit message**. +1. Select **Merge when pipeline succeeds**. -You can prevent merge requests from being merged if: +If a new comment is added to the merge request after you select **Merge when pipeline succeeds**, +but before the pipeline completes, GitLab blocks the merge until you +resolve all existing threads. -- No pipeline ran. -- The pipeline did not succeed. +## Cancel an auto-merge -This works for both: +If a merge request is set to MWPS, you can cancel it. -- GitLab CI/CD pipelines -- Pipelines run from an [external CI integration](../integrations/index.md#available-integrations) +Prerequisites: + +- You must either be the author of the merge request, or a project member with + at least the Developer role. +- The merge request's pipeline must still be in progress. + +To do this: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests**. +1. Scroll to the merge request reports section. +1. Select **Cancel auto-merge**. + + + +## Require a successful pipeline for merge + +You can configure your project to require a complete and successful pipeline before +merge. This configuration works for both: + +- GitLab CI/CD pipelines. +- Pipelines run from an [external CI integration](../integrations/index.md#available-integrations). As a result, [disabling GitLab CI/CD pipelines](../../../ci/enable_or_disable_ci.md) -does not disable this feature, as it is possible to use pipelines from external -CI providers with this feature. To enable it, you must: +does not disable this feature, but you can use pipelines from external +CI providers with it. + +Prerequisites: + +- Ensure CI/CD is configured to run a pipeline for every merge request. +- You must have at least the Maintainer role in the project. + +To enable this setting: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. +1. Scroll to **Merge checks**, and select **Pipelines must succeed**. + This setting also prevents merge requests from being merged if there is no pipeline, + which can [conflict with some rules](#merge-requests-dont-merge-when-successful-pipeline-is-required). +1. Select **Save**. + +### Allow merge after skipped pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211482) in GitLab 13.1. + +When the **Pipelines must succeed** checkbox is checked, +[skipped pipelines](../../../ci/pipelines/index.md#skip-a-pipeline) prevent +merge requests from being merged. + +Prerequisite: -1. On the top bar, select **Menu > Projects** and find your project. +- You must have at least the Maintainer role in the project. + +To change this behavior: + +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Merge requests**. -1. Under **Merge checks**, select the **Pipelines must succeed** checkbox. +1. Under **Merge checks**: + - Select **Pipelines must succeed**. + - Select **Skipped pipelines are considered successful**. 1. Select **Save**. -This setting also prevents merge requests from being merged if there is no pipeline. -You should be careful to configure CI/CD so that pipelines run for every merge request. +## Troubleshooting -### Limitations +### Merge requests don't merge when successful pipeline is required -When this setting is enabled, a merge request is prevented from being merged if there -is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/index.md#only--except) -or [`rules`](../../../ci/yaml/index.md#rules) are used and they don't generate any pipelines. +If you require a successful pipeline for a merge, this setting can conflict with some +use cases that do not generate pipelines, such as [`only/except`](../../../ci/yaml/index.md#only--except) +or [`rules`](../../../ci/yaml/index.md#rules). Ensure your project +[runs a pipeline](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54226) for +every merge request, and that the pipeline is successful. -You should ensure that [there is always a pipeline](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54226) -and that it's successful. +### Ensure test parity between pipeline types -If both a branch pipeline and a merge request pipeline are triggered for a single -merge request, only the success or failure of the *merge request pipeline* is checked. -If the merge request pipeline is configured with fewer jobs than the branch pipeline, -it could allow code that fails tests to be merged: +If a merge request triggers both a branch pipeline and a merge request pipeline, +the success or failure of only the *merge request pipeline* is checked. +If the merge request pipeline contains fewer jobs than the branch pipeline, +it could allow code that fails tests to be merged, like in this example: ```yaml branch-pipeline-job: rules: - if: $CI_PIPELINE_SOURCE == "push" script: - - echo "Code testing scripts here, for example." + - echo "Testing happens here." merge-request-pipeline-job: rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" script: - - echo "No tests run, but this pipeline always succeeds and enables merge." + - echo "No testing happens here. This pipeline always succeeds, and enables merge." - echo true ``` -You should avoid configuration like this, and only use branch (`push`) pipelines -or merge request pipelines, when possible. See [`rules` documentation](../../../ci/jobs/job_control.md#avoid-duplicate-pipelines) -for details on avoiding two pipelines for a single merge request. - -### Skipped pipelines - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211482) in GitLab 13.1. - -When the **Pipelines must succeed** checkbox is checked, [skipped pipelines](../../../ci/pipelines/index.md#skip-a-pipeline) prevent -merge requests from being merged. To change this behavior: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. -1. Under **Merge checks**: - - Ensure **Pipelines must succeed** is selected. - - Select the **Skipped pipelines are considered successful** checkbox. -1. Select **Save**. - -## From the command line - -You can use [Push Options](../push_options.md) to enable merge when pipeline succeeds -for a merge request when pushing from the command line. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +Instead, use branch (`push`) pipelines or merge request pipelines, when possible. +For details on avoiding two pipelines for a single merge request, read the +[`rules` documentation](../../../ci/jobs/job_control.md#avoid-duplicate-pipelines). diff --git a/doc/user/project/merge_requests/methods/index.md b/doc/user/project/merge_requests/methods/index.md index c4e4b40dc48..68dd6477408 100644 --- a/doc/user/project/merge_requests/methods/index.md +++ b/doc/user/project/merge_requests/methods/index.md @@ -12,9 +12,8 @@ merge requests are merged into an existing branch. ## Configure a project's merge method -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. 1. In the **Merge method** section, select your desired merge method. 1. Select **Save changes**. @@ -110,7 +109,7 @@ gitGraph ``` This method is equivalent to `git merge --ff <source-branch>` for regular merges, and to -`git merge -squash <source-branch>` for squash merges. +`git merge --squash <source-branch>` for squash merges. When the fast-forward merge ([`--ff-only`](https://git-scm.com/docs/git-merge#git-merge---ff-only)) setting diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md index 8f433c13887..a6e0740ff78 100644 --- a/doc/user/project/merge_requests/revert_changes.md +++ b/doc/user/project/merge_requests/revert_changes.md @@ -6,50 +6,86 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Revert changes **(FREE)** -You can use Git's powerful feature to [revert any commit](https://git-scm.com/docs/git-revert "Git revert documentation") -by clicking the **Revert** button in merge requests and commit details. +You can revert individual commits or an entire merge request in GitLab. +When you revert a commit in Git, you create a new commit that reverses all actions +taken in the original commit: + +- Lines added in the original commit are removed. +- Lines removed in the original commit are added back. +- Lines modified in the original commit are restored to their previous state. + +Your **revert commit** is still subject to your project's access controls and processes. ## Revert a merge request -NOTE: -The **Revert** button is shown only for projects that use the -merge method "Merge Commit", which can be set under the project's -**Settings > General > Merge request**. [Fast-forward commits](methods/index.md#fast-forward-merge) -can not be reverted by using the merge request view. +After a merge request is merged, you can revert all changes in the merge request. + +Prerequisites: -After the merge request has been merged, use the **Revert** button -to revert the changes introduced by that merge request. +- You must have a role in the project that allows you to edit merge requests, and add + code to the repository. +- Your project must use the [merge method](methods/index.md#fast-forward-merge) **Merge Commit**, + which is set in the project's **Settings > General > Merge request**. You can't revert + fast-forwarded commits from the GitLab UI. - +To do this: -After you select that button, a modal appears where you can choose to -revert the changes directly into the selected branch or you can opt to -create a new merge request with the revert changes. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and identify your merge request. +1. Scroll to the merge request reports area, and find the report showing when the + merge request was merged. +1. Select **Revert**. +1. In **Revert in branch**, select the branch to revert your changes into. +1. Optional. Select **Start a new merge request** to start a new merge request with the new revert commit. +1. Select **Revert**. -After the merge request has been reverted, the **Revert** button is no longer available. +The option to **Revert** is no longer shown after a merge request is reverted. ## Revert a commit -You can revert a commit from the commit details page: +You can revert any commit in a repository into either: + +- The current branch. +- A new merge request. + +Prerequisites: + +- You must have a role in the project that allows you to edit merge requests, and add + code to the repository. + +To do this: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. If you know the merge request that contains the commit: + 1. On the left sidebar, select **Merge requests** and identify your merge request. + 1. Select **Commits**, then select the title of the commit you want to revert. GitLab displays the contents of the commit. +1. If you don't know the merge request the commit originated from: + 1. On the left sidebar, select **Repository > Commits**. + 1. Select the title of the commit to display full information about the commit. +1. In the top right corner, select **Options**, then select **Revert**. +1. In **Revert in branch**, select the branch to revert your changes into. +1. Optional. Select **Start a new merge request** to start a new merge request with the new revert commit. +1. Select **Revert**. + +The option to **Revert** is no longer shown after a commit is reverted. - +### Revert a merge commit to a different parent commit -Similar to reverting a merge request, you can opt to revert the changes -directly into the target branch or create a new merge request to revert the -changes. +When you revert a merge commit, the branch you merged to (usually `main`) is always the +first parent. To revert a merge commit to a different parent, +you must revert the commit from the command line: -After a commit is reverted, the **Revert** button is no longer available. +1. Identify the SHA of the parent commit you want to revert to. +1. Identify the parent number of the commit you want to revert to. (Defaults to 1, for the first parent.) +1. Modify this command, replacing `2` with the parent number, and `7a39eb0` with the commit SHA: -When reverting merge commits, the mainline is always the -first parent. If you want to use a different mainline, you need to do that -from the command line. + ```shell + git revert -m 2 7a39eb0 + ``` -Here's an example to revert a merge commit using the second parent as the -mainline: +## Related topics -```shell -git revert -m 2 7a39eb0 -``` +- [Official `git revert` documentation](https://git-scm.com/docs/git-revert) <!-- ## Troubleshooting diff --git a/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v15_3.png b/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v15_3.png Binary files differdeleted file mode 100644 index 38e18115803..00000000000 --- a/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v15_3.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v15_4.png b/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v15_4.png Binary files differnew file mode 100644 index 00000000000..47b7be3886d --- /dev/null +++ b/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v15_4.png diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md index 27b223c48ec..78f82b019b8 100644 --- a/doc/user/project/merge_requests/reviews/index.md +++ b/doc/user/project/merge_requests/reviews/index.md @@ -56,12 +56,11 @@ displays next to your name. You can submit your completed review in multiple ways: - Use the `/submit_review` [quick action](../../quick_actions.md) in the text of a non-review comment. -- Select **Finish review** and then **Submit review** in the footer at the bottom of the screen. +- Select **Finish review**, then select **Submit review** at the bottom of the modal window. + In the modal window, you can supply a **Summary comment**, approve the merge request, and + include quick actions: -Selecting **Finish review** opens a modal window to add an optional comment to summarize your review. -You can also include quick actions: - - +  When you submit your review, GitLab: @@ -69,6 +68,7 @@ When you submit your review, GitLab: - Sends a single email to every notifiable user of the merge request, with your review comments attached. Replying to this email creates a new comment on the merge request. - Perform any quick actions you added to your review comments. +- Optional. Approves the merge request. ### Resolve or unresolve thread with a comment @@ -96,7 +96,7 @@ If you have a review in progress, you can also add a comment from the **Overview ### Approval Rule information for Reviewers **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233736) in GitLab 13.8. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/293742) in GitLab 13.9. +> - [Feature flag `reviewer_approval_rules` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/293742) in GitLab 13.9. When editing the **Reviewers** field in a new or existing merge request, GitLab displays the name of the matching [approval rule](../approvals/rules.md) diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index 7e37990b9bf..066149afbb5 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -60,9 +60,8 @@ squash the commits as part of the merge process: To configure the default squashing behavior for all merge requests in your project: -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. 1. In the **Squash commits when merging** section, select your desired behavior: - **Do not allow**: Squashing is never performed, and the option is not displayed. - **Allow**: Squashing is allowed, but cleared by default. diff --git a/doc/user/project/merge_requests/status_checks.md b/doc/user/project/merge_requests/status_checks.md index 0d7794a3ebd..da705a53153 100644 --- a/doc/user/project/merge_requests/status_checks.md +++ b/doc/user/project/merge_requests/status_checks.md @@ -1,5 +1,5 @@ --- -stage: Manage +stage: Govern group: Compliance info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" type: reference, concepts @@ -61,9 +61,8 @@ using the API. You don't need to wait for a merge request webhook payload to be Within each project's settings, you can see a list of status checks added to the project: -1. In your project, go to **Settings > General**. -1. Expand the **Merge requests** section. -1. Scroll down to the **Status checks** sub-section. +1. In your project, go to **Settings > Merge requests** section. +1. Scroll down to **Status checks**.  diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index ef734225fb4..723ca17ee56 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -37,27 +37,75 @@ For information about project and group milestones API, see: To view the milestone list: -1. On the top bar, select **Menu > Projects** and find your project or - **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Projects** and find your project or + **Main menu > Groups** and find your group. 1. Select **Issues > Milestones**. In a project, GitLab displays milestones that belong to the project. In a group, GitLab displays milestones that belong to the group and all projects in the group. -NOTE: +### View milestones in a project with issues turned off + If a project has issue tracking [turned off](../settings/index.md#configure-project-visibility-features-and-permissions), you can get to the milestones page -by going to its URL. To do so, add: `/-/milestones` to your project or group URL. -For example `https://gitlab.com/gitlab-org/sample-data-templates/sample-gitlab-project/-/milestones`. -This is tracked in [issue 339009](https://gitlab.com/gitlab-org/gitlab/-/issues/339009). +by going to its URL. + +To do so: + +1. Go to your project. +1. Add: `/-/milestones` to your project URL. + For example `https://gitlab.com/gitlab-org/sample-data-templates/sample-gitlab-project/-/milestones`. + +Alternatively, this project's issues are visible in the group's milestone page. + +Improving this experience is tracked in issue [339009](https://gitlab.com/gitlab-org/gitlab/-/issues/339009). ### View all milestones You can view all the milestones you have access to in the entire GitLab namespace. You might not see some milestones because they're in projects or groups you're not a member of. -To do so, on the top bar select **Menu > Milestones**. +To do so, on the top bar select **Main menu > Milestones**. + +### View milestone details + +To view more information about a milestone, +in the milestone list select the title of the milestone you want to view. + +The milestone view shows the title and description. + +There are also tabs below these that show the following: + +- **Issues**: Shows all issues assigned to the milestone. These are displayed in three columns named: + - Unstarted Issues (open and unassigned) + - Ongoing Issues (open and assigned) + - Completed Issues (closed) +- **Merge Requests**: Shows all merge requests assigned to the milestone. These are displayed in four columns named: + - Work in progress (open and unassigned) + - Waiting for merge (open and assigned) + - Rejected (closed) + - Merged +- **Participants**: Shows all assignees of issues assigned to the milestone. +- **Labels**: Shows all labels that are used in issues assigned to the milestone. + +#### Burndown charts + +The milestone view contains a [burndown and burnup chart](burndown_and_burnup_charts.md), +showing the progress of completing a milestone. + + + +#### Milestone sidebar + +The milestone sidebar on the milestone view shows the following: + +- Percentage complete, which is calculated as number of closed issues divided by total number of issues. +- The start date and due date. +- The total time spent on all issues and merge requests assigned to the milestone. +- The total issue weight of all issues assigned to the milestone. + + ## Create a milestone @@ -71,7 +119,7 @@ Prerequisites: To create a milestone: -1. On the top bar, select **Menu > Projects** and find your project or **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Projects** and find your project or **Main menu > Groups** and find your group. 1. On the left sidebar, select **Issues > Milestones**. 1. Select **New milestone**. 1. Enter the title. @@ -90,7 +138,7 @@ Prerequisites: To edit a milestone: -1. On the top bar, select **Menu > Projects** and find your project or **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Projects** and find your project or **Main menu > Groups** and find your group. 1. Select a milestone's title. 1. Select **Edit**. 1. Edit the title, start date, due date, or description. @@ -106,7 +154,7 @@ Prerequisites: To edit a milestone: -1. On the top bar, select **Menu > Projects** and find your project or **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Projects** and find your project or **Main menu > Groups** and find your group. 1. Select a milestone's title. 1. Select **Delete**. 1. Select **Delete milestone**. @@ -131,7 +179,7 @@ Prerequisites: To promote a project milestone: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Either: - Select **Promote to Group Milestone** (**{level-up}**). - Select the milestone title, and then select **Promote**. @@ -153,13 +201,13 @@ To assign or unassign a milestone: You can also use the `/assign` [quick action](../quick_actions.md) in a comment. -## Filtering issues and merge requests by milestone +## Filter issues and merge requests by milestone -### Filtering in list pages +### Filters in list pages From the project and group issue/merge request list pages, you can filter by both group and project milestones. -### Filtering in issue boards +### Filters in issue boards From [project issue boards](../issue_board.md), you can filter by both group milestones and project milestones in: @@ -181,42 +229,6 @@ When filtering by milestone, in addition to choosing a specific project mileston - **Upcoming**: Show issues or merge requests that have been assigned the open milestone and has the nearest due date in the future. - **Started**: Show issues or merge requests that have an open assigned milestone with a start date that is before today. -## Milestone view - -The milestone view shows the title and description. - -There are also tabs below these that show the following: - -- **Issues**: Shows all issues assigned to the milestone. These are displayed in three columns named: - - Unstarted Issues (open and unassigned) - - Ongoing Issues (open and assigned) - - Completed Issues (closed) -- **Merge Requests**: Shows all merge requests assigned to the milestone. These are displayed in four columns named: - - Work in progress (open and unassigned) - - Waiting for merge (open and assigned) - - Rejected (closed) - - Merged -- **Participants**: Shows all assignees of issues assigned to the milestone. -- **Labels**: Shows all labels that are used in issues assigned to the milestone. - -### Burndown Charts - -The milestone view contains a [burndown and burnup chart](burndown_and_burnup_charts.md), -showing the progress of completing a milestone. - - - -### Milestone sidebar - -The milestone sidebar on the milestone view shows the following: - -- Percentage complete, which is calculated as number of closed issues divided by total number of issues. -- The start date and due date. -- The total time spent on all issues and merge requests assigned to the milestone. -- The total issue weight of all issues assigned to the milestone. - - - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index 2c668e2c409..03f8ecac77f 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -7,17 +7,22 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Custom domains and SSL/TLS certificates **(FREE)** -Setting up GitLab Pages with custom domains, and adding SSL/TLS certificates to them, are optional features of GitLab Pages. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/238461) in GitLab 15.4, you can use verified domains to [bypass user email confirmation for SAML- or SCIM-provisioned users](../../../group/saml_sso/index.md#bypass-user-email-confirmation-with-verified-domains). -To use one or more custom domain names with your Pages site, you can: +You can use custom domains: -- Add a [custom **root domain** or a **subdomain**](#set-up-pages-with-a-custom-domain). +- With GitLab Pages. +- To [bypass user email confirmation for SAML- or SCIM-provisioned users](../../../group/saml_sso/index.md#bypass-user-email-confirmation-with-verified-domains). + When using custom domains this way, you use the GitLab Pages feature but can skip the [requirements](#requirements). + +To use one or more custom domain names: + +- Add a [custom **root domain** or a **subdomain**](#set-up-a-custom-domain). - Add [SSL/TLS certification](#adding-an-ssltls-certificate-to-pages). -## Set up Pages with a custom domain +## Set up a custom domain -To set up Pages with a custom domain name, read the requirements -and steps below. +To set up Pages with a custom domain name, read the requirements and steps below. ### Requirements @@ -45,7 +50,7 @@ and steps below. Follow the steps below to add your custom domain to Pages. See also this document for an [overview on DNS records](dns_concepts.md). -#### 1. Add a custom domain to Pages +#### 1. Add a custom domain Navigate to your project's **Setting > Pages** and select **+ New domain** to add your custom domain to GitLab Pages. You can choose whether to: @@ -64,7 +69,7 @@ and paste them in your domain's control panel as a `TXT` record on the next step  -#### 3. Set up DNS records for Pages +#### 3. Set up DNS records Read this document for an [overview of DNS records for Pages](dns_concepts.md). If you're familiar with the subject, follow the instructions below @@ -144,7 +149,7 @@ They require: | `_gitlab-pages-verification-code.www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | If you're using Cloudflare, check -[Redirecting `www.domain.com` to `domain.com` with Cloudflare](#redirecting-wwwdomaincom-to-domaincom-with-cloudflare). +[Redirecting `www.domain.com` to `domain.com` with Cloudflare](#redirect-wwwdomaincom-to-domaincom-with-cloudflare). > **Notes**: > @@ -186,7 +191,7 @@ from the GitLab project. in place. Your domain is periodically reverified, and may be disabled if the record is removed. -##### Troubleshooting Pages domain verification +##### Troubleshoot domain verification To manually verify that you have properly configured the domain verification `TXT` DNS entry, you can run the following command in your terminal: @@ -218,7 +223,7 @@ For a subdomain: | `www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | | `_gitlab-pages-verification-code.www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | -### Adding more domain aliases +### Add more domain aliases You can add more than one alias (custom domains and subdomains) to the same project. An alias can be understood as having many doors leading to the same room. @@ -226,7 +231,7 @@ An alias can be understood as having many doors leading to the same room. All the aliases you've set to your site are listed on **Setting > Pages**. From that page, you can view, add, and remove them. -### Redirecting `www.domain.com` to `domain.com` with Cloudflare +### Redirect `www.domain.com` to `domain.com` with Cloudflare If you use Cloudflare, you can redirect `www` to `domain.com` without adding both `www.domain.com` and `domain.com` to GitLab. diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md index 0cc6cb808d1..b5487f7a465 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md @@ -30,7 +30,7 @@ Before you can enable automatic provisioning of an SSL certificate for your doma - Acquired a domain (`example.com`) and added a [DNS entry](index.md) pointing it to your Pages website. The top-level domain (`.com`) must be a [public suffix](https://publicsuffix.org/). -- [Added your domain to your Pages project](index.md#1-add-a-custom-domain-to-pages) +- [Added your domain to your Pages project](index.md#1-add-a-custom-domain) and verified your ownership. - Verified your website is up and running, accessible through your custom domain. @@ -76,7 +76,7 @@ If you get an error **Something went wrong while obtaining the Let's Encrypt cer 1. Make sure you have properly set only one `CNAME` or `A` DNS record for your domain. 1. Make sure your domain **doesn't have** an `AAAA` DNS record. 1. If you have a `CAA` DNS record for your domain or any higher level domains, make sure [it includes `letsencrypt.org`](https://letsencrypt.org/docs/caa/). - 1. Make sure [your domain is verified](index.md#1-add-a-custom-domain-to-pages). + 1. Make sure [your domain is verified](index.md#1-add-a-custom-domain). 1. Go to step 1. ### Message "GitLab is obtaining a Let's Encrypt SSL certificate for this domain. This process can take some time. Please try again later." hangs for more than an hour @@ -85,7 +85,7 @@ If you've enabled Let's Encrypt integration, but a certificate is absent after a 1. Go to your project's **Settings > Pages**. 1. Select **Remove** on your domain. -1. [Add the domain again and verify it](index.md#1-add-a-custom-domain-to-pages). +1. [Add the domain again and verify it](index.md#1-add-a-custom-domain). 1. [Enable Let's Encrypt integration for your domain](#enabling-lets-encrypt-integration-for-your-custom-domain). 1. If you still see the same message after some time: 1. Make sure you have properly set only one `CNAME` or `A` DNS record for your domain. diff --git a/doc/user/project/pages/getting_started/pages_ci_cd_template.md b/doc/user/project/pages/getting_started/pages_ci_cd_template.md index 510f9332e7b..58e15104815 100644 --- a/doc/user/project/pages/getting_started/pages_ci_cd_template.md +++ b/doc/user/project/pages/getting_started/pages_ci_cd_template.md @@ -16,7 +16,7 @@ Use a `.gitlab-ci.yml` template when you have an existing project that you want Your GitLab repository should contain files specific to an SSG, or plain HTML. After you complete these steps, you may have to do additional configuration for the Pages site to generate properly. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select the project's name. 1. From the **Add** (**{plus}**) dropdown, select **New file**. 1. From the **Select a template type** dropdown, select `.gitlab-ci.yml`. diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md index 68a2a6a80ad..1c3d5d722cb 100644 --- a/doc/user/project/pages/getting_started/pages_from_scratch.md +++ b/doc/user/project/pages/getting_started/pages_from_scratch.md @@ -266,6 +266,7 @@ pages: - public rules: - if: $CI_COMMIT_BRANCH == "main" + environment: production ``` Now add another job to the CI file, telling it to @@ -289,6 +290,7 @@ pages: - public rules: - if: $CI_COMMIT_BRANCH == "main" + environment: production test: stage: test @@ -342,6 +344,7 @@ pages: - public rules: - if: $CI_COMMIT_BRANCH == "main" + environment: production test: stage: test @@ -386,6 +389,7 @@ pages: - public rules: - if: $CI_COMMIT_BRANCH == "main" + environment: production test: stage: test @@ -420,7 +424,7 @@ Now GitLab CI/CD not only builds the website, but also: For more information, see the following blog posts. -- Use GitLab CI/CD `environments` to +- Use GitLab CI/CD `environments` to [deploy your web app to staging and production](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/). - Learn how to run jobs [sequentially, in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2016/07/29/the-basics-of-gitlab-ci/). diff --git a/doc/user/project/pages/getting_started/pages_ui.md b/doc/user/project/pages/getting_started/pages_ui.md new file mode 100644 index 00000000000..7e618fbaec8 --- /dev/null +++ b/doc/user/project/pages/getting_started/pages_ui.md @@ -0,0 +1,58 @@ +--- +stage: Create +group: Incubation +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Tutorial: Use the GitLab UI to deploy your static site **(FREE)** + +This tutorial assumes you have a project that either: + +- Generates static sites or a client-rendered single-page application (SPA), + such as [Eleventy](https://www.11ty.dev), [Astro](https://astro.build), or [Jekyll](https://jekyllrb.com). +- Contains a framework configured for static output, such as [Next.js](https://nextjs.org), + [Nuxt.js](https://nuxtjs.org), or [SvelteKit](https://kit.svelte.dev). + +## Update your app to output files to the `public` folder + +GitLab Pages requires all files intended to be part of the published website to +be in a root-level folder called `public`. If you create this folder during the build +pipeline, committing it to Git is not required. + +For detailed instructions, read [Configure the public files folder](../public_folder.md). + +## Set up the `.gitlab-ci.yml` file + +GitLab helps you write the `.gitlab-ci.yml` needed to create your first GitLab Pages +deployment pipeline. Rather than building the file from scratch, it asks you to +provide the build commands, and creates the necessary boilerplate for you. + +To build your YAML file from the GitLab UI: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Pages** to display the friendly + interface **Get Started With Pages**. +1. If your framework's build process does not need one of the provided build + commands, you can either: + - Skip the step by selecting **Next**. + - Enter `:` (the bash "do nothing" command) if you still want to incorporate that + step's boilerplate into your `.gitlab-ci.yml` file. +1. Optional. Edit and adjust the generated `.gitlab-ci.yml` file as needed. +1. Commit your `.gitlab-ci.yml` to your repository. This commit triggers your first + GitLab Pages deployment. + +## Troubleshooting + +### If you can't see the "Get Started with Pages" interface + +GitLab doesn't show this interface if you have either: + +- Deployed a GitLab Pages site before. +- Committed a `.gitlab-ci.yml` through this interface at least once. + +To fix this problem: + +- If you see the message **Waiting for the Pages Pipeline to complete**, select + **Start over** to start the wizard again. +- If your project has previously deployed GitLab Pages successfully, + [manually update](pages_from_scratch.md) your `.gitlab-ci.yml`. diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index af49522efe2..1f3628b74ec 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -38,12 +38,13 @@ Learn more about To create a GitLab Pages website: -| Document | Description | -|----------|-------------| +| Document | Description | +|--------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------| +| [Use the GitLab UI to create a simple `.gitlab-ci.yml`](getting_started/pages_ui.md) | Add a Pages site to an existing project. Use the UI to set up a simple `.gitlab-ci.yml`. | | [Create a `.gitlab-ci.yml` file from scratch](getting_started/pages_from_scratch.md) | Add a Pages site to an existing project. Learn how to create and configure your own CI file. | -| [Use a `.gitlab-ci.yml` template](getting_started/pages_ci_cd_template.md) | Add a Pages site to an existing project. Use a pre-populated CI template file. | -| [Fork a sample project](getting_started/pages_forked_sample_project.md) | Create a new project with Pages already configured by forking a sample project. | -| [Use a project template](getting_started/pages_new_project_template.md) | Create a new project with Pages already configured by using a template. | +| [Use a `.gitlab-ci.yml` template](getting_started/pages_ci_cd_template.md) | Add a Pages site to an existing project. Use a pre-populated CI template file. | +| [Fork a sample project](getting_started/pages_forked_sample_project.md) | Create a new project with Pages already configured by forking a sample project. | +| [Use a project template](getting_started/pages_new_project_template.md) | Create a new project with Pages already configured by using a template. | To update a GitLab Pages website: diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 3bd16a17f23..da024881ed6 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -314,6 +314,7 @@ pages: artifacts: paths: - public + environment: production ``` The `FF_USE_FASTZIP` variable enables the [feature flag](https://docs.gitlab.com/runner/configuration/feature-flags.html#available-feature-flags) which is needed for [`ARTIFACT_COMPRESSION_LEVEL`](../../../ci/runners/configure_runners.md#artifact-and-cache-settings). diff --git a/doc/user/project/pages/public_folder.md b/doc/user/project/pages/public_folder.md new file mode 100644 index 00000000000..f9c80875cc9 --- /dev/null +++ b/doc/user/project/pages/public_folder.md @@ -0,0 +1,153 @@ +--- +description: 'Learn how to configure the build output folder for the most +common static site generators' +stage: Create +group: Incubation +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Configure the public files folder **(FREE)** + +GitLab Pages requires all files you intend to be available in the published website to +be in a root-level folder called `public`. This page describe how +to set this up for some common static site generators. + +## Guide by framework + +### Eleventy + +For Eleventy, you should either: + +1. Add the `--output=public` flag in Eleventy's build commands, for example: + + `npx @11ty/eleventy --input=path/to/sourcefiles --output=public` + +1. Add the following to your `.eleventy.js` file: + + ```javascript + // .eleventy.js + module.exports = function(eleventyConfig) { + return { + dir: { + output: "public" + } + } + }; + ``` + +### Astro + +By default, Astro uses the `public` folder to store static assets. For GitLab Pages, +rename that folder to a collision-free alternative first: + +1. In your project directory, run: + + ```shell + mv public static + ``` + +1. Add the following to your `astro.config.mjs`. This code informs Astro about + our folder name remapping: + + ```javascript + // astro.config.mjs + export default { + // GitLab Pages requires exposed files to be located in a folder called "public". + // So we're instructing Astro to put the static build output in a folder of that name. + dist: 'public', + + // The folder name Astro uses for static files (`public`) is already reserved + // for the build output. So in deviation from the defaults we're using a folder + // called `static` instead. + public: 'static', + }; + ``` + +### SvelteKit + +NOTE: +GitLab Pages supports only static sites. For SvelteKit, +we recommend using [`adapter-static`](https://kit.svelte.dev/docs/adapters#supported-environments-static-sites). + +When using `adapter-static`, add the following to your `svelte.config.js`: + +```javascript +// svelte.config.js +import adapter from '@sveltejs/adapter-static'; + +export default { + kit: { + adapter: adapter({ + pages: 'public' + }) + } +}; +``` + +### Next.js + +NOTE: +GitLab Pages supports only static sites. For Next.js, we +recommend using Next's [Static HTML export functionality](https://nextjs.org/docs/advanced-features/static-html-export) + +Use the `-o public` flag after `next export` as the build command, for +example: + +```shell +next export -o public +``` + +### Nuxt.js + +NOTE: +GitLab Pages supports only static sites. + +1. Add the following to your `nuxt.config.js`: + + ```javascript + export default { + target: 'static', + generate: { + dir: 'public' + } + } + ``` + +1. Configure your Nuxt.js application for + [Static Site Generation](https://nuxtjs.org/docs/features/deployment-targets#static-hosting). + +### Vite + +Update your `vite.config.js` to include the following: + +```javascript +// vite.config.js +export default { + build: { + outDir: 'public' + } +} +``` + +### Webpack + +Update your `webpack.config.js` to include the following: + +```javascript +// webpack.config.js +module.exports = { + output: { + path: __dirname + '/public' + } +}; +``` + +## Should you commit the `public` folder? + +Not necessarily. However, when the GitLab Pages deploy pipeline runs, it looks +for an [artifact](../../../ci/pipelines/job_artifacts.md) of that name. So +If you set up a job that creates the `public` folder before deploy, such as by +running `npm run build`, committing the folder isn't required. + +If you prefer to build your site locally, you can commit the `public` folder and +omit the build step during the job, instead. diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 3e40a7962ae..9b685592c9d 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -41,7 +41,7 @@ Prerequisite: To protect a branch: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. @@ -63,7 +63,7 @@ Prerequisite: To protect multiple branches at the same time: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, type the branch name and a wildcard. @@ -96,7 +96,7 @@ from the command line or from a Git client application. To create a new branch through the user interface: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Branches**. 1. Select **New branch**. 1. Fill in the branch name and select an existing branch, tag, or commit to @@ -109,7 +109,7 @@ You can force everyone to submit a merge request, rather than allowing them to check in directly to a protected branch. This setting is compatible with workflows like the [GitLab workflow](../../topics/gitlab_flow.md). -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. @@ -125,7 +125,7 @@ like the [GitLab workflow](../../topics/gitlab_flow.md). You can allow everyone with write access to push to the protected branch. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. @@ -153,7 +153,7 @@ Prerequisites: To allow a deploy key to push to a protected branch: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. @@ -172,7 +172,7 @@ protected branches. To protect a new branch and enable force push: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. @@ -184,7 +184,7 @@ To protect a new branch and enable force push: To enable force pushes on branches that are already protected: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. In the list of protected branches, next to the branch, turn on the **Allowed to force push** toggle. @@ -200,7 +200,7 @@ For a protected branch, you can require at least one approval by a [Code Owner]( To protect a new branch and enable Code Owner's approval: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. @@ -210,7 +210,7 @@ To protect a new branch and enable Code Owner's approval: To enable Code Owner's approval on branches that are already protected: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. In the list of protected branches, next to the branch, turn on the **Code owner approval** toggle. @@ -242,7 +242,7 @@ for details about the pipelines security model. Users with at least the Maintainer role can manually delete protected branches by using the GitLab web interface: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Branches**. 1. Next to the branch you want to delete, select **Delete** (**{remove}**). 1. On the confirmation dialog, type the branch name. diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index 870c544cf4c..9b1e862af58 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -97,7 +97,7 @@ Prerequisite: To do this: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Tags**. 1. Next to the tag you want to delete, select **Delete** (**{remove}**). 1. On the confirmation dialog, enter the tag name and select **Yes, delete protected tag**. diff --git a/doc/user/project/push_options.md b/doc/user/project/push_options.md index d02609cbdc7..3eb333f5785 100644 --- a/doc/user/project/push_options.md +++ b/doc/user/project/push_options.md @@ -102,7 +102,7 @@ long Git commands. ### Merge when pipeline succeeds alias -To set up a Git alias for the +To set up a Git alias for the [merge when pipeline succeeds Git push option](#push-options-for-merge-requests): ```shell diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 216d040734d..8b8eba62cce 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -67,7 +67,7 @@ threads. Some quick actions might not be available to all subscription tiers. | `/copy_metadata <#issue>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another issue in the project. | | `/create_merge_request <branch name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Create a new merge request starting from the current issue. | | `/done` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mark to do as done. | -| `/draft` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Toggle the [draft status](merge_requests/drafts.md). | +| `/draft` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set the [draft status](merge_requests/drafts.md). Use for toggling the draft status ([deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92654) in GitLab 15.4.) | | `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. | | `/duplicate <#issue>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Close this issue and mark as a duplicate of another issue. Also, mark both as related. | | `/epic <epic>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. | @@ -100,7 +100,7 @@ threads. Some quick actions might not be available to all subscription tiers. | `/remove_milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove milestone. | | `/remove_parent_epic` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove parent epic from epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10556) in GitLab 12.1). | | `/remove_time_spent` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time spent. | -| `/remove_zoom` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove Zoom meeting from this issue ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16609) in GitLab 12.4). | +| `/remove_zoom` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove Zoom meeting from this issue. | | `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. | | `/severity <severity>` | **{check-circle}** Yes | **{check-circle}** No | **{check-circle}** No | Set the severity. Options for `<severity>` are `S1` ... `S4`, `critical`, `high`, `medium`, `low`, `unknown`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334045) in GitLab 14.2. | | `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\＿(ツ)＿/¯`. | @@ -110,6 +110,7 @@ threads. Some quick actions might not be available to all subscription tiers. | `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. | | `/target_branch <local branch name>` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set target branch. | | `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. | +| `/timeline <timeline comment> \| <date(YYYY-MM-DD)> <time(HH:MM)>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a timeline event to this incident. For example, `/timeline DB load spiked \| 2022-09-07 09:30`. ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368721) in GitLab 15.4). | | `/todo` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add a to-do item. | | `/unapprove` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Unapprove the merge request. ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8103) in GitLab 14.3 | | `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove specific assignees. | @@ -121,7 +122,7 @@ threads. Some quick actions might not be available to all subscription tiers. | `/unlock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Unlock the discussions. | | `/unsubscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Unsubscribe from notifications. | | `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, `2`, and so on. | -| `/zoom <Zoom URL>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add Zoom meeting to this issue ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16609) in GitLab 12.4). | +| `/zoom <Zoom URL>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a Zoom meeting to this issue or incident. In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/230853) users on GitLab Premium can add a short description when [adding a Zoom link to an incident](../../operations/incident_management/linked_resources.md#link-zoom-meetings-from-an-incident).| ## Commit messages diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index d3456e086ce..87ea95524fe 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -57,8 +57,6 @@ switch between ascending or descending order, select **Sort order**. ## Create a release -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32812) in GitLab 12.9. Releases can be created directly in the GitLab UI. - You can create a release: - [Using a job in your CI/CD pipeline](#creating-a-release-by-using-a-cicd-job). @@ -68,16 +66,16 @@ You can create a release: We recommend creating a release as one of the last steps in your CI/CD pipeline. +### Create a release in the Releases page + Prerequisites: - You must have at least the Developer role for a project. For more information, read [Release permissions](#release-permissions). -### Create a release in the Releases page - To create a release in the Releases page: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Releases** and select **New release**. 1. From the [**Tag name**](release_fields.md#tag-name) dropdown, either: - Select an existing Git tag. Selecting an existing tag that is already associated with a release @@ -99,7 +97,7 @@ To create a release in the Tags page, add release notes to either an existing or To add release notes to a new Git tag: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Tags**. 1. Select **New tag**. 1. Optional. Enter a tag message in the **Message** text box. @@ -109,7 +107,7 @@ To add release notes to a new Git tag: To edit release notes of an existing Git tag: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Tags**. 1. Select **Edit release notes** (**{pencil}**). 1. In the **Release notes** text box, enter the release's description. @@ -124,8 +122,11 @@ You can create a release directly as part of the GitLab CI/CD pipeline by using The release is created only if the job processes without error. If the API returns an error during release creation, the release job fails. -For examples of how you can create a release of your application in the CI/CD pipeline, -see [Release CI/CD examples](release_cicd_examples.md). +Methods for creating a release using a CI/CD job include: + +- [Create a release when a Git tag is created](release_cicd_examples.md#create-a-release-when-a-git-tag-is-created). +- [Create a release when a commit is merged to the default branch](release_cicd_examples.md#create-a-release-when-a-commit-is-merged-to-the-default-branch). +- [Create release metadata in a custom script](release_cicd_examples.md#create-release-metadata-in-a-custom-script). ### Use a custom SSL CA certificate authority @@ -232,7 +233,7 @@ Prerequisites: To delete a release in the UI: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Releases**. 1. In the top-right corner of the release you want to delete, select **Edit this release** (**{pencil}**). 1. On the **Edit Release** page, select **Delete**. @@ -312,6 +313,7 @@ deploy_to_production: script: deploy_to_prod.sh rules: - if: $CI_DEPLOY_FREEZE == null + environment: production ``` To set a deploy freeze window in the UI, complete these steps: diff --git a/doc/user/project/releases/release_cicd_examples.md b/doc/user/project/releases/release_cicd_examples.md index f1d3e55a707..bfd83a20caf 100644 --- a/doc/user/project/releases/release_cicd_examples.md +++ b/doc/user/project/releases/release_cicd_examples.md @@ -12,9 +12,13 @@ CI/CD pipeline. ## Create a release when a Git tag is created -In this CI/CD example, pushing a Git tag to the repository, or creating a Git tag in the UI triggers -the release. You can use this method if you prefer to create the Git tag manually, and create a -release as a result. +In this CI/CD example, the release is triggered by one of the following events: + +- Pushing a Git tag to the repository. +- Creating a Git tag in the UI. + +You can use this method if you prefer to create the Git tag manually, and create a release as a +result. NOTE: Do not provide Release notes when you create the Git tag in the UI. Providing release notes @@ -40,8 +44,8 @@ release_job: ## Create a release when a commit is merged to the default branch -In this CI/CD example, merging a commit to the default branch triggers the pipeline. You can use -this method if your release workflow does not create a tag manually. +In this CI/CD example, the release is triggered when you merge a commit to the default branch. You +can use this method if your release workflow does not create a tag manually. Key points in the following _extract_ of an example `.gitlab-ci.yml` file: @@ -69,16 +73,75 @@ Environment variables set in `before_script` or `script` are not available for e in the same job. Read more about [potentially making variables available for expanding](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6400). +## Create release metadata in a custom script + +In this CI/CD example the release preparation is split into separate jobs for greater flexibility: + +- The `prepare_job` job generates the release metadata. Any image can be used to run the job, + including a custom image. The generated metadata is stored in the variable file `variables.env`. + This metadata is [passed to the downstream job](../../../ci/variables/index.md#pass-an-environment-variable-to-another-job). +- The `release_job` uses the content from the variables file to create a release, using the + metadata passed to it in the variables file. This job must use the + `registry.gitlab.com/gitlab-org/release-cli:latest` image because it contains the release CLI. + +```yaml +prepare_job: + stage: prepare # This stage must run before the release stage + rules: + - if: $CI_COMMIT_TAG + when: never # Do not run this job when a tag is created manually + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run this job when commits are pushed or merged to the default branch + script: + - echo "EXTRA_DESCRIPTION=some message" >> variables.env # Generate the EXTRA_DESCRIPTION and TAG environment variables + - echo "TAG=v$(cat VERSION)" >> variables.env # and append to the variables.env file + artifacts: + reports: + dotenv: variables.env # Use artifacts:reports:dotenv to expose the variables to other jobs + +release_job: + stage: release + image: registry.gitlab.com/gitlab-org/release-cli:latest + needs: + - job: prepare_job + artifacts: true + rules: + - if: $CI_COMMIT_TAG + when: never # Do not run this job when a tag is created manually + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run this job when commits are pushed or merged to the default branch + script: + - echo "running release_job for $TAG" + release: + name: 'Release $TAG' + description: 'Created using the release-cli $EXTRA_DESCRIPTION' # $EXTRA_DESCRIPTION and the $TAG + tag_name: '$TAG' # variables must be defined elsewhere + ref: '$CI_COMMIT_SHA' # in the pipeline. For example, in the + milestones: # prepare_job + - 'm1' + - 'm2' + - 'm3' + released_at: '2020-07-15T08:00:00Z' # Optional, is auto generated if not defined, or can use a variable. + assets: + links: + - name: 'asset1' + url: 'https://example.com/assets/1' + - name: 'asset2' + url: 'https://example.com/assets/2' + filepath: '/pretty/url/1' # optional + link_type: 'other' # optional +``` + ## Skip multiple pipelines when creating a release Creating a release using a CI/CD job could potentially trigger multiple pipelines if the associated tag does not exist already. To understand how this might happen, consider the following workflows: - Tag first, release second: + 1. A tag is created via UI or pushed. 1. A tag pipeline is triggered, and runs `release` job. 1. A release is created. - Release first, tag second: + 1. A pipeline is triggered when commits are pushed or merged to default branch. The pipeline runs `release` job. 1. A release is created. 1. A tag is created. diff --git a/doc/user/project/releases/release_cli.md b/doc/user/project/releases/release_cli.md index 9e65ab4bc01..90363fca8b0 100644 --- a/doc/user/project/releases/release_cli.md +++ b/doc/user/project/releases/release_cli.md @@ -46,7 +46,7 @@ Once installed, [the `release` keyword](../../../ci/yaml/index.md#release) is av Or from the GitLab Package Registry: ```shell - curl --location --output /usr/local/bin/release-cli "https://gitlab.com/api/v4/projects/gitlab-org%2Frelease-cli/packages/generic/release-cli/latest/release-cli-darwin-amd64" + curl --location --output /usr/local/bin/release-cli "https://gitlab.com/api/v4/projects/gitlab-org%2Frelease-cli/packages/generic/release-cli/latest/release-cli-linux-amd64" ``` 1. Give it permissions to execute: diff --git a/doc/user/project/repository/branches/default.md b/doc/user/project/repository/branches/default.md index 3083ca5da3c..d31c64f4640 100644 --- a/doc/user/project/repository/branches/default.md +++ b/doc/user/project/repository/branches/default.md @@ -64,7 +64,7 @@ GitLab [administrators](../../../permissions.md) of self-managed instances can customize the initial branch for projects hosted on that instance. Individual groups and subgroups can override this instance-wide setting for their projects. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Default initial branch name**. 1. Change the default initial branch to a custom name of your choice. @@ -115,7 +115,7 @@ you must either: Administrators of self-managed instances can customize the initial default branch protection for projects hosted on that instance. Individual groups and subgroups can override this instance-wide setting for their projects. -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Default branch**. 1. Select [**Initial default branch protection**](#protect-initial-default-branches). @@ -132,7 +132,7 @@ can be overridden on a per-group basis by the group's owner. In [GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can disable this privilege for group owners, enforcing the instance-level protection rule: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository**. 1. Expand the **Default branch** section. 1. Clear the **Allow owners to manage default branch protection per group** checkbox. @@ -152,7 +152,7 @@ can be overridden on a per-group basis by the group's owner. In [enforce protection of initial default branches](#prevent-overrides-of-default-branch-protection) which locks this setting for group owners. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Default branch**. 1. Select [**Initial default branch protection**](#protect-initial-default-branches). diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index 6da2e5fc7ee..5e5a42a061b 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -59,7 +59,8 @@ To compare branches in a repository:  This feature allows merged branches to be deleted in bulk. Only branches that -have been merged and [are not protected](../../protected_branches.md) are deleted as part of +have been merged into the project's default branch and +[are not protected](../../protected_branches.md) are deleted as part of this operation. It's particularly useful to clean up old branches that were not deleted diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index b2a6c8848ce..d307a6a8580 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -193,10 +193,10 @@ you can sign individual commits manually, or configure Git to default to signed You can review commits for a merge request, or for an entire project: 1. To review commits for a project: - 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Commits**. 1. To review commits for a merge request: - 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests**, then select your merge request. 1. Select **Commits**. 1. Identify the commit you want to review. Signed commits show either a **Verified** diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 8e1286548b9..4926cf3812e 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -49,7 +49,7 @@ to a branch in the repository. When you use the command line, you can commit mul on their respective thread. - **Cherry-pick a commit:** In GitLab, you can - [cherry-pick a commit](../merge_requests/cherry_pick_changes.md#cherry-pick-a-commit) + [cherry-pick a commit](../merge_requests/cherry_pick_changes.md#cherry-pick-a-single-commit) from the UI. - **Revert a commit:** [Revert a commit](../merge_requests/revert_changes.md#revert-a-commit) @@ -201,6 +201,14 @@ To render an OpenAPI file: ## Repository size +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368150) in GitLab 15.3, feature flags `gitaly_revlist_for_repo_size` and `gitaly_catfile_repo_size` for alternative repository size calculations. + +FLAG: +On self-managed GitLab, by default GitLab uses the `du -sk` command to determine the size of a repository. GitLab can use either +`git-rev-list` (enabled with feature flag `gitaly_revlist_for_repo_size`) or `git-cat-file` (enabled with feature flag +`gitaly_catfile_repo_size`) instead. To switch between different calculation methods, ask an administrator to +[enable or disable](../../../administration/feature_flags.md) these feature flags. + The **Project information** page shows the size of all files in the repository. The size is updated, at most, every 15 minutes. The file size includes repository files, artifacts, and LFS. diff --git a/doc/user/project/repository/managing_large_repositories.md b/doc/user/project/repository/managing_large_repositories.md index ba425ae3dc7..d2ca49c118c 100644 --- a/doc/user/project/repository/managing_large_repositories.md +++ b/doc/user/project/repository/managing_large_repositories.md @@ -10,13 +10,13 @@ description: "Documentation on large repositories." GitLab, like any Git based system, is subject to similar performance restraints when it comes to large repositories that size into the gigabytes. -On this page we detail several best practices to improve performance with these large repositories on GitLab. +In the following sections, we detail several best practices for improving performance with these large repositories on GitLab. ## Large File System (LFS) -It's *strongly* recommended in any Git system that binary or blob files (for example, packages, audio, video, graphics, etc.) are stored as Large File Storage (LFS) objects. In such setup, the Objects are stored elsewhere, such as in Object Storage, and this can reduce the repository size significantly, thus improving performance. +It's *strongly* recommended in any Git system that binary or blob files (for example, packages, audio, video, or graphics) are stored as Large File Storage (LFS) objects. In such setup, the Objects are stored elsewhere, such as in Object Storage, and this can reduce the repository size significantly, thus improving performance. -To analyze if the repository has these sorts of objects, it's recommended to run [`git-sizer`](https://github.com/github/git-sizer) to get a detailed analysis. This tool shows in detail what makes up the repository as well as highlights any areas of concern. +To analyze if the repository has these sorts of objects, it's recommended to run a tool like [`git-sizer`](https://github.com/github/git-sizer) to get a detailed analysis. These tools can show in detail what makes up the repository as well as highlights any areas of concern. If any large objects are found, it's then recommended removing them with tools such as [`git filter-repo`](reducing_the_repo_size_using_git.md). Refer to the [Git LFS documentation for more information](../../../topics/git/lfs/index.md). diff --git a/doc/user/project/repository/mirror/bidirectional.md b/doc/user/project/repository/mirror/bidirectional.md index 340d7b48a47..793ca2a5f1f 100644 --- a/doc/user/project/repository/mirror/bidirectional.md +++ b/doc/user/project/repository/mirror/bidirectional.md @@ -45,7 +45,7 @@ and [pull](pull.md#pull-from-a-remote-repository) mirrors in the upstream GitLab To create the webhook in the downstream instance: 1. Create a [personal access token](../../../profile/personal_access_tokens.md) with `API` scope. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Webhooks**. 1. Add the webhook **URL**, which (in this case) uses the [Pull Mirror API](../../../../api/projects.md#start-the-pull-mirroring-process-for-a-project) diff --git a/doc/user/project/repository/mirror/index.md b/doc/user/project/repository/mirror/index.md index b08530c34b3..176461aeba7 100644 --- a/doc/user/project/repository/mirror/index.md +++ b/doc/user/project/repository/mirror/index.md @@ -41,7 +41,7 @@ Prerequisite: - If your mirror connects with `ssh://`, the host key must be detectable on the server, or you must have a local copy of the key. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Mirroring repositories**. 1. Enter a **Git repository URL**. For security reasons, the URL to the original @@ -89,7 +89,7 @@ Prerequisite: - You must have at least the Maintainer role for the project. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Mirroring repositories**. 1. Scroll to **Mirrored repositories** and identify the mirror to update. @@ -141,7 +141,7 @@ When you mirror a repository and select the **SSH public key** as your authentication method, GitLab generates a public key for you. The non-GitLab server needs this key to establish trust with your GitLab repository. To copy your SSH public key: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Mirroring repositories**. 1. Scroll to **Mirrored repositories**. @@ -249,7 +249,7 @@ If you receive this error after creating a new project using Check if the repository owner is specified in the URL of your mirrored repository: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Mirroring repositories**. 1. If no repository owner is specified, delete and add the URL again in this format, diff --git a/doc/user/project/repository/mirror/pull.md b/doc/user/project/repository/mirror/pull.md index d0f2b9a8088..159580dcfa5 100644 --- a/doc/user/project/repository/mirror/pull.md +++ b/doc/user/project/repository/mirror/pull.md @@ -61,7 +61,7 @@ Prerequisite: with the `repo` scope. If 2FA is enabled, this personal access token serves as your GitHub password. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Mirroring repositories**. 1. Enter the **Git repository URL**. Include the username diff --git a/doc/user/project/repository/mirror/push.md b/doc/user/project/repository/mirror/push.md index c00ebf415c9..10bdc54ecee 100644 --- a/doc/user/project/repository/mirror/push.md +++ b/doc/user/project/repository/mirror/push.md @@ -33,7 +33,7 @@ section. To set up push mirroring for an existing project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Mirroring repositories**. 1. Enter a repository URL. diff --git a/doc/user/project/repository/push_rules.md b/doc/user/project/repository/push_rules.md index 46a9585604e..90d2fdb89d0 100644 --- a/doc/user/project/repository/push_rules.md +++ b/doc/user/project/repository/push_rules.md @@ -36,7 +36,7 @@ Prerequisite: To create global push rules: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Push Rules**. 1. Expand **Push rules**. 1. Set the rule you want. @@ -48,7 +48,7 @@ The push rule of an individual project overrides the global push rule. To override global push rules for a specific project, or to update the rules for an existing project to match new global push rules: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Push rules**. 1. Set the rule you want. diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 344c288b607..f209c7ef137 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -46,7 +46,7 @@ To purge files from a GitLab repository: [`git-sizer`](https://github.com/github/git-sizer#getting-started) using a supported package manager or from source. -1. Generate a fresh +1. Generate a fresh [export from the project](../settings/import_export.md#export-a-project-and-its-data) and download it. This project export contains a backup copy of your repository *and* refs we can use to purge files from your repository. diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index f32035102bb..bafa2005fdf 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -150,7 +150,7 @@ The templates are inherited. For example, in a project, you can also access temp To use a custom description template with Service Desk: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. [Create a description template](description_templates.md#create-an-issue-template). 1. On the left sidebar, select **Settings > General > Service Desk**. 1. From the dropdown list **Template to append to all Service Desk issues**, search or select your template. @@ -164,7 +164,7 @@ this name in the `From` header. The default display name is `GitLab Support Bot` To edit the custom email display name: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General > Service Desk**. 1. Enter a new name in **Email display name**. 1. Select **Save Changes**. @@ -358,9 +358,23 @@ to everyone who can view the project. Behind the scenes, Service Desk works by the special Support Bot user creating issues. This user does not count toward the license limit count. +### Moving a Service Desk issue + +Service Desk issues can be moved like any other issue in GitLab. + +You can move a Service Desk issue the same way you +[move a regular issue](issues/managing_issues.md#move-an-issue) in GitLab. + +If a Service Desk issue is moved to a different project the customer who created the issue stops receiving emails. + ## Troubleshooting Service Desk ### Emails to Service Desk do not create issues Your emails might be ignored because they contain one of the [email headers that GitLab ignores](../../administration/incoming_email.md#rejected-headers). + +### Responses to a Service Desk issue do not generate emails + +Your issue might have been moved to a different project. +Moved Service Desk issues do not retain email participants. diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 4eeb7c5ba83..375e4a62b86 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -43,7 +43,7 @@ Prerequisites: To export a project and its data, follow these steps: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. Select **Export project**. @@ -57,13 +57,34 @@ moved to your configured `uploads_directory`. Every 24 hours, a worker deletes t ### Items that are exported -The following items are exported: +The [`import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) +file lists the items exported and imported when migrating projects using file exports. View this file in the branch +for your version of GitLab to see the list of items relevant to you. For example, +[`import_export.yml` on the `14-10-stable-ee` branch](https://gitlab.com/gitlab-org/gitlab/-/blob/14-10-stable-ee/lib/gitlab/import_export/project/import_export.yml). + +Migrating projects with file exports uses the same export and import mechanisms as creating projects from templates at the [group](../../group/custom_project_templates.md) and +[instance](../../admin_area/custom_project_templates.md) levels. Therefore, the list of exported items is the same. + +Items that are exported include: - Project and wiki repositories - Project uploads - Project configuration, excluding integrations -- Issues with comments, merge requests with diffs and comments, labels, milestones, snippets, time - tracking, and other project entities +- Issues + - Issue comments + - Issue resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) +- Merge requests + - Merge request diffs + - Merge request comments + - Merge request resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Merge request multiple assignees ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request reviewers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request approvers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) +- Labels +- Milestones +- Snippets +- Time tracking and other project entities - Design Management files and data - LFS objects - Issue boards @@ -73,7 +94,7 @@ The following items are exported: - Group members are exported as project members, as long as the user has the Maintainer role in the exported project's group, or is an administrator -The following items are **not** exported: +Items that are **not** exported include: - [Child pipeline history](https://gitlab.com/gitlab-org/gitlab/-/issues/221088) - Build traces and artifacts @@ -82,20 +103,11 @@ The following items are **not** exported: - Pipeline triggers - Webhooks - Any encrypted tokens -- Merge Request Approvers and [the number of required approvals](https://gitlab.com/gitlab-org/gitlab/-/issues/221088) +- [Number of required approvals](https://gitlab.com/gitlab-org/gitlab/-/issues/221088) - Repository size limits - Deploy keys allowed to push to protected branches - Secure Files -These content rules also apply to creating projects from templates on the -[group](../../group/custom_project_templates.md) -or [instance](../../admin_area/custom_project_templates.md) -levels, because the same export and import mechanisms are used. - -NOTE: -For more details on the specific data persisted in a project export, see the -[`import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) file. - ## Import a project and its data > Default maximum import file size [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. @@ -166,7 +178,8 @@ Imported users can be mapped by their public email addresses on self-managed ins - Public email addresses are not set by default. Users must [set it in their profiles](../../profile/index.md#set-your-public-email) for mapping to work correctly. - For contributions to be mapped correctly, users must be an existing member of the namespace, - or they can be added as a member of the project. Otherwise, a supplementary comment is left to mention that the original author and the MRs, notes, or issues that are owned by the importer. + or they can be added as a member of the project. Otherwise, a supplementary comment is left to mention that the original + author and the merge requests, notes, or issues that are owned by the importer. - Imported users are set as [direct members](../members/index.md) in the imported project. @@ -403,5 +416,5 @@ Error adding importer user to Project members. Validation failed: User project bots cannot be added to other groups / projects ``` -To use [Import REST APIs](../../../api/project_import_export.md), +To use [Import REST API](../../../api/project_import_export.md), pass regular user account credentials such as [personal access tokens](../../profile/personal_access_tokens.md). diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index b973a0f56d1..5c2118e02cf 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -13,7 +13,7 @@ Use the **Settings** page to manage the configuration options in your [project]( You must have at least the Maintainer role to view project settings. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. To display all settings in a section, select **Expand**. 1. Optional. Use the search box to find a setting. @@ -23,7 +23,7 @@ You must have at least the Maintainer role to view project settings. Use the project general settings to edit your project details. 1. Sign in to GitLab with at least the Maintainer role. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. In the **Project name** text box, enter your project name. 1. In the **Project description** text box, enter your project description. @@ -35,7 +35,7 @@ Use topics to categorize projects and find similar new projects. To assign topics to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings** > **General**. 1. In the **Topics** text box, enter the project topics. Popular topics are suggested as you type. 1. Select **Save changes**. @@ -51,7 +51,7 @@ requirements or needs additional oversight. The label can optionally apply Group owners can create, edit, and delete compliance frameworks: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings** > **General**. 1. Expand the **Compliance frameworks** section. @@ -170,7 +170,7 @@ include: # Execute individual project's configuration (if project contains .git When used to enforce scan execution, this feature has some overlap with [scan execution policies](../../application_security/policies/scan-execution-policies.md), as we have not [unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312). For details on the similarities and differences between these features, see -[Enforce scan execution](../../application_security/#enforce-scan-execution). +[Enforce scan execution](../../application_security/index.md#enforce-scan-execution). ### Ensure compliance jobs are always run @@ -214,10 +214,10 @@ Compliance pipelines start on the run of _every_ pipeline in a relevant project. triggers a child pipeline, the compliance pipeline runs first. This can trigger the parent pipeline, instead of the child pipeline. Therefore, in projects with compliance frameworks, we recommend replacing -[parent-child pipelines](../../../ci/pipelines/parent_child_pipelines.md) with the following: +[parent-child pipelines](../../../ci/pipelines/downstream_pipelines.md#parent-child-pipelines) with the following: - Direct [`include`](../../../ci/yaml/index.md#include) statements that provide the parent pipeline with child pipeline configuration. -- Child pipelines placed in another project that are run using the [trigger API](../../../ci/triggers/) rather than the parent-child +- Child pipelines placed in another project that are run using the [trigger API](../../../ci/triggers/index.md) rather than the parent-child pipeline feature. This alternative ensures the compliance pipeline does not re-start the parent pipeline. @@ -226,7 +226,7 @@ This alternative ensures the compliance pipeline does not re-start the parent pi To configure visibility, features, and permissions for a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility, project features, permissions** section. 1. To change the project visibility, select the dropdown list. If you select to **Public**, you limit access to some features to **Only Project Members**. @@ -241,17 +241,17 @@ Use the toggles to enable or disable features in the project. | Option | More access limit options | Description | |:---------------------------------|:--------------------------|:--------------| | **Issues** | ✓ | Activates the GitLab issues tracker. | -| **Repository** | ✓ | Enables [repository](../repository/) functionality | -| **Merge requests** | ✓ | Enables [merge request](../merge_requests/) functionality; also see [Merge request settings](#configure-merge-request-settings-for-a-project). | +| **Repository** | ✓ | Enables [repository](../repository/index.md) functionality | +| **Merge requests** | ✓ | Enables [merge request](../merge_requests/index.md) functionality; also see [Merge request settings](#configure-merge-request-settings-for-a-project). | | **Forks** | ✓ | Enables [forking](../repository/forking_workflow.md) functionality. | | **Git Large File Storage (LFS)** | | Enables the use of [large files](../../../topics/git/lfs/index.md#git-large-file-storage-lfs). | | **Packages** | | Supports configuration of a [package registry](../../../administration/packages/index.md#gitlab-package-registry-administration) functionality. | | **CI/CD** | ✓ | Enables [CI/CD](../../../ci/index.md) functionality. | -| **Container Registry** | | Activates a [registry](../../packages/container_registry/) for your Docker images. | -| **Analytics** | ✓ | Enables [analytics](../../analytics/). | +| **Container Registry** | | Activates a [registry](../../packages/container_registry/index.md) for your Docker images. | +| **Analytics** | ✓ | Enables [analytics](../../analytics/index.md). | | **Requirements** | ✓ | Control access to [Requirements Management](../requirements/index.md). | | **Security & Compliance** | ✓ | Control access to [security features](../../application_security/index.md). | -| **Wiki** | ✓ | Enables a separate system for [documentation](../wiki/). | +| **Wiki** | ✓ | Enables a separate system for [documentation](../wiki/index.md). | | **Snippets** | ✓ | Enables [sharing of code and text](../../snippets.md). | | **Pages** | ✓ | Allows you to [publish static websites](../pages/index.md). | | **Operations** | ✓ | Control access to Operations-related features, including [Operations Dashboard](../../../operations/index.md), [Environments and Deployments](../../../ci/environments/index.md), [Feature Flags](../../../operations/feature_flags.md). | @@ -286,7 +286,7 @@ In some environments, users can submit a [CVE identifier request](../../applicat To disable the CVE identifier request option in issues in your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility, project features, permissions** section. 1. Under **Issues**, turn off the **CVE ID requests in the issue sidebar** toggle. @@ -298,7 +298,7 @@ Prerequisites: - You must be an Owner of the project to disable email notifications related to the project. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility, project features, permissions** section. 1. Clear the **Disable email notifications** checkbox. @@ -339,7 +339,7 @@ other features are read-only. Archived projects are also hidden from project lis To archive a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. In the **Archive project** section, select **Archive project**. @@ -355,7 +355,7 @@ Prerequisites: - To unarchive a project, you must be an administrator or a project Owner. 1. Find the archived project. - 1. On the top bar, select **Menu > Project**. + 1. On the top bar, select **Main menu > Projects > View all projects**. 1. Select **Explore projects**. 1. In the **Sort projects** dropdown list, select **Show archived projects**. 1. In the **Filter by name** field, enter the project name. @@ -380,7 +380,7 @@ When you change the repository path, users may experience issues if they push to To rename a repository: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Advanced** section. 1. In the **Change path** text box, edit the path. @@ -402,7 +402,7 @@ Prerequisites: To transfer a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. Under **Transfer project**, choose the namespace to transfer the project to. @@ -420,7 +420,7 @@ to move any project to any namespace. When you transfer a project from a namespace licensed for GitLab SaaS Premium or Ultimate to GitLab Free, the following paid feature data is deleted: - [Project access tokens](../../../user/project/settings/project_access_tokens.md) are revoked -- [Pipeline subscriptions](../../../ci/pipelines/multi_project_pipelines.md#trigger-a-pipeline-when-an-upstream-project-is-rebuilt) +- [Pipeline subscriptions](../../../ci/pipelines/index.md#trigger-a-pipeline-when-an-upstream-project-is-rebuilt) and [test cases](../../../ci/test_cases/index.md) are deleted. ## Delete a project @@ -433,7 +433,7 @@ Prerequisite: To delete a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. In the "Delete project" section, select **Delete project**. @@ -472,7 +472,7 @@ Prerequisites: To immediately delete a project marked for deletion: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. In the "Permanently delete project" section, select **Delete project**. @@ -504,7 +504,7 @@ To restore the fork relationship, [use the API](../../../api/projects.md#create- To remove a fork relationship: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. In the **Remove fork relationship** section, select **Remove fork relationship**. diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index 77a53874777..c9c5efce9b1 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -27,6 +27,13 @@ and [personal access tokens](../../profile/personal_access_tokens.md). In self-managed instances, project access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. +WARNING: +The ability to create project access tokens without expiry was +[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and is planned for removal in GitLab +16.0. When this ability is removed, existing project access tokens without an expiry are planned to have an expiry added. +The automatic adding of an expiry occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry +occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. + You can use project access tokens: - On GitLab SaaS if you have the Premium license tier or higher. Project access tokens are not available with a [trial license](https://about.gitlab.com/free-trial/). @@ -43,11 +50,12 @@ configured for personal access tokens. ## Create a project access token -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89114) in GitLab 15.1, Owners can select Owner role for project access tokens. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89114) in GitLab 15.1, Owners can select Owner role for project access tokens. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348660) in GitLab 15.3, default expiration of 30 days and default role of Guest is populated in the UI. To create a project access token: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Enter a name. The token name is visible to any user with permissions to view the project. 1. Optional. Enter an expiry date for the token. The token expires on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. @@ -62,7 +70,7 @@ A project access token is displayed. Save the project access token somewhere saf To revoke a project access token: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Next to the project access token to revoke, select **Revoke**. @@ -85,7 +93,7 @@ The scope determines the actions you can perform when you authenticate with a pr To enable or disable project access token creation for all projects in a top-level group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. 1. Under **Permissions**, turn on or off **Allow project and group access token creation**. diff --git a/doc/user/project/time_tracking.md b/doc/user/project/time_tracking.md index 971ecf66a3c..522ec962e53 100644 --- a/doc/user/project/time_tracking.md +++ b/doc/user/project/time_tracking.md @@ -170,7 +170,7 @@ The following time units are available: In GitLab self-managed instances, you can limit the display of time units to hours. To do so: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. 1. Expand **Localization**. 1. Under **Time tracking**, select the **Limit display of time tracking units to hours** checkbox. diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 5a4e300a210..2a197c733cf 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -260,7 +260,7 @@ a `main` entry point inside the Web IDE. Live Preview is enabled for all projects on GitLab.com. If you are an administrator of a self-managed GitLab instance, and you want to enable Live Preview: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Scroll to **Web IDE** and select **Expand**:  diff --git a/doc/user/project/wiki/group.md b/doc/user/project/wiki/group.md index a3ba5789d39..03838a62d59 100644 --- a/doc/user/project/wiki/group.md +++ b/doc/user/project/wiki/group.md @@ -29,7 +29,7 @@ and higher can edit group wikis. Group wiki repositories can be moved using the To access a group wiki: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. To display the wiki, either: - On the left sidebar, select **Wiki**. - On any page in the project, use the <kbd>g</kbd> + <kbd>w</kbd> @@ -69,7 +69,7 @@ can enable or disable a group wiki through the group settings. To open group settings: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. 1. Scroll to **Wiki** and select one of these options: diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index c7f675417bb..b8924c33b13 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -31,7 +31,7 @@ with sibling pages listed in alphabetical order. To view a list of all pages, se To access a project wiki: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. To display the wiki, either: - On the left sidebar, select **Wiki**. - On any page in the project, use the <kbd>g</kbd> + <kbd>w</kbd> @@ -61,7 +61,7 @@ When a wiki is created, it is empty. On your first visit, you can create the home page users see when viewing the wiki. This page requires a specific title to be used as your wiki's home page. To create it: -1. On the top bar, select **Menu**. +1. On the top bar, select **Main menu**. - For project wikis, select **Projects** and find your project. - For group wikis, select **Groups** and find your group. 1. On the left sidebar, select **Wiki**. @@ -79,7 +79,7 @@ to be used as your wiki's home page. To create it: Users with at least the Developer role can create new wiki pages: -1. On the top bar, select **Menu**. +1. On the top bar, select **Main menu**. - For project wikis, select **Projects** and find your project. - For group wikis, select **Groups** and find your group. 1. On the left sidebar, select **Wiki**. @@ -142,7 +142,7 @@ may not be able to check out the wiki locally afterward. You need at least the Developer role to edit a wiki page: -1. On the top bar, select **Menu**. +1. On the top bar, select **Main menu**. - For project wikis, select **Projects** and find your project. - For group wikis, select **Groups** and find your group. 1. On the left sidebar, select **Wiki**. @@ -161,7 +161,7 @@ For an example, read [Table of contents](../../markdown.md#table-of-contents). You need at least the Developer role to delete a wiki page: -1. On the top bar, select **Menu**. +1. On the top bar, select **Main menu**. - For project wikis, select **Projects** and find your project. - For group wikis, select **Groups** and find your group. 1. On the left sidebar, select **Wiki**. @@ -174,7 +174,7 @@ You need at least the Developer role to delete a wiki page: You need at least the Developer role to move a wiki page: -1. On the top bar, select **Menu**. +1. On the top bar, select **Main menu**. - For project wikis, select **Projects** and find your project. - For group wikis, select **Groups** and find your group. 1. On the left sidebar, select **Wiki**. @@ -200,7 +200,7 @@ The history page shows: To view the changes for a wiki page: -1. On the top bar, select **Menu**. +1. On the top bar, select **Main menu**. - For project wikis, select **Projects** and find your project. - For group wikis, select **Groups** and find your group. 1. On the left sidebar, select **Wiki**. @@ -213,7 +213,7 @@ To view the changes for a wiki page: You can see the changes made in a version of a wiki page, similar to versioned diff file views: -1. On the top bar, select **Menu**. +1. On the top bar, select **Main menu**. - For project wikis, select **Projects** and find your project. - For group wikis, select **Groups** and find your group. 1. On the left sidebar, select **Wiki**. @@ -246,7 +246,7 @@ You need at least the Developer role to customize the wiki navigation sidebar. This process creates a wiki page named `_sidebar` which fully replaces the default sidebar navigation: -1. On the top bar, select **Menu**. +1. On the top bar, select **Main menu**. - For project wikis, select **Projects** and find your project. - For group wikis, select **Groups** and find your group. 1. On the left sidebar, select **Wiki**. @@ -284,7 +284,7 @@ You can disable group wikis from the [group settings](group.md#configure-group-w To add a link to an external wiki from a project's left sidebar: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **External wiki**. 1. Add the URL to your external wiki. @@ -300,7 +300,7 @@ To hide the internal wiki from the sidebar, [disable the project's wiki](#disabl To hide the link to an external wiki: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Integrations**. 1. Select **External wiki**. 1. In the **Enable integration** section, clear the **Active** checkbox. @@ -310,7 +310,7 @@ To hide the link to an external wiki: To disable a project's internal wiki: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Go to your project and select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. Scroll down to find **Wiki** and toggle it off (in gray). diff --git a/doc/user/project/working_with_projects.md b/doc/user/project/working_with_projects.md index 2501fa8b45c..e58bf5aa557 100644 --- a/doc/user/project/working_with_projects.md +++ b/doc/user/project/working_with_projects.md @@ -11,22 +11,9 @@ code are saved in projects, and most features are in the scope of projects. ## View projects -To explore projects: +To view projects, on the top bar, select **Main menu > Projects > View all projects**. -1. On the top bar, select **Menu > Projects**. -1. Select **Explore projects**. - -The **Projects** page shows a list of projects, sorted by last updated date. - -- To view projects with the most [stars](#star-a-project), select **Most stars**. -- To view projects with the largest number of comments in the past month, select **Trending**. - -NOTE: -The **Explore projects** tab is visible to unauthenticated users unless the -[**Public** visibility level](../admin_area/settings/visibility_and_access_controls.md#restrict-visibility-levels) -is restricted. Then the tab is visible only to signed-in users. - -### Who can view the **Projects** page +### Who can view the Projects page When you select a project, the project landing page shows the project contents. @@ -53,11 +40,16 @@ visit the `/projects/:id` URL in your browser or other tool accessing the projec To explore project topics: -1. On the top bar, select **Menu > Projects**. -1. Select **Explore topics**. +1. On the top bar, select **Main menu > Projects > View all projects**. +1. Select the **Explore topics** tab. +1. To view projects associated with a topic, select a topic. + +The **Explore topics** tab shows a list of topics sorted by the number of associated projects. -The **Projects** page shows list of topics sorted by the number of associated projects. -To view projects associated with a topic, select a topic from the list. +NOTE: +The **Explore projects** tab is visible to unauthenticated users unless the +[**Public** visibility level](../admin_area/settings/visibility_and_access_controls.md#restrict-visibility-levels) +is restricted. Then the tab is visible only to signed-in users. You can assign topics to a project on the [Project Settings page](settings/index.md#assign-topics-to-a-project). @@ -68,8 +60,9 @@ If you're an instance administrator, you can administer all project topics from To create a project in GitLab: -1. On the top bar, select **Menu > Project > Create new project**. -1. On the **Create new project** page, choose if you want to: +1. On the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. +1. Select an option: - Create a [blank project](#create-a-blank-project). - Create a project from a: - [built-in template](#create-a-project-from-a-built-in-template). @@ -88,7 +81,8 @@ To create a project in GitLab: To create a blank project: -1. On the top bar, select **Menu > Projects > Create new project**. +1. On the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Create blank project**. 1. Enter the project details: - In the **Project name** field, enter the name of your project. You cannot use special characters at @@ -119,7 +113,8 @@ Anyone can [contribute a built-in template](../../development/project_templates. To create a project from a built-in template: -1. On the top bar, select **Menu > Projects > Create new project**. +1. On the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Create from template**. 1. Select the **Built-in** tab. 1. From the list of templates: @@ -145,7 +140,8 @@ Custom project templates are available at: - The [instance-level](../../user/admin_area/custom_project_templates.md) - The [group-level](../../user/group/custom_project_templates.md) -1. On the top bar, select **Menu > Projects > Create new project**. +1. On the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Create from template**. 1. Select the **Instance** or **Group** tab. 1. From the list of templates: @@ -171,7 +167,8 @@ HIPAA Audit Protocol published by the U.S Department of Health and Human Service To create a project from the HIPAA Audit Protocol template: -1. On the top bar, select **Menu > Projects > Create new project**. +1. On the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Create from template**. 1. Select the **Built-in** tab. 1. Locate the **HIPAA Audit Protocol** template: @@ -210,9 +207,7 @@ Prerequisites: [added to your GitLab account](../ssh.md#add-an-ssh-key-to-your-gitlab-account). - You must have permission to add new projects to a namespace. To check if you have permission: - 1. On the top bar, select **Menu > Projects**. - 1. Select **Groups**. - 1. Select a group. + 1. On the top bar, select **Main menu > Groups** and find your group. 1. Confirm that **New project** is visible in the upper right corner. Contact your GitLab administrator if you require permission. @@ -258,15 +253,13 @@ You can add a star to projects you use frequently to make them easier to find. To add a star to a project: -1. On the top bar, select **Menu > Projects**. -1. Select **Your projects** or **Explore projects**. -1. Select a project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. In the upper right corner of the page, select **Star**. ## View starred projects -1. On the top bar, select **Menu > Projects**. -1. Select **Starred projects**. +1. On the top bar, select **Main menu > Projects > View all projects**. +1. Select the **Starred projects** tab. 1. GitLab displays information about your starred projects, including: - Project description, including name, description, and icon. @@ -284,8 +277,8 @@ called `my-project` under your username, the project is created at `https://gitl To view your personal projects: -1. On the top bar, select **Menu > Projects > Your Projects**. -1. Under **Your projects**, select **Personal**. +1. On the top bar, select **Main menu > Projects > View all projects**. +1. In the **Your projects** tab, select **Personal**. ## Delete a project @@ -294,9 +287,7 @@ you can [enable delayed project removal](../group/manage.md#enable-delayed-proje To delete a project: -1. On the top bar, select **Menu > Projects**. -1. Select **Your projects** or **Explore projects**. -1. Select a project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Select **Settings > General**. 1. Expand the **Advanced** section. 1. Scroll down to the **Delete project** section. @@ -315,7 +306,7 @@ projects within that group are not deleted immediately, but only after a delay. To view a list of all projects that are pending deletion: -1. On the top bar, select **Menu > Projects > Explore projects**. +1. On the top bar, select **Main menu > Projects > View all projects**. 1. Based on your GitLab version: - GitLab 14.6 and later: select the **Pending deletion** tab. - GitLab 14.5 and earlier: select the **Deleted projects** tab. @@ -330,9 +321,7 @@ Each project in the list shows: To view the activity of a project: -1. On the top bar, select **Menu > Projects**. -1. Select **Your projects** or **Explore projects**. -1. Select a project. +1. On the top bar, select **Main menu > Projects** and find your project.. 1. On the left sidebar, select **Project information > Activity**. 1. Select a tab to view the type of project activity. @@ -340,7 +329,7 @@ To view the activity of a project: You can search through your projects. -1. On the top bar, select **Menu**. +1. On the top bar, select **Main menu**. 1. In **Search your projects**, type the project name. GitLab filters as you type. @@ -366,9 +355,7 @@ member and cannot contribute. To leave a project: -1. On the top bar, select **Menu > Projects**. -1. Select **Your projects** or **Explore projects**. -1. Select a project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. Select **Leave project**. The **Leave project** option only displays on the project dashboard when a project is part of a group under a [group namespace](../namespace/index.md). diff --git a/doc/user/public_access.md b/doc/user/public_access.md index d821c1abe47..3accb0f5e3d 100644 --- a/doc/user/public_access.md +++ b/doc/user/public_access.md @@ -24,6 +24,8 @@ Public projects can be cloned **without any** authentication over HTTPS. They are listed in the public access directory (`/public`) for all users. +Public groups can have public, internal, or private subgroups. + **Any signed-in user** has the Guest role on the repository. NOTE: @@ -38,6 +40,8 @@ Internal projects can be cloned by any signed-in user except They are also listed in the public access directory (`/public`), but only for signed-in users. +Internal groups can have internal or private subgroups. + Any signed-in users except [external users](permissions.md#external-users) have the Guest role on the repository. @@ -53,13 +57,15 @@ Private projects can only be cloned and viewed by project members (except for gu They appear in the public access directory (`/public`) for project members only. +Private groups can only have private subgroups. + ## Change project visibility Prerequisite: - You must have the Owner role for a project. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. Change **Project visibility** to either **Private**, **Internal**, or **Public**. @@ -71,9 +77,9 @@ Prerequisite: - You must have the Owner role for a group. - Subgroups and projects must already have visibility settings that are at least as - restrictive as the new setting for the group. + restrictive as the new setting of the parent group. -1. On the top bar, select **Menu > Groups** and find your project. +1. On the top bar, select **Main menu > Groups** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Naming, visibility**. 1. Under **Visibility level** select either **Private**, **Internal**, or **Public**. diff --git a/doc/user/search/advanced_search.md b/doc/user/search/advanced_search.md index 90d6a15901a..bec1a8f3d4c 100644 --- a/doc/user/search/advanced_search.md +++ b/doc/user/search/advanced_search.md @@ -5,60 +5,44 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference --- -# GitLab Advanced Search **(PREMIUM)** +# Advanced Search **(PREMIUM)** > Moved to GitLab Premium in 13.9. -Advanced Search uses Elasticsearch for faster, more advanced search across the entire -GitLab instance. +You can use Advanced Search for faster, more efficient search across the entire GitLab +instance. Advanced Search is based on Elasticsearch, a purpose-built full-text search +engine you can horizontally scale to get results in up to a second in most cases. -Use Advanced Search when searching in: +You can find code you want to update in all projects at once to save +maintenance time and promote innersourcing. + +You can use Advanced Search in: - Projects - Issues - Merge requests - Milestones - Users -- Epics (when searching in a group only) +- Epics (in groups only) - Code - Comments - Commits -- Wiki (except [group wikis](../project/wiki/group.md)) - -Advanced Search can be useful in various scenarios: - -- **Faster searches:** - Advanced Search is based on Elasticsearch, which is a purpose-built full - text search engine that can be horizontally scaled so that it can provide - search results in 1-2 seconds in most cases. -- **Code Maintenance:** - Finding all the code that needs to be updated at once across an entire - instance can save time spent maintaining code. - This is especially helpful for organizations with more than 10 active projects. - This can also help build confidence is code refactoring to identify unknown impacts. -- **Promote innersourcing:** - Your company may consist of many different developer teams each of which has - their own group where the various projects are hosted. Some of your applications - may be connected to each other, so your developers need to instantly search - throughout the GitLab instance and find the code they search for. - -## Configuring Advanced Search - -For self-managed GitLab instances, an administrator must -[configure Advanced Search](../../integration/advanced_search/elasticsearch.md). +- Project wikis (not [group wikis](../project/wiki/group.md)) -On GitLab.com, Advanced Search is enabled. +## Configure Advanced Search -## Advanced Search syntax +- On GitLab.com, Advanced Search is enabled for groups with paid subscriptions. +- For self-managed GitLab instances, an administrator must + [configure Advanced Search](../../integration/advanced_search/elasticsearch.md). -See the documentation on [Advanced Search syntax](global_search/advanced_search_syntax.md). +## Syntax -## Search by issue or merge request ID +See [Advanced Search syntax](global_search/advanced_search_syntax.md) for more information. -You can search a specific issue or merge request by its ID with a special prefix. +## Search by ID -- To search by issue ID, use prefix `#` followed by issue ID. For example, [#23456](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=%2323456&group_id=9970&project_id=278964) -- To search by merge request ID, use prefix `!` followed by merge request ID. For example [!23456](https://gitlab.com/search?snippets=&scope=merge_requests&repository_ref=&search=%2123456&group_id=9970&project_id=278964) +- To search by issue ID, use the `#` prefix followed by the issue ID (for example, [`#23456`](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=%2323456&group_id=9970&project_id=278964)). +- To search by merge request ID, use the `!` prefix followed by the merge request ID (for example, [`!23456`](https://gitlab.com/search?snippets=&scope=merge_requests&repository_ref=&search=%2123456&group_id=9970&project_id=278964)). ## Global search scopes **(FREE SELF)** diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 76a85b55585..41eff28b088 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -44,7 +44,7 @@ search, or choose a specific group or project. To search through code or other documents in a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the top bar, in the search field, type the string you want to search for. 1. Press **Enter**. @@ -68,7 +68,7 @@ where the results were found. You can search for a commit SHA. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the top bar, in the search field, type the SHA. If a single result is returned, GitLab redirects to the commit result diff --git a/doc/user/snippets.md b/doc/user/snippets.md index cab18b221c1..f4683414dea 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -23,9 +23,15 @@ GitLab provides two types of snippets: - **Personal snippets**: Created independent of any project. You can set a [visibility level](public_access.md) - for your snippet: public, internal, or private. + for your snippet: public or private. - **Project snippets**: Always related to a specific project. - Project snippets can be visible publicly or to only group members. + Project snippets can be visible publicly, or to only project members. + +NOTE: +From July 2019, the `Internal` visibility setting is disabled for new projects, groups, +and snippets on GitLab.com. Existing snippets using the `Internal` +visibility setting keep this setting. You can read more about the change in the +[relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/12388). ## Create snippets @@ -61,10 +67,10 @@ In GitLab versions 13.0 and later, snippets are [versioned by default](#versione To discover all snippets visible to you in GitLab, you can: - **View all snippets visible to you**: On the top bar of your GitLab - instance, select **Menu > Snippets** to view your snippets dashboard. + instance, select **Main menu > Snippets** to view your snippets dashboard. - **Visit [GitLab snippets](https://gitlab.com/dashboard/snippets)** for your snippets on GitLab.com. - **Explore all public snippets**: On the top bar of your GitLab - instance, select **Menu > Snippets** and select **Explore snippets** to view + instance, select **Main menu > Snippets** and select **Explore snippets** to view [all public snippets](https://gitlab.com/explore/snippets). - **View a project's snippets**: In your project, go to **Snippets**. @@ -219,7 +225,7 @@ Prerequisites: To do this task: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Snippets**. 1. Select the snippet you want to report as spam. 1. Select **Submit as spam**. diff --git a/doc/user/ssh.md b/doc/user/ssh.md index 5667890757a..913140d2a01 100644 --- a/doc/user/ssh.md +++ b/doc/user/ssh.md @@ -254,6 +254,8 @@ A public and private key are generated. ## Add an SSH key to your GitLab account +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271239) in GitLab 15.4, default expiration date suggested in UI. + To use SSH with GitLab, copy your public key to your GitLab account: 1. Copy the contents of your public key file. You can do this manually or use a script. @@ -289,7 +291,7 @@ To use SSH with GitLab, copy your public key to your GitLab account: `ssh-ed25519`, `sk-ecdsa-sha2-nistp256@openssh.com`, or `sk-ssh-ed25519@openssh.com`, and may end with a comment. 1. In the **Title** box, type a description, like `Work Laptop` or `Home Workstation`. -1. Optional. In the **Expires at** box, select an expiration date. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36243) in GitLab 12.9.) +1. Optional. Update **Expiration date** to modify the default expiration date. In: - GitLab 13.12 and earlier, the expiration date is informational only. It doesn't prevent you from using the key. Administrators can view expiration dates and use them for @@ -419,6 +421,9 @@ as both have a different home directory: You can either copy over the `.ssh/` directory to use the same key, or generate a key in each environment. +If you're running Windows 11 and using [OpenSSH for Windows](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview), ensure the `HOME` +environment variable is set correctly. Otherwise, your private SSH key might not be found. + Alternative tools include: - [Cygwin](https://www.cygwin.com) @@ -468,6 +473,7 @@ This indicates that something is wrong with your SSH setup. - Try to manually register your private SSH key by using `ssh-agent`. - Try to debug the connection by running `ssh -Tv git@example.com`. Replace `example.com` with your GitLab URL. +- Ensure you followed all the instructions in [Use SSH on Microsoft Windows](#use-ssh-on-microsoft-windows). ### `Could not resolve hostname` error diff --git a/doc/user/tasks.md b/doc/user/tasks.md index 16e5c85a354..15abc77ad47 100644 --- a/doc/user/tasks.md +++ b/doc/user/tasks.md @@ -10,13 +10,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Creating, editing, and deleting tasks](https://gitlab.com/groups/gitlab-org/-/epics/7169) introduced in GitLab 15.0. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/334812) in GitLab 15.3. -WARNING: -Tasks are in [**Alpha**](../policy/alpha-beta-support.md#alpha-features). - -The following list are the known limitations: +Known limitation: - [Tasks currently cannot be accessed via REST API.](https://gitlab.com/gitlab-org/gitlab/-/issues/368055) -- An issue's tasks can only currently be accessed via a reference within a description, comment, or direct URL (`.../-/work_items/[global_id]`). For the latest updates, check the [Tasks Roadmap](https://gitlab.com/groups/gitlab-org/-/epics/7103). @@ -40,7 +36,7 @@ to work items and adding custom work item types, visit ## View tasks -View tasks in issues, in the **Child items** section. +View tasks in issues, in the **Tasks** section. You can also [filter the list of issues](project/issues/managing_issues.md#filter-the-list-of-issues) for `Type = task`. @@ -53,7 +49,7 @@ Prerequisites: To create a task: -1. In an issue description, in the **Child items** section, select **Add a task**. +1. In the issue description, in the **Tasks** section, select **Add**. 1. Enter the task title. 1. Select **Create task**. @@ -65,7 +61,7 @@ Prerequisites: To edit a task: -1. In the issue description, in the **Child items** section, select the task you want to edit. +1. In the issue description, in the **Tasks** section, select the task you want to edit. The task window opens. 1. Optional. To edit the title, select it and make your changes. 1. Optional. To edit the description, select the edit icon (**{pencil}**), make your changes, and @@ -83,7 +79,7 @@ It's not possible to connect them again. To remove a task from an issue: -1. In the issue description, in the **Child items** section, next to the task you want to remove, select the options menu (**{ellipsis_v}**). +1. In the issue description, in the **Tasks** section, next to the task you want to remove, select the options menu (**{ellipsis_v}**). 1. Select **Remove task**. ## Delete a task @@ -96,6 +92,75 @@ Prerequisites: To delete a task: -1. In the issue description, in the **Child items** section, select the task you want to edit. +1. In the issue description, in the **Tasks** section, select the task you want to edit. 1. In the task window, in the options menu (**{ellipsis_v}**), select **Delete task**. 1. Select **OK**. + +## Assign users to a task + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334810) in GitLab 15.4. + +To show who is responsible for a task, you can assign users to it. + +Users on GitLab Free can assign one user per task. +Users on GitLab Premium and higher can assign multiple users to a single task. +See also [multiple assignees for issues](project/issues/multiple_assignees_for_issues.md). + +Prerequisites: + +- You must have at least the Reporter role for the project. + +To change the assignee on a task: + +1. In the issue description, in the **Tasks** section, select the title of the task you want to edit. + The task window opens. +1. Next to **Assignees**, select **Add assignees**. +1. From the dropdown list, select the user(s) to add as an assignee. +1. Select any area outside the dropdown list. + +## Set a start and due date + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/365399) in GitLab 15.4. + +You can set a [start and due date](project/issues/due_dates.md) on a task. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +You can set start and due dates on a task to show when work should begin and end. + +To set a due date: + +1. In the issue description, in the **Tasks** section, select the title of the task you want to edit. + The task window opens. +1. If the task already has a due date next to **Due date**, select it. Otherwise, select **Add due date**. +1. In the date picker, select the desired due date. + +To set a start date: + +1. In the issue description, in the **Tasks** section, select the title of the task you want to edit. + The task window opens. +1. If the task already has a start date next to **Start date**, select it. Otherwise, select **Add start date**. +1. In the date picker, select the desired due date. + + The due date must be the same or later than the start date. + If you select a start date to be later than the due date, the due date is then changed to the same day. + +## Set task weight **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362550) in GitLab 15.3. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +You can set weight on each task to show how much work it needs. +This value is visible only when you view a task. + +To set issue weight of a task: + +1. In the issue description, in the **Tasks** section, select the title of the task you want to edit. + The task window opens. +1. Next to **Weight**, enter a whole, positive number. +1. Select the close icon (**{close}**). diff --git a/doc/user/usage_quotas.md b/doc/user/usage_quotas.md index 5d78b4bb795..828c9a3c4b0 100644 --- a/doc/user/usage_quotas.md +++ b/doc/user/usage_quotas.md @@ -32,36 +32,36 @@ To prevent exceeding the namespace storage quota, you can: 1. Reduce storage consumption by following the suggestions in the [Manage Your Storage Usage](#manage-your-storage-usage) section of this page. 1. Apply for [GitLab for Education](https://about.gitlab.com/solutions/education/join/), [GitLab for Open Source](https://about.gitlab.com/solutions/open-source/join/), or [GitLab for Startups](https://about.gitlab.com/solutions/startups/) if you meet the eligibility requirements. -1. Consider using a [self-managed instance](../subscriptions/self_managed/) of GitLab which does not have these limits on the free tier. +1. Consider using a [self-managed instance](../subscriptions/self_managed/index.md) of GitLab which does not have these limits on the free tier. 1. [Purchase additional storage](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer) units at $60/year for 10GB of storage. 1. [Start a trial](https://about.gitlab.com/free-trial/) or [upgrade to GitLab Premium or Ultimate](https://about.gitlab.com/pricing) which include higher limits and features that enable growing teams to ship faster without sacrificing on quality. 1. [Talk to an expert](https://page.gitlab.com/usage_limits_help.html) to learn more about your options and ask questions. ### Namespace storage limit enforcement schedule -Storage limits for GitLab SaaS Free tier namespaces will not be enforced prior to 2022-10-19. Storage limits for GitLab SaaS Paid tier namespaces will not be enforced for prior to 2023-02-15. +Storage limits for GitLab SaaS Free tier namespaces will not be enforced prior to 2022-10-19. Storage limits for GitLab SaaS Paid tier namespaces will not be enforced for prior to 2023-02-15. Enforcement will not occur until all storage types are accurately measured, including deduplication of forks for [Git](https://gitlab.com/gitlab-org/gitlab/-/issues/371671) and [LFS](https://gitlab.com/gitlab-org/gitlab/-/issues/370242). -Impacted users are notified via email and in-app notifications at least 60 days prior to enforcement. +Impacted users are notified via email and in-app notifications at least 60 days prior to enforcement. ### Project storage limit -Namespaces on a GitLab SaaS **paid** tier (Premium and Ultimate) have a storage limit on their project repositories. -A project's repository has a storage quota of 10 GB. A namespace has either a namespace-level storage limit or a project-level storage limit, but not both. +Projects on GitLab SaaS have a 10GB storage limit on their Git repository and LFS storage. +Once namespace-level storage limits are enforced, the project limit will be removed. A namespace has either a namespace-level storage limit or a project-level storage limit, but not both. -- Paid tier namespaces have project-level storage limits enforced. -- Free tier namespaces have namespace-level storage limits. - -When a project's repository reaches the quota, the project is locked. You cannot push changes to a locked project. To monitor the size of each +When a project's repository and LFS reaches the quota, the project is locked. You cannot push changes to a locked project. To monitor the size of each repository in a namespace, including a breakdown for each project, you can -[view storage usage](#view-storage-usage). To allow a project's repository to exceed the free quota +[view storage usage](#view-storage-usage). To allow a project's repository and LFS to exceed the free quota you must purchase additional storage. For more details, see [Excess storage usage](#excess-storage-usage). ## View storage usage -You can view storage usage for your project or [namespace](../user/namespace/index.md). +Prerequisites: + +- To view storage usage for a project, you must have at least the Maintainer role for the project or Owner role for the namespace. +- To view storage usage for a namespace, you must have the Owner role for the namespace. 1. Go to your project or namespace: - - For a project, on the top bar, select **Menu > Projects** and find your project. + - For a project, on the top bar, select **Main menu > Projects** and find your project. - For a namespace, enter the URL in your browser's toolbar. 1. From the left sidebar, select **Settings > Usage Quotas**. 1. Select the **Storage** tab. @@ -87,20 +87,20 @@ The following storage usage statistics are available to a maintainer: ## Manage your storage usage -You can use several methods to manage and reduce your usage for some storage types. +To manage your storage, if you are a namespace Owner you can [purchase more storage for the namespace](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer). -For more information, see the following pages: +Depending on your role, you can also use the following methods to manage or reduce your storage: -- [Reduce package registry storage](packages/package_registry/reduce_package_registry_storage.md) -- [Reduce dependency proxy storage](packages/dependency_proxy/reduce_dependency_proxy_storage.md) -- [Reduce repository size](project/repository/reducing_the_repo_size_using_git.md) -- [Reduce container registry storage](packages/container_registry/reduce_container_registry_storage.md) -- [Reduce container registry data transfers](packages/container_registry/reduce_container_registry_data_transfer.md) -- [Reduce wiki repository size](../administration/wikis/index.md#reduce-wiki-repository-size) +- [Reduce package registry storage](packages/package_registry/reduce_package_registry_storage.md). +- [Reduce dependency proxy storage](packages/dependency_proxy/reduce_dependency_proxy_storage.md). +- [Reduce repository size](project/repository/reducing_the_repo_size_using_git.md). +- [Reduce container registry storage](packages/container_registry/reduce_container_registry_storage.md). +- [Reduce container registry data transfers](packages/container_registry/reduce_container_registry_data_transfer.md). +- [Reduce wiki repository size](../administration/wikis/index.md#reduce-wiki-repository-size). ## Excess storage usage -Excess storage usage is the amount that a project's repository exceeds the free storage quota. If no +Excess storage usage is the amount that a project's repository and LFS exceeds the free storage quota. If no purchased storage is available the project is locked. You cannot push changes to a locked project. To unlock a project you must [purchase more storage](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer) for the namespace. When the purchase is completed, locked projects are automatically unlocked. The @@ -115,7 +115,7 @@ The **Storage** tab of the **Usage Quotas** page warns you of the following: ### Excess storage example -The following example describes an excess storage scenario for namespace _Example Company_: +The following example describes an excess storage scenario for a namespace: | Repository | Storage used | Excess storage | Quota | Status | |------------|--------------|----------------|--------|-------------------| @@ -125,12 +125,12 @@ The following example describes an excess storage scenario for namespace _Exampl | Yellow | 2 GB | 0 GB | 10 GB | Not locked | | **Totals** | **30 GB** | **0 GB** | - | - | -The Red and Green projects are locked because their repositories have reached the quota. In this +The Red and Green projects are locked because their repositories and LFS have reached the quota. In this example, no additional storage has yet been purchased. To unlock the Red and Green projects, 50 GB additional storage is purchased. -Assuming the Green and Red projects' repositories grow past the 10 GB quota, the purchased storage +Assuming the Green and Red projects' repositories and LFS grow past the 10 GB quota, the purchased storage available decreases. All projects remain unlocked because 40 GB purchased storage is available: 50 GB (purchased storage) - 10 GB (total excess storage used). |
